1 year ago

#360504

test-img

Angelo Canesso

Stored procedure to batch update a table in postgresql

I am running an Extract Transform Load job, triggered every minute, that inserts or updates a few rows (1k) in a table T. Table T has 4.5 million rows.

A second Job needs to update a field with a stored function update_field(field) for every row of the table every hour. I thought of writing a stored procedure to update the table within the database server.

In order to avoid deadlocks with the ETL job, I need to batch the update task into several transactions to release the locks at the end of each transaction.

I tried something like this but cannot figure out the proper syntax to use FETCH FORWARD to iterate over the cursor in a batch fashion .. Is there a better solution ?

-- Procedure that is triggered every hour 
CREATE OR REPLACE PROCEDURE my_procedure() AS $$     
    DECLARE         
        T_curssor CURSOR FOR SELECT * FROM T;                 
        T_batch T[];
        batch_size INT := 100000;     
    BEGIN 
        OPEN T_curssor;                 
            LOOP;                     
                FETCH FORWARD batch_size FROM T_curssor INTO T_batch;           
                IF NOT FOUND THEN EXIT;
                PERFORM update_field(field) FROM T_batch;
                COMMIT;                 
            END LOOP;     
    END; 
$$ LANGUAGE plpgsql;

database

postgresql

stored-procedures

plpgsql

database-cursor

0 Answers

Your Answer

Accepted video resources