1 year ago
#360504
Angelo Canesso
Stored procedure to batch update a table in postgresql
I am running an Extract Transform Load job, triggered every minute, that inserts or updates a few rows (1k) in a table T. Table T has 4.5 million rows.
A second Job needs to update a field with a stored function update_field(field) for every row of the table every hour. I thought of writing a stored procedure to update the table within the database server.
In order to avoid deadlocks with the ETL job, I need to batch the update task into several transactions to release the locks at the end of each transaction.
I tried something like this but cannot figure out the proper syntax to use FETCH FORWARD to iterate over the cursor in a batch fashion .. Is there a better solution ?
-- Procedure that is triggered every hour
CREATE OR REPLACE PROCEDURE my_procedure() AS $$
DECLARE
T_curssor CURSOR FOR SELECT * FROM T;
T_batch T[];
batch_size INT := 100000;
BEGIN
OPEN T_curssor;
LOOP;
FETCH FORWARD batch_size FROM T_curssor INTO T_batch;
IF NOT FOUND THEN EXIT;
PERFORM update_field(field) FROM T_batch;
COMMIT;
END LOOP;
END;
$$ LANGUAGE plpgsql;
database
postgresql
stored-procedures
plpgsql
database-cursor
0 Answers
Your Answer