1 year ago
#327792
tech-ss
Consuming multiple SQS messages in AWS Lambda for performing join operations
I'm currently working on a use case where I'm required to capture data using CDC approach and the data I'm receiving is through SQS only. The data change takes place in multiple tables and so multiple events are fired from SQS even for a single record change. Now I'm required to perform some operations on the data for which I have to perform joins on table data received through SQS. As there might be huge transactions going on simultaneously I cannot prefer to go with FIFO for SQS because of it's restriction of batch size as well it might make operations a bit slow.
Also I'm not sure whether I should go with an approach where I could store some cached data unless I receive the required data to perform join operations in AWS Elasticache.
Is there any alternative which I can give a try to?
amazon-web-services
aws-lambda
amazon-sqs
cdc
amazon-elasticache
0 Answers
Your Answer