So if you have 1000 messages all in the same message group, and you set the batch size to 1. The lambda will only ever pick up 1 message, complete before a new lambda function is invoked
Right, and if a message is not within a message group you are telling me it will process them concurrently. e.g if the batch size was 10, you will have 100 lambda functions be invoked (if there are 1000 messages on the queue).
What is the limitation on that concurrency?
Math.min(your lambda total allowed concurrency, number of messages in the queue)
Tbh im not sure i 100% need a FIFO queue, i guess main reason was because i could avoid handling duplicate messages
I think for my scenario with the rate limit on the api, i should limit the concurrency of the lambda function
That’s a terrible reason to use FIFO. unless 99% of your messages are duplicates. Otherwise, just set up idempotent processing, it’s far more efficient.
I think 5 lambda functions running concurrently receiving a batch of 10 messages each and making an api call sequentially for each message i should be fine
Definitely avoid FIFO queues unless you are absolutely certain you need them.
Well with standard queues i saw that the consumer would have a chance of receiving a duplicate mssage. It was not a concern of mine that the 1000 messages beging put onto the queue would have duplicates
• “At-least once delivery”
@wparad
:flag-ch:in FIFO you have to specify a message group id - https://docs.aws.amazon.com/sns/latest/dg/fifo-message-grouping.html
The group ID is a mandatory token that specifies that a message belongs to a specific message group
So im not sure if what you were mentioning earlier about concurrency applies to FIFO?
or am i interrupting that message incorrectly?
