Lambda limits with AWS SQS FIFO?

DevOps AWS
#1

I have read AWS SQS FIFO only support batching messages of up to 10 at a time? Does that mean, my lambda function that is listening to the SQS can only receive a batch of 10 messages at a time?

#2

What are the two (or more) possibilities you are trying to decide between?

#3

Im not sure what youre trying to ask me?

#4

I just wnat to know if batch of 10 is a hard limit ( i saw something which mentioned high throughput fifo queues)

#5

SQS has a hard limit of 10 messages per batch, FIFO or not FIFO

#6

Okay so that means the consumer can only recieve 10 messages at a time?

#7

I just want to be clear, if i have a lambda function listening to the sqs it can only receive an event with 10 messages

#8

At most, yes.

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-client-side-buffering-request-batching.html

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues-understanding-logic.html

#9

Ok thats confusing becasue - https://www.serverless.com/framework/docs/providers/aws/events/sqs

The default batchSize is 10. The max batchSize is 10000 for a standard queue, 10 for a FIFO queue.

#10

I thought the 10 messages was only a limitation for FIFO

#11

NO, it means that one call to the API at max returns 10. You can make 1000 calls to the api and get back 10k results.

#12

> I just want to be clear, if i have a lambda function listening to the sqs it can only receive an event with 10 messages (edited)
Listening using an Lambda EventSource or calling receiveMessage?

#13

Ok so let me give you more context about my scneario…

#14

I have a scneario:

• Each day i have 1000 job (messages) taht will come into the queue
◦ for each job:
:black_small_square:︎ i need to make an api call to a third party
:black_small_square:︎ store the response data in a database
• There is a limitation, the api rate limits 5 requests per second.

If i use a FIFO queue and i receive 1 message at a time and lets say it takes 1 minute to complete the job. It would take 16.6 hours to complete (not great).

#15

The event source may spin up 1000 lambdas executions concurrently. That means 10k messages processed at the same tim. Each with 10.

#16

^ this will only occur for standard FIFO queues right?

becuase my assumption is that with a FIFO queue, the first message has to be completed first before it moves to the next

#17

It has nothing to do with FIFO queues, it is 100% based on your event source configuration. It just so happens that FIFO event sources can only set the batch size at 10.

#18

Hm i am under the assumption that with a FIFO queue you must first complete a message (or batch of messages - 10) before receiving the next. So it cannot work concurrently?

#19

The only messages that are ordered are ones in the same message group

#20

Maybe thats what i read