All concepts
Architectures

Lambda + SQS

Serverless consumers with built-in scaling and partial-failure support.

Why it matters

Removes the operational burden of running long-lived consumer processes; Lambda's event source mapping handles polling and scaling.

Event source mapping

Lambda polls the SQS queue on your behalf — you don't call ReceiveMessage in your function. Configure the event source mapping with batch size, batching window, maximum concurrency, and function timeout.

  • Standard queues: batch size 1–10 (up to 10,000 with maximumBatchingWindowInSeconds).
  • FIFO queues: batch size 1–10.
  • Scaling: up to 60 additional pollers per minute, 1,000 concurrent invocations default.

Partial batch failures

For standard and FIFO queues, return a batchItemFailures array listing failed message IDs. Lambda retries only failed messages; successful messages in the batch are not returned to the queue.

export const handler = async (event) => {
  const failures = [];
  for (const record of event.Records) {
    try { await process(record); }
    catch { failures.push({ itemIdentifier: record.messageId }); }
  }
  return { batchItemFailures: failures };
};

Visibility timeout alignment

Set the queue visibility timeout to at least 6 times the Lambda function timeout. If the function times out or crashes without deleting, the message becomes visible again after visibility timeout.

  • Function timeout < visibility timeout (required).
  • Recommended: visibility timeout = 6 × function timeout.

FIFO concurrency

For FIFO queues, Lambda scales concurrent executions based on the number of active MessageGroupIds with messages in flight — not unbounded like standard queues.

DLQ for Lambda

Configure a DLQ on the SQS queue (not just Lambda's async DLQ) to capture messages that exceed maxReceiveCount. Lambda's own DLQ captures async invocation failures, not SQS processing failures.

Gotchas
  • !Without batchItemFailures, one failed message fails the whole batch — every successful message gets reprocessed.
  • !Function timeout must be less than visibility timeout, otherwise duplicate processing occurs.
  • !ReportBatchItemFailures must be enabled on the event source mapping for partial failure handling.