Event-driven Systems
Why queues sit at the heart of every distributed system.
Queues decouple producers from consumers in time, throughput, and failure modes — the three axes that make distributed systems hard.
What SQS provides
Amazon SQS is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. Messages are stored redundantly across multiple servers for durability and availability.
Decoupling in time
Producers don't wait for consumers to finish. Work is enqueued immediately; consumers process when ready. Spikes are absorbed by the queue instead of overwhelming downstream services.
Decoupling in throughput
Producers and consumers scale independently. A 10× write burst doesn't require 10× consumers if the queue can buffer work until consumers catch up — bounded by retention period.
Decoupling in failure
A consumer outage doesn't lose work — messages persist until processed or until retention expires. Combine with DLQs to isolate poison messages without blocking the pipeline.
When NOT to use queues
Queues add latency and complexity. For synchronous request/response (user waiting for a result), use API Gateway + Lambda, direct HTTP/gRPC, or Step Functions for orchestration. Use SQS when the producer can fire-and-forget.
- Good fit: async workflows, background jobs, event notification, burst absorption.
- Poor fit: real-time user-facing responses, strict sub-second latency.
SQS vs SNS vs MQ
SQS is queue-based (one consumer typically processes each message). SNS is pub/sub (one-to-many push). Amazon MQ supports JMS/AMQP for legacy broker migrations. SQS + SNS is recommended for new cloud-native applications.
- !Queues hide failures. Without DLQ alarms and queue-age monitoring, you'll discover problems only when retention drops messages.
- !Decoupling adds latency. For synchronous request/response, use a different pattern.
- !At-least-once delivery requires idempotent consumers — this is non-negotiable for standard queues.