Reliability

This guide covers the reliability mechanisms across both surfaces: SQS visibility and in-flight tracking, FIFO ordering and deduplication, SQS DLQ/redrive, the SNS HTTP delivery retry → circuit-breaker → DLQ pipeline, and at-least-once delivery. Several behaviors have node-local caveats — read the callouts.

SQS visibility and in-flight

A received message is hidden for a visibility window before it becomes receivable again. Precedence: per-request VisibilityTimeout (0–43200) > the queue default > 30 s. ChangeMessageVisibility(timeout>0) moves the deadline; ChangeMessageVisibility(0) NAcks the message so it becomes visible again at the tail. A 250 ms sweeper NAcks expired in-flight entries back to the tail, and ApproximateReceiveCount increments on each re-receive.

Received-but-not-deleted messages count against a per-queue / per-node in-flight cap, MaxInflightPerQueue (default 20,000). Exceeding it returns OverLimit. Full SQS basics — send/receive/delete, long polling, batch — are in SQS queues and consumers.

SQS FIFO ordering and deduplication

A .fifo queue (FifoQueue is immutable post-create) gives per-group ordering and dedup:

• MessageGroupId is required on send (1–128 printable ASCII characters); per-message DelaySeconds is rejected; ReceiveRequestAttemptId is accepted and ignored.
• Each group maps to its own channel sqs.{name}.fifo.g.{enc(group)}.
• Deduplication uses an explicit MessageDeduplicationId, or the SHA-256 of the body when ContentBasedDeduplication is on. A 5-minute LRU window is keyed {queue}:{dedupId} (or {queue}:{group}:{dedupId} under messageGroup scope). A duplicate returns the original MessageId / SequenceNumber without re-publishing.
• Ordering allows at most one un-acked downstream Get per group at a time (a per-group lock); ReceiveMessage drains groups round-robin, ≤ 10 total.

SQS DLQ / redrive

Set a queue RedrivePolicy of {deadLetterTargetArn, maxReceiveCount}. The connector stamps each message with MaxReceiveCount / MaxReceiveQueue=sqs.{dlq} at send. The message broker moves a message to the DLQ when its receive count exceeds maxReceiveCount, surfacing DeadLetterQueueSourceArn on the redriven message. ListDeadLetterSourceQueues reverse-resolves which queues redrive into a given DLQ.

SNS delivery: retry → circuit-breaker → DLQ

HTTP / HTTPS subscriptions are delivered by an in-memory engine: 8 workers, a bounded job queue of 10,000 (overflow drops the message, increments a metric, and logs a WARN), and a 10 s per-request timeout.

The default retry schedule (overridable by a stored DeliveryPolicy — subscription wins over topic wins over default) is: 4 immediate attempts + 2 @ 10 s + 10 exponential (1 s → 60 s) + 35 @ 60 s = 50 retries / 51 attempts ≈ 39 minutes. Retries fire on 5xx / 429 / timeout / connection errors; other 4xx responses are terminal.

The circuit breaker is per-endpoint: 5 consecutive failures → open for 30 s.

On retry exhaustion, the body is redriven to the subscription's RedrivePolicy DLQ queue; if there is none, it is dropped and a metric increments.

Outgoing delivery headers are x-amz-sns-message-type, x-amz-sns-message-id, x-amz-sns-topic-arn, and x-amz-sns-subscription-arn; the Content-Type is text/plain; charset=UTF-8.

At-least-once delivery

Unacked SQS deliveries are NAcked back to the queue tail by the 250 ms sweeper at visibility expiry, or on graceful shutdown (≤ 10 s). The registry survives restart (a shared StorePath). So no message is lost on a graceful path, but a message may be redelivered (ApproximateReceiveCount increments) — make consumers idempotent. Exactly-once is not provided.

Node-local caveat (cluster)

Error quick reference

Related