KubeMQ
ConnectorsAWS (SQS & SNS)How-to guides

SNS fan-out

The AWS connector's SNS surface — virtual topics, SQS and HTTP/HTTPS subscriptions, the confirmation flow, MessageAttributes filtering, PublishBatch, and FIFO.

This guide covers the SNS surface: topic management, subscriptions and the confirmation flow, message filtering, raw vs enveloped delivery, PublishBatch, FIFO topics, and MessageStructure=json. SNS topics are virtual BoltDB registry entries (no native channel) that fan out, at publish time, to SQS subscriptions and HTTP/HTTPS webhooks. The SNS surface ships 17 actions (see Capabilities).

Topic management

  • CreateTopic — idempotent on an existing name; a FIFO topic uses the .fifo suffix.
  • DeleteTopic — cascades its subscriptions.
  • ListTopics — ARN-sorted, 100 per page; not authorization-filtered.
  • GetTopicAttributes / SetTopicAttributes — the only writable attributes are DisplayName and DeliveryPolicy.

Policy is rejected. Setting a topic Policy returns InvalidParameter. Topic-level ContentBasedDeduplication is not supported — a set returns InvalidParameter and a get always returns "false". For FIFO topics, pass an explicit MessageDeduplicationId (see below).

Subscriptions

Subscribe accepts three protocols only:

ProtocolBehavior
sqsThe endpoint must be a registry queue ARN; the subscription is auto-confirmed immediately (Confirmed=true).
http / httpsGoes pending with a 48 h confirmation token; a SubscriptionConfirmation envelope is POSTed best-effort; expired pending subscriptions are swept hourly. Confirm via ConfirmSubscription.
anything elseemail / email-json / sms / lambda / application / firehoseInvalidParameter "protocol not supported".

The writable subscription attributes are RawMessageDelivery (bool), FilterPolicy (JSON, parsed and validated at set), FilterPolicyScope, RedrivePolicy (JSON), and DeliveryPolicy.

The http/https confirmation GET is SigV4-exempt so the SubscribeURL embedded in a SubscriptionConfirmation envelope is usable as-is — see Authentication.

Message filtering

A FilterPolicy filters which subscriptions receive a publish. All 8 AWS operators are supported: exact string/number/bool, prefix, suffix, anything-but, numeric comparison/range, exists true/false, and CIDR. Keys are ANDed, values within a key ORed; an empty policy matches everything. Write-time limits: ≤ 5 keys, ≤ 150 value combinations. Binary attributes match only exists:true.

MessageBody-scope filtering is unsupported. Only FilterPolicyScope = MessageAttributes works. Setting FilterPolicyScope = MessageBody is rejected at the attribute setter with InvalidParameter "MessageBody scope is not supported". Filter only on MessageAttributes. See Capabilities.

Raw vs enveloped delivery

RawMessageDelivery controls the delivered shape:

  • Enveloped (false, the default) — the body is the SNS Notification JSON: {Type:"Notification", MessageId, TopicArn, Subject?, Message, Timestamp, SignatureVersion:"1", Signature:"", SigningCertURL:"", UnsubscribeURL, MessageAttributes?}. Each MessageAttributes entry is {Type, Value} (Binary is base64).
  • Raw (true) — for an SQS subscription, the bare message bytes plus the attribute tag codec and sns_topic_arn / sns_subject tags; for an HTTP subscription, the bare payload with attributes mapped to x-amz-sns-attr-{name} headers.

SNS notifications are unsigned. Signature and SigningCertURL are present in the envelope but empty (SignatureVersion is "1"). No SDK-side signature verification can be performed. Do not rely on verifying SNS message signatures. See Reliability.

Raw-HTTP attribute deviation. Real AWS drops message attributes for raw HTTP delivery; this connector instead maps them to x-amz-sns-attr-{name} headers. See Channel mapping.

Fan-out semantics

Publish / PublishBatch fan out to every confirmed, filter-matching subscription:

  • there is one MessageId per publish, shared across all deliveries;
  • all sqs deliveries of one publish go out in a single SendQueueMessagesBatch;
  • per-target failures (deleted queue, unauthorized, oversize, FIFO mismatch) are dropped with a metric and do not fail the publish;
  • zero matching subscriptions → the publish succeeds and the message is dropped;
  • a per-target Casbin write check applies on each sqs.{queue}.

MessageStructure=json is supported: pass a JSON object with a string default key plus optional per-protocol string overrides.

FIFO topics

A .fifo topic restricts Subscribe to the sqs protocol onto a .fifo queue (http / https are rejected). Publish / PublishBatch:

  • require MessageGroupId (and reject it on a standard topic);
  • treat MessageDeduplicationId as optional — topic-level content-based dedup is unsupported, so pass it explicitly.

Group and dedup ids propagate into the FIFO queue-message build. See Reliability for the full FIFO ordering and dedup detail.

Error quick reference

TriggerAWS error code
Subscribe with email / sms / lambda / firehose / … protocolInvalidParameter
Subscribe with FilterPolicyScope=MessageBodyInvalidParameter
Publish with MessageGroupId to a standard topicInvalidParameter
FIFO topic Subscribe with http / a non-FIFO queueInvalidParameter
Set a topic Policy / topic-level ContentBasedDeduplicationInvalidParameter
Publish with TargetArn / PhoneNumberInvalidParameter
Topic / subscription not in the registryNotFound (404)
Publish to a topic with zero matching subscriptionsnone — succeeds, message dropped

Was this page helpful?

On this page