# Architecture (/connectors/aws/concepts/architecture)



The KubeMQ **AWS SQS + SNS connector** is an embedded, wire-protocol bridge inside
kubemq-server that speaks the genuine AWS SQS and SNS HTTP protocols on a dedicated second
HTTP listener (default TCP **4566**, the LocalStack convention). It is built **only when the
connector is enabled** (`CONNECTORS_AWS_ENABLE=true`), because enabling it opens a new port.
Any standard AWS SDK connects to it with **only an endpoint-URL change** — no code changes,
no library swap, no LocalStack.

It is **one connector binary with two service surfaces** that map onto two distinct KubeMQ
models:

* **SQS** maps each queue onto a native KubeMQ **Queue** channel `sqs.{name}`, so AWS
  producers and native gRPC/REST consumers share the same messages.
* **SNS** topics are **virtual** registry entries — replicated across cluster nodes — that
  fan out at publish time to subscribed SQS queues (a batch send) and HTTP/HTTPS webhooks (a
  delivery engine).

Unlike connectors that touch every KubeMQ pattern, the AWS connector touches the **Queue
primitive** (SQS) and a **topic registry** (SNS) — it does **not** map onto KubeMQ's
events / events-store / commands / queries patterns, and there is **no RPC** anywhere. This
single fact drives the mental model.

## How SQS & SNS map to KubeMQ [#how-sqs--sns-map-to-kubemq]

A request arrives at the single AWS-style endpoint; the connector detects the protocol,
verifies the SigV4 signature, and dispatches. SQS operations land on the KubeMQ Queue channel
`sqs.{name}` through the message broker. SNS publishes resolve the virtual topic registry and
fan out to every confirmed, filter-matching subscription.

<Mermaid
  chart="`
graph TB
CLIENT[&#x22;AWS SDK client<br/>http://host:4566 (endpoint override)&#x22;]
LISTEN[&#x22;Dedicated HTTP listener :4566<br/>POST / and GET / dispatch&#x22;]
DETECT{{&#x22;Protocol detection<br/>SQS = JSON · SNS = Query&#x22;}}
SIGV4[&#x22;SigV4 middleware<br/>(accept-any OR static credentials)&#x22;]
SQS[&#x22;SQS surface<br/>SendQueueMessage(s) / Get (AutoAck=false)&#x22;]
SNS[&#x22;SNS surface<br/>virtual registry · publish-time fan-out&#x22;]
CH[&#x22;Queue channel<br/>sqs.{name} · sqs.{name}.fifo.g.{enc(group)}&#x22;]
HOOK[&#x22;HTTP/HTTPS webhook<br/>delivery engine&#x22;]
BROKER[&#x22;Message Broker<br/>(durable store)&#x22;]

CLIENT --> LISTEN
LISTEN --> DETECT
DETECT --> SIGV4
SIGV4 --> SQS
SIGV4 --> SNS
SQS --> CH
SNS -- &#x22;sqs subscriptions&#x22; --> CH
SNS -- &#x22;http/https subscriptions&#x22; --> HOOK
CH --> BROKER

class CLIENT client
class LISTEN,DETECT,SIGV4,SQS,SNS connector
class CH,BROKER broker
`"
/>

*SQS operations land on the KubeMQ Queue channel `sqs.{name}`; SNS publishes resolve the virtual, cluster-replicated registry and fan out to subscribed SQS queues and HTTP/HTTPS webhooks. The Queue channel is backed by the message broker's durable store.*

The channel mapping is the single most important mental model:

| Concept                            | Behavior                                                                                                                                                                                                                              |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Listener**                       | A dedicated second HTTP server on `Connectors.Aws.Port` (default **4566**). `POST /` and `GET /` both dispatch; there is no per-route REST path — everything is a single AWS-style endpoint.                                          |
| **Protocol detection**             | SQS = AWS **JSON protocol** (`X-Amz-Target: AmazonSQS.{Op}`) with a Query-protocol fallback. SNS = AWS **Query protocol** only (form body / GET query → XML).                                                                         |
| **SQS queue → channel**            | SQS queue `orders` ↔ native KubeMQ **Queue** channel `sqs.orders` (`channelPrefix = "sqs."`). AWS producers and native gRPC/REST consumers share the same messages.                                                                   |
| **FIFO group → per-group channel** | A FIFO queue `{name}.fifo` fans each message group onto its own channel `sqs.{name}.fifo.g.{enc(group)}`, where `enc` percent-encodes bytes outside `[a-zA-Z0-9_-]`.                                                                  |
| **SNS topic = virtual**            | SNS topics have **no native channel** — they are registry entries replicated across cluster nodes. The authorization pseudo-resource is `sns.{topic}`. Fan-out resolves to target SQS channels + HTTP/HTTPS webhooks at publish time. |
| **Registry is authoritative**      | Only resources created via the AWS API are visible. A native `sqs.foo` channel never `CreateQueue`d returns `NonExistentQueue`.                                                                                                       |
| **ARNs / URLs**                    | `arn:aws:sqs:{Region}:{AccountId}:{name}`; queue URL form `{scheme}://{host}/{AccountId}/{name}`. Resolution parses **path only** → addressing is **path-style**, so stale hosts in saved URLs still work.                            |

Region defaults to the ARN segment `kubemq` and is **NOT enforced** in SigV4; AccountId
defaults to `000000000000`. See
[Channel mapping](/connectors/aws/reference/channel-mapping) and
[Configuration](/connectors/aws/concepts/configuration).

## SQS → KubeMQ internally [#sqs--kubemq-internally]

Each SQS operation maps onto a KubeMQ Queue operation:

| AWS operation                                    | KubeMQ internal                                                |
| ------------------------------------------------ | -------------------------------------------------------------- |
| `SendMessage` / `SendMessageBatch`               | `SendQueueMessage` / `SendQueueMessagesBatch`                  |
| `ReceiveMessage`                                 | a downstream `Get` with `AutoAck=false`                        |
| `DeleteMessage`                                  | fire-and-forget `AckRange(seq)`                                |
| visibility expiry / `ChangeMessageVisibility(0)` | `NAckRange(seq)` (message visible at the tail)                 |
| `PurgeQueue`                                     | `AckAllQueueMessages` (60 s cooldown → `PurgeQueueInProgress`) |

Received-but-not-deleted messages are tracked **node-locally** (per-queue maps + a global
visibility-deadline min-heap); a 250 ms sweeper NAcks expired entries back to the tail. A
receipt handle minted on one node is rejected on another, so clustered deployments need a
**sticky load balancer**. See [SQS queues](/connectors/aws/how-to/sqs-queues) and
[SQS queues and consumers](/connectors/aws/how-to/sqs-queues-and-consumers).

## SNS → KubeMQ internally [#sns--kubemq-internally]

Topics and subscriptions live **only** in the registry (replicated across cluster nodes) —
they have no native channel. At publish time the connector resolves every **confirmed**,
filter-matching subscription and:

* for `sqs` subscriptions, sends all targets of one publish in a single
  `SendQueueMessagesBatch` onto each target queue's `sqs.{queue}` channel;
* for `http`/`https` subscriptions, hands the body to the in-memory webhook delivery engine.

**One `MessageId` per publish** is shared across all deliveries. A publish with zero matching
subscriptions **succeeds** (the message is dropped). Because the topic registry is replicated
across cluster nodes, a topic created on one node is visible on the others; the cluster uses a
registry-sync conflict resolution to converge concurrent writes. See
[Fan-out](/connectors/aws/how-to/fan-out) and
[SNS fan-out](/connectors/aws/how-to/sns-fan-out).

## Cross-protocol interop [#cross-protocol-interop]

Because every SQS queue is a normal KubeMQ Queue channel, an SQS `SendMessage` to `sqs.orders`
is consumable by a gRPC/REST queue client on the same channel — and vice-versa. This lets you
migrate one side at a time, or run AWS-SDK producers alongside native KubeMQ consumers.

<Mermaid
  chart="`
graph LR
AWS[&#x22;AWS SDK client<br/>queue: orders&#x22;]
GRPCCLIENT[&#x22;gRPC / REST client<br/>channel: sqs.orders&#x22;]
BROKER[&#x22;Message Broker&#x22;]

AWS -- &#x22;SendMessage&#x22; --> BROKER
BROKER -- &#x22;consume&#x22; --> GRPCCLIENT
GRPCCLIENT -- &#x22;produce&#x22; --> BROKER
BROKER -- &#x22;ReceiveMessage&#x22; --> AWS

class AWS,GRPCCLIENT client
class BROKER broker
`"
/>

*The same KubeMQ Queue channel `sqs.orders` backs both sides, so an AWS SDK client and a gRPC/REST client interoperate transparently.*

<Callout type="info">
  A message produced by a **native** KubeMQ client on `sqs.*` lacks the connector's `sqs_*`
  tags. On the SQS receive side its `MessageId` falls back to the broker MessageID, it has no
  `SenderId`, and no policy stamping is applied. This is harmless for interop — the body and
  tags round-trip — but do not assume an SQS-style `MessageId`/`SenderId` on natively-produced
  messages. See [Cross-protocol interop](/connectors/aws/concepts/cross-protocol-interop).
</Callout>

## Related [#related]

<Cards>
  <Card title="Channel mapping" href="/connectors/aws/reference/channel-mapping" description="The sqs.{name} grammar, FIFO group encoding, the sns.{topic} pseudo-resource, and attribute mapping." />

  <Card title="SQS queues" href="/connectors/aws/how-to/sqs-queues" description="Visibility timeouts, long-poll, batch, message attributes, and FIFO over sqs.{name}." />

  <Card title="SNS topics" href="/connectors/aws/how-to/sns-topics" description="Virtual topics, subscriptions, confirmation, and the publish-time fan-out model." />

  <Card title="Cross-protocol interop" href="/connectors/aws/concepts/cross-protocol-interop" description="Sharing an SQS channel with native gRPC/REST clients and the native-producer caveat." />
</Cards>