KubeMQ
ConnectorsRabbitMQ (AMQP 0-9-1)How-to guides

Reliability

Delivery guarantees on the KubeMQ RabbitMQ connector — publisher confirms, mandatory/return, dead-letter exchanges, per-message TTL, and at-least-once delivery.

This guide covers publisher confirms, mandatory / basic.return, dead-letter exchanges (DLX), per-message TTL, delayed delivery (x-delay), and at-least-once delivery — plus the gotchas that bite RabbitMQ migrants. Several behaviours deviate from RabbitMQ; the most dangerous is the publish-then-close loss below.

Fire-and-forget publishes followed by an immediate close are silently lost unless you use publisher confirms (or keep the connection open until the consumer has drained). This is the single most dangerous behaviour for a multi-message producer — it fails with no error, no nack, and no log on the client. See Publish-then-close.

Publisher confirms

confirm.select enables a per-channel, monotonically-increasing publish sequence from 1. A sequence is acked only after all routed queues accept the message:

  • single routed queue → after the queue send;
  • multiple routed queues → after the batch send for all queues.

multiple=true coalesces consecutive acked sequences; confirmations may arrive out of order.

Publisher confirms have no rollback. On any-queue failure the connector sends basic.nack(seq) — but queues that already accepted the message stay delivered. A naive retry on a nack may therefore duplicate the message in the queues that already got it. Design retries to be idempotent.

tx.* is unsupported

tx.select never succeeds, so confirm.select can never follow a transaction:

  • on a normal channel → 540 not-implemented (connection error);
  • on a channel already in confirm mode → 406 precondition-failed (channel error).

Publish-then-close silently loses unconfirmed messages

This is the single most dangerous behaviour for a multi-message producer to get wrong, because it fails silently — no error, no nack, no log on the client.

Fire-and-forget publishes then an immediate close are silently lost. A basic.publish without confirms does not block: it only hands the message to the connector's per-channel executor queue, which sends to the KubeMQ queue asynchronously. If you close the channel or connection before that executor has drained, every still-buffered publish is abandoned — never sent, with no error returned to the client. The buffer holds up to 64 pending publishes, so a tight publish-then-close loop can drop dozens of messages at once.

In an internal test run against this connector, with no confirms: 30 fire-and-forget publishes immediately followed by a close lost 16; 100 lost 84. These exact counts are timing-dependent (they reflect a race between the executor's send rate and how soon you close) and will vary by environment — the only guarantee is "more than zero." On a confirm channel that waits for acks before closing, the same loops lose 0, which is the invariant to rely on.

The fix — pick one

  1. Use a confirm channel and wait for all acks before closing (recommended). Call confirm.select, publish, then block until every publish is acked. A publish is acked only after the connector has actually sent it to the queue, so waiting for confirms forces the executor queue to drain before you close.
  2. Or keep the connection open until the consumer has drained. If you genuinely cannot use confirms, don't close immediately after publishing — keep the channel/connection alive until you have independent evidence (a consumer ack, a queue-depth check) that the messages were ingested.

Remember: confirms have no rollback, so make any retry idempotent.

// amqp091-go — confirm mode; wait for acks before closing.
_ = ch.Confirm(false)
confirms := ch.NotifyPublish(make(chan amqp.Confirmation, 1))
_ = ch.Publish("", "orders", false, false, amqp.Publishing{Body: body})
if c := <-confirms; !c.Ack {
    log.Println("nacked — retry idempotently")
}
// only now is it safe to close
# pika — confirm mode; BlockingChannel raises on a nack.
channel.confirm_delivery()
try:
    channel.basic_publish(exchange="", routing_key="orders", body=body)
except pika.exceptions.UnroutableError:
    ...  # retry idempotently
# publish_delivery blocks until confirmed, so it is now safe to close
// amqp-client — confirm mode; block until all publishes are confirmed.
channel.confirmSelect();
channel.basicPublish("", "orders", null, body);
channel.waitForConfirmsOrDie(5_000); // drains the executor queue before close
// amqplib — ConfirmChannel; await each publish callback before closing.
const ch = await connection.createConfirmChannel();
await new Promise<void>((resolve, reject) => {
  ch.publish("", "orders", body, {}, (err) => (err ? reject(err) : resolve()));
});
// safe to close now
// RabbitMQ.Client — confirm mode; wait for confirms before closing.
channel.ConfirmSelect();
channel.BasicPublish("", "orders", body: body);
channel.WaitForConfirmsOrDie(TimeSpan.FromSeconds(5));
# bunny — confirm mode; wait for confirms before closing.
channel.confirm_select
exchange.publish(body, routing_key: "orders")
channel.wait_for_confirms # blocks until the executor queue drains
// lapin — publisher confirms; await the returned confirmation.
let confirm = channel
    .basic_publish("", "orders", BasicPublishOptions::default(), body,
        BasicProperties::default())
    .await?
    .await?; // second await resolves the confirm
// confirm is now Ack/Nack — safe to close

Mandatory / return

basic.publish(mandatory=true) on an unroutable message returns basic.return(312 NO_ROUTE) with the full message content, sent before the ack in confirm mode. Without mandatory, an unroutable message is silently dropped.

Dead-letter exchange (DLX)

Configure with the queue arguments x-dead-letter-exchange / x-dead-letter-routing-key.

DLX is rejected-trigger only. The only trigger is an explicit basic.reject / basic.nack(requeue=false). TTL expiry and per-queue length limits do NOT dead-letter — RabbitMQ also dead-letters on expired and maxlen; KubeMQ does not.

When a message is dead-lettered:

  • the x-death array is RabbitMQ-exact (queue, reason="rejected", time, exchange, routing-keys, count), most-recent-first;
  • the x-first-death-* / x-last-death-* convenience headers are set;
  • the original expiration property is moved to x-death[0].original-expiration;
  • the cycle cap is DeadLetterMaxHops (16) per (queue, reason);
  • a missing DLX exchange → drop + WARN + ack the original.

Per-message TTL

Set the expiration property to milliseconds as a numeric string (^\d+$, else 406). The connector computes ceil(ms/1000) seconds, clamped to a per-queue maximum (default 12h).

TTL never dead-letters. Expired messages are eager-dropped inside the broker, never dead-lettered — even with a DLX configured. (RabbitMQ lazily expires at the head and dead-letters with reason expired; KubeMQ does neither.) Note also that x-message-ttl / x-expires are inert — only the per-message expiration property drives TTL.

Delayed delivery (x-delay)

Set the x-delay header to milliseconds; the connector computes ceil(ms/1000) seconds, clamped to a per-queue maximum (12h), and strips the x-delay header on delivery (matching the RabbitMQ delayed-message-exchange plugin).

At-least-once delivery

Unacked deliveries are requeued on disconnect — zero loss, even on an ungraceful disconnect. Graceful shutdown sends connection.close(320), nacks pending, and requeues unacked. Durable queues persist across restart with Redelivered == true on recovery. Exactly-once is NOT provided. See Queues and consumers.

Node-local caveat (cluster)

Exclusive queues and direct reply-to are node-local. In a cluster, exclusive queues and the amq.rabbitmq.reply-to pseudo-queue live only on the node owning the connection. The requester and responder (or producer and exclusive consumer) must land on the same node — use load-balancer session affinity, or switch to an explicit reply-queue + correlation-id. Single-node deployments are unaffected.

Error quick reference

TriggerCode
mandatory=true + unroutable312
expiration not ^\d+$406
tx.select (normal channel)540
tx.select (confirm-mode channel)406
Graceful shutdown / connection limit320
Fire-and-forget publish then immediate closenone — silently dropped

Was this page helpful?

On this page