# Error Handling & Dead-Lettering (/integrations/masstransit/how-to/error-handling-dlq)



When a consumer throws or a message can't be delivered, MassTransit doesn't lose it — it routes the message to a side channel so you can inspect, replay, or alert on it. On the KubeMQ transport this maps to a pair of dedicated channels per endpoint, plus an optional native dead-letter queue (DLQ) that the broker manages by receive count. This page covers both, along with the retry pipeline that runs before a message ever reaches `_error`, message expiration as an error-avoidance tool, the transport exception hierarchy, and a troubleshooting checklist.

## Prerequisites [#prerequisites]

* A MassTransit bus already configured with `UsingKubeMQ` (or `AddKubeMQRider`) and at least one `ReceiveEndpoint` (see [Configuration](/integrations/masstransit/how-to/configuration))

## Error and Skipped Channels [#error-and-skipped-channels]

Every receive endpoint gets two automatically-derived side channels, following KubeMQ's `_error` / `_skipped` naming convention:

| Channel | Suffix              | What lands here                                                           |
| ------- | ------------------- | ------------------------------------------------------------------------- |
| Error   | `{channel}_error`   | Faulted messages — a consumer threw and all retries were exhausted        |
| Skipped | `{channel}_skipped` | Skipped / dead-lettered messages — no matching consumer, or undeliverable |

For an endpoint named `order-processing`, faulted messages route to `order-processing_error` and skipped messages to `order-processing_skipped`. No topology to provision — the channels are created on first use, exactly like any other KubeMQ channel.

<Mermaid
  chart="`
flowchart LR
  Send[&#x22;Send / Publish&#x22;] --> Q[&#x22;order-processing&#x22;]
  Q --> C[&#x22;Consumer&#x22;]
  C -->|&#x22;success (ack)&#x22;| Done[&#x22;Done&#x22;]
  C -->|&#x22;faulted (retries exhausted)&#x22;| Err[&#x22;order-processing_error&#x22;]
  Q -->|&#x22;no consumer / undeliverable&#x22;| Skip[&#x22;order-processing_skipped&#x22;]
`"
/>

*A faulted message (after retries) routes to `_error`; an undeliverable one routes to `_skipped`.*

<Callout type="info">
  The `_error` / `_skipped` split is a MassTransit convention that is transport-independent — it works the same way it would on RabbitMQ or Azure Service Bus. The KubeMQ transport simply realizes those channels as native KubeMQ channels.
</Callout>

## Fault Metadata Tags [#fault-metadata-tags]

When a message is routed to `{channel}_error`, the transport attaches fault metadata as KubeMQ tags (surfaced as MassTransit headers). Read them on the error channel to understand what failed:

<TypeTable
  type="{
  &#x22;MT-Fault-ExceptionType&#x22;: {
    type: &#x22;string&#x22;,
    description: &#x22;Fully-qualified type name of the thrown exception.&#x22;,
  },
  &#x22;MT-Fault-Message&#x22;: {
    type: &#x22;string&#x22;,
    description: &#x22;The exception message.&#x22;,
  },
  &#x22;MT-Fault-StackTrace&#x22;: {
    type: &#x22;string&#x22;,
    description: &#x22;The exception stack trace.&#x22;,
  },
  &#x22;MT-Fault-Timestamp&#x22;: {
    type: &#x22;string&#x22;,
    description: &#x22;ISO 8601 timestamp when the fault occurred.&#x22;,
  },
  &#x22;MT-Fault-RetryCount&#x22;: {
    type: &#x22;string&#x22;,
    description: &#x22;Number of delivery attempts before the message faulted.&#x22;,
  },
}"
/>

To inspect faults, register a consumer on the `_error` channel and read the headers:

```csharp title="ErrorChannel/Program.cs"
builder.Services.AddMassTransit(x =>
{
    x.AddConsumer<FaultyOrderConsumer>();
    x.AddConsumer<ErrorChannelConsumer>();
    x.UsingKubeMQ((context, cfg) =>
    {
        cfg.Host("localhost", 50000, h => h.ClientId = "error-channel-demo");

        // Main endpoint -- consumer throws on certain messages
        cfg.ReceiveEndpoint("error-demo-orders", e =>
        {
            e.ConfigureKubeMQ(t => { });
        });

        // Error channel endpoint -- receives faulted messages
        cfg.ReceiveEndpoint("error-demo-orders_error", e =>
        {
            e.ConfigureKubeMQ(t => { });
        });
    });
});
```

```csharp title="ErrorChannelConsumer.cs"
public class ErrorChannelConsumer(ILogger<ErrorChannelConsumer> logger)
    : IConsumer<OrderMessage>
{
    public Task Consume(ConsumeContext<OrderMessage> context)
    {
        logger.LogWarning(
            "Faulted order on error channel: OrderId={OrderId}",
            context.Message.OrderId);

        // Inspect fault metadata from headers/tags
        if (context.Headers.TryGetHeader("MT-Fault-ExceptionType", out var exType))
            logger.LogWarning("  Fault exception type: {ExceptionType}", exType);
        if (context.Headers.TryGetHeader("MT-Fault-Message", out var exMsg))
            logger.LogWarning("  Fault message: {FaultMessage}", exMsg);
        if (context.Headers.TryGetHeader("MT-Fault-Timestamp", out var ts))
            logger.LogWarning("  Fault timestamp: {Timestamp}", ts);

        return Task.CompletedTask;
    }
}
```

## Native DLQ via MaxReceiveCount [#native-dlq-via-maxreceivecount]

Beyond the MassTransit-managed `_error` channel, the KubeMQ transport exposes a **native dead-letter queue** backed by the broker's receive-count tracking. Configure it per endpoint with `ConfigureKubeMQ`:

```csharp title="IKubeMQEndpointTransportConfigurator.cs"
public interface IKubeMQEndpointTransportConfigurator
{
    /// <summary>Set message expiration in seconds.</summary>
    int ExpirationSeconds { set; }

    /// <summary>Configure KubeMQ native DLQ via MaxReceiveCount.</summary>
    void UseNativeDlq(int maxReceiveCount, string? dlqChannel = null);
}
```

`UseNativeDlq(maxReceiveCount, dlqChannel)` tells the broker to move a message to the DLQ channel once it has been received `maxReceiveCount` times without a successful ack. The `dlqChannel` argument is optional; pass a name to route dead-lettered messages to a channel of your choosing.

```csharp title="NativeDlq/Program.cs"
builder.Services.AddMassTransit(x =>
{
    x.AddConsumer<DlqOrderConsumer>();
    x.AddConsumer<DlqInspectorConsumer>();
    x.UsingKubeMQ((context, cfg) =>
    {
        cfg.Host("localhost", 50000, h => h.ClientId = "native-dlq-demo");

        // After 3 failed receives, move the message to "orders-dlq"
        cfg.ReceiveEndpoint("dlq-demo-orders", e =>
        {
            e.ConfigureKubeMQ(t => t.UseNativeDlq(3, "orders-dlq"));
        });

        // DLQ inspection endpoint -- reads messages that exceeded max receive count
        cfg.ReceiveEndpoint("orders-dlq", e =>
        {
            e.ConfigureKubeMQ(t => { });
        });
    });
});
```

The DLQ inspector consumer reads the same fault metadata tags that the error channel carries:

```csharp title="DlqInspectorConsumer.cs"
public class DlqInspectorConsumer(ILogger<DlqInspectorConsumer> logger)
    : IConsumer<DlqOrderMessage>
{
    public Task Consume(ConsumeContext<DlqOrderMessage> context)
    {
        logger.LogError(
            "Message reached DLQ after max retries: OrderId={OrderId}",
            context.Message.OrderId);

        if (context.Headers.TryGetHeader("MT-Fault-RetryCount", out var retryCount))
            logger.LogError("  Retry count: {RetryCount}", retryCount);
        if (context.Headers.TryGetHeader("MT-Fault-ExceptionType", out var exType))
            logger.LogError("  Last exception: {ExceptionType}", exType);

        return Task.CompletedTask;
    }
}
```

This is the same `e.ConfigureKubeMQ(k => k.UseNativeDlq(maxReceiveCount, dlqChannel))` shape shown in the [receive-endpoint reference](/integrations/masstransit/reference/api). Use the native DLQ when you want the broker — not the application — to enforce a hard ceiling on redelivery attempts.

## Error Channel vs Skipped Channel [#error-channel-vs-skipped-channel]

The two side channels capture different failure modes, and you inspect them the same way: by registering a consumer on the suffixed channel.

<Tabs items="[&#x22;Error channel&#x22;, &#x22;Skipped channel&#x22;, &#x22;Inspect both&#x22;]">
  <Tab value="Error channel">
    A message lands on `{channel}_error` when a **consumer throws** and retries are exhausted. Register a consumer on the `_error` channel and read the `MT-Fault-*` tags.

    ```csharp title="ErrorChannel/Program.cs"
    cfg.ReceiveEndpoint("error-demo-orders", e =>
    {
        e.ConfigureKubeMQ(t => { });
    });

    cfg.ReceiveEndpoint("error-demo-orders_error", e =>
    {
        e.ConfigureKubeMQ(t => { });
    });
    ```
  </Tab>

  <Tab value="Skipped channel">
    A message lands on `{channel}_skipped` when it is **undeliverable** — for example, no consumer is registered for the message type on that endpoint. Register a consumer on the `_skipped` channel to capture dead-lettered messages.

    ```csharp title="SkippedChannel/Program.cs"
    // Only the skipped-channel consumer is registered -- the main channel
    // has no matching consumer, so messages are skipped.
    cfg.ReceiveEndpoint("skipped-demo-events", e =>
    {
        e.ConfigureKubeMQ(t => { });
    });

    cfg.ReceiveEndpoint("skipped-demo-events_skipped", e =>
    {
        e.ConfigureKubeMQ(t => { });
    });
    ```
  </Tab>

  <Tab value="Inspect both">
    For full visibility, attach inspector consumers to both side channels of the same endpoint. The error channel carries the `MT-Fault-*` tags; the transport also records `MT-Original-Channel` so you can trace a dead-lettered message back to its source.

    ```csharp title="DeadLetterInspection/Program.cs"
    cfg.ReceiveEndpoint("inspection-payments", e =>
    {
        e.ConfigureKubeMQ(t => { });
    });

    // Faulted messages with fault metadata
    cfg.ReceiveEndpoint("inspection-payments_error", e =>
    {
        e.ConfigureKubeMQ(t => { });
    });

    // Skipped / undeliverable messages
    cfg.ReceiveEndpoint("inspection-payments_skipped", e =>
    {
        e.ConfigureKubeMQ(t => { });
    });
    ```
  </Tab>
</Tabs>

## Retry Pipeline [#retry-pipeline]

The MassTransit retry and redelivery pipeline runs **before** a message is moved to `_error` — a message faults to the error channel only after all configured retries are exhausted. Retry and redelivery policies are part of the MassTransit middleware pipeline, which is transport-independent: it behaves identically on KubeMQ as it does on any other transport.

```csharp title="RetryPolicy/Program.cs"
cfg.ReceiveEndpoint("retry-demo-queue", e =>
{
    // Native DLQ as the final fallback after retries are exhausted
    e.ConfigureKubeMQ(t => t.UseNativeDlq(5, "retry-demo-dlq"));
});
```

A common pattern is to combine an application-level retry with exponential backoff and the native DLQ as the terminal fallback. The consumer retries transient failures in-process; if every attempt fails, it re-throws so the transport applies the receive-count DLQ rule:

```csharp title="RetryConsumer.cs"
public class RetryConsumer(ILogger<RetryConsumer> logger)
    : IConsumer<RetryableMessage>
{
    // Exponential backoff intervals (ms)
    private static readonly int[] RetryIntervals = [1000, 2000, 4000];

    public async Task Consume(ConsumeContext<RetryableMessage> context)
    {
        Exception? lastException = null;

        for (var attempt = 0; attempt <= RetryIntervals.Length; attempt++)
        {
            try
            {
                // ... process the message; return on success ...
                return;
            }
            catch (Exception ex) when (attempt < RetryIntervals.Length)
            {
                lastException = ex;
                await Task.Delay(RetryIntervals[attempt], context.CancellationToken);
            }
        }

        // All retries exhausted -- re-throw so the native DLQ catches it
        // after maxReceiveCount.
        if (lastException != null)
            throw lastException;
    }
}
```

<Callout type="warn">
  MassTransit's `UseMessageRetry` middleware requires an `IReceiveEndpointConfigurator`, which is not exposed on the KubeMQ receive-endpoint configurator. For per-message retry on the KubeMQ transport, use an application-level retry loop inside the consumer (as above) with `UseNativeDlq` as the terminal fallback.
</Callout>

## Message Expiration [#message-expiration]

Expiration is an error-avoidance tool: messages that age out are discarded by the broker rather than being delivered late and failing downstream. Set a per-endpoint TTL via `k.ExpirationSeconds`, which maps to the underlying `QueueMessage.ExpirationSeconds`. A message not consumed within the TTL is automatically dropped by KubeMQ.

```csharp title="MessageExpiration/Program.cs"
cfg.ReceiveEndpoint("expiration-demo-queue", e =>
{
    // Messages not consumed within 10 seconds are discarded by KubeMQ
    e.ConfigureKubeMQ(t => t.ExpirationSeconds = 10);
});
```

You can also set TTL per message at send time. MassTransit's `TimeToLive` maps to `QueueMessage.ExpirationSeconds`:

```csharp title="send_with_ttl.cs"
await endpoint.Send(new SubmitOrder { OrderId = "123" }, ctx =>
{
    ctx.TimeToLive = TimeSpan.FromHours(1);
});
```

<Callout type="info">
  Expiration applies to Queue messages (`Send`). It complements the DLQ: expiration prevents stale work from ever being processed, while the DLQ captures work that was attempted and failed.
</Callout>

## Transport Exception Hierarchy [#transport-exception-hierarchy]

When the transport itself fails (as opposed to a consumer faulting), it raises a `KubeMQTransportException` or one of its subclasses. All derive from MassTransit's `MassTransitException` and carry the `Channel` and `ServerAddress` involved, when applicable.

<TypeTable
  type="{
  &#x22;KubeMQTransportException&#x22;: {
    type: &#x22;base&#x22;,
    description: &#x22;Base transport exception. Wraps underlying KubeMQ SDK errors. Thrown for general failures — e.g. \&#x22;Queue send failed\&#x22;, \&#x22;Query request failed\&#x22;, \&#x22;Command request failed\&#x22;.&#x22;,
  },
  &#x22;KubeMQTransportConfigurationException&#x22;: {
    type: &#x22;KubeMQTransportException&#x22;,
    description: &#x22;Invalid transport configuration. Thrown during bus startup validation — e.g. empty Host, Port out of range, PollTimeoutSeconds or MaxPollMessages out of range, or delayed delivery on Events.&#x22;,
  },
  &#x22;KubeMQTransportConnectionException&#x22;: {
    type: &#x22;KubeMQTransportException&#x22;,
    description: &#x22;The connection could not be established or was lost. Thrown on \&#x22;Connection closed\&#x22; or gRPC UNAVAILABLE from the server.&#x22;,
  },
  &#x22;KubeMQTransportTimeoutException&#x22;: {
    type: &#x22;KubeMQTransportException&#x22;,
    description: &#x22;A send/publish operation exceeded its configured timeout.&#x22;,
  },
}"
/>

A few representative triggers, drawn from the validation and runtime paths:

* **`KubeMQTransportConfigurationException`** — `Host is invalid: must be non-empty`, `Port is invalid: {value} must be between 1 and 65535`, `Delayed delivery is not supported for Events. Use Queues.` `KubeMQTransportOptions.Validate()` runs automatically at bus startup and raises this.
* **`KubeMQTransportConnectionException`** — `Connection closed` when the client connection cannot be re-established, or a gRPC `UNAVAILABLE` status from the server.
* **`KubeMQTransportTimeoutException`** — a send or publish operation that exceeds the configured timeout (check network latency and `ConnectionTimeout`).
* **`KubeMQTransportException`** (general) — `Queue send failed: {error}`, or `Query request failed: {error}` / `Command request failed: {error}` when a CQ request returns `Executed = false`.

## Messages Not Being Received [#messages-not-being-received]

If consumers go quiet, the cause is usually a pattern or naming mismatch rather than a fault. Work through this checklist:

<Steps>
  <Step>
    **Queues (Send).** Verify a consumer is running and connected. Queue messages persist until consumed, so a missing consumer means a growing backlog, not a fault.
  </Step>

  <Step>
    **Events (Publish).** Verify subscribers are active **before** publishing. Events are fire-and-forget — if no subscriber is active when you publish, the message is silently dropped.
  </Step>

  <Step>
    **EventsStore.** Check the subscription start position. `StartFromNew` (the default) only delivers messages published after the subscription begins; use a replay start position to catch up on history.
  </Step>

  <Step>
    **Consumer-group mismatch.** Competing consumers must share the same endpoint name (the endpoint name is the consumer group). Different endpoint names create independent consumers, not a shared group.
  </Step>
</Steps>

## Debug Logging [#debug-logging]

When the cause isn't obvious, raise the log levels for the transport and the KubeMQ SDK in `appsettings.json`:

```json title="appsettings.json"
{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "MassTransit": "Debug",
      "MassTransit.KubeMQTransport": "Debug",
      "KubeMQ": "Debug"
    }
  }
}
```

What to look for in the output:

* **Connection state changes** — `ConnectionState` transitions in the KubeMQ SDK logs (`Idle → Connecting → Ready`, or `Reconnecting`) tell you whether the client is actually connected.
* **Poll-loop activity** — the queue receive transport logs each poll cycle; empty polls increment the `kubemq.transport.poll.empty` counter, which is a quick way to confirm the receiver is alive but idle.
* **Send / publish durations** — the transport records `kubemq.transport.send.duration` and `kubemq.transport.publish.duration`, so slow or failing operations show up here before they surface as exceptions.

## Related [#related]

* [Observability](/integrations/masstransit/how-to/observability) — OpenTelemetry metrics and counters, including `kubemq.transport.errors` for transport-level failure counts and health-check states.
* [Error codes](/integrations/masstransit/reference/error-codes) — the transport exception hierarchy and the `_error` / `_skipped` channel naming rules.
* [Concepts](/integrations/masstransit/concepts) — how Send, Publish, and Request/Response map to KubeMQ channels and where dead-letter routing fits.