# Scaling & Flow Control (/learn/concepts/scaling-and-flow)



When work piles up faster than one worker can handle it, you have two completely different levers — and reaching for the wrong one quietly breaks your system. This page is about telling them apart: **scaling out** (adding workers to share the load) and **flow control** (keeping a fast producer from drowning a slow consumer).

Picture a busy coffee shop. To serve more customers, you put more baristas behind **one** counter — each drink order goes to whichever barista is free. That is scaling by *competing consumers*. Now picture the shop's radio: every barista hears the same announcement, no matter how many you hire. That is *fan-out*. Adding baristas speeds up the counter; it does nothing to the radio. Confusing the two is how you end up processing every order three times — or building a "load balancer" that never balances.

## Competing consumers vs fan-out — the idea [#competing-consumers-vs-fan-out--the-idea]

Both shapes start with one stream of messages and several consumers. The difference is **who gets each message**.

**Competing consumers** (point-to-point): the consumers share a single logical destination, and each message is handed to **exactly one** of them. Add a consumer and total throughput goes up, because the work is split. This is how you scale processing.

**Fan-out** (pub/sub): every consumer is an independent subscriber, and each message is **copied to all** of them. Add a subscriber and you get one more *full copy* of the stream — useful for independent reactions, not for sharing load.

<Mermaid
  chart="graph LR
  P[&#x22;Producer&#x22;]
  Q[[&#x22;Shared destination<br/>(competing consumers)&#x22;]]
  W1[&#x22;Worker A&#x22;]
  W2[&#x22;Worker B&#x22;]
  W3[&#x22;Worker C&#x22;]

  P -- send --> Q
  Q -- &#x22;1 of 3&#x22; --> W1
  Q -- &#x22;1 of 3&#x22; --> W2
  Q -- &#x22;1 of 3&#x22; --> W3
  W1 -. ack .-> Q

  class Q queue
  class P,W1,W2,W3 client"
/>

*Competing consumers: one destination, each message to exactly one worker — add workers to share the load.*

<Mermaid
  chart="graph LR
  PUB[&#x22;Publisher&#x22;]
  EV{{&#x22;Broadcast channel<br/>(fan-out)&#x22;}}
  S1[&#x22;Subscriber A&#x22;]
  S2[&#x22;Subscriber B&#x22;]
  S3[&#x22;Subscriber C&#x22;]

  PUB -- publish --> EV
  EV -- &#x22;copy&#x22; --> S1
  EV -- &#x22;copy&#x22; --> S2
  EV -- &#x22;copy&#x22; --> S3

  class EV events
  class PUB,S1,S2,S3 client"
/>

*Fan-out: one channel, every subscriber gets its own copy — add subscribers for independent reactions, not for throughput.*

### Consumer groups: getting both from one channel [#consumer-groups-getting-both-from-one-channel]

What if you want load-balancing **and** broadcast on the same channel? A **consumer group** is the bridge. Each consumer declares a group name when it subscribes. Within a group, members compete — a message goes to exactly one of them. Across groups, each group gets its own copy.

So three workers in group `billing` split the stream between them, while a separate `analytics` group (its own members) receives the full stream in parallel. One channel, two behaviors, decided entirely by group membership.

<Callout type="info">
  The group mechanics on this page apply to both Events and Events Store, but durability does not: **Events** groups are fire-and-forget — a member that is offline simply misses messages, with nothing to replay. **Events Store** groups are durable and position-tracked — the store remembers each group's progress, so a member that reconnects resumes from where it left off instead of losing messages. The example below uses plain Events; see [In KubeMQ](#in-kubemq) for the Events Store distinction in full.
</Callout>

<Mermaid
  chart="graph LR
  PUB[&#x22;Publisher&#x22;]
  CH{{&#x22;Channel<br/>order-events&#x22;}}
  B1[&#x22;billing-1&#x22;]
  B2[&#x22;billing-2&#x22;]
  A1[&#x22;analytics-1&#x22;]

  PUB -- publish --> CH
  CH -- &#x22;1 of group&#x22; --> B1
  CH -- &#x22;1 of group&#x22; --> B2
  CH -- &#x22;full copy&#x22; --> A1

  class CH events
  class PUB,B1,B2,A1 client"
/>

*Group `billing` competes for messages (load-balanced); group `analytics` gets its own full copy — group name decides the behavior.*

## Backpressure and slow consumers [#backpressure-and-slow-consumers]

Scaling out assumes consumers can keep up. When they cannot — a producer bursts, a downstream API slows down, a worker stalls on a long job — the system needs a way to push back. That feedback is **backpressure**: signalling upstream to slow down (or buffer, or shed load) so a fast producer does not overwhelm a slow consumer.

Without backpressure, the gap has to go *somewhere*, and every option is bad: an unbounded in-memory buffer grows until the process runs out of memory; a fixed buffer overflows and silently drops messages; or the producer blocks and the whole pipeline stalls. The healthy outcome is the producer feeling resistance and easing off — exactly like water pressure backing up a pipe when the drain is too small.

<Mermaid
  chart="graph LR
  P[&#x22;Fast producer&#x22;]
  BUF[[&#x22;Buffer / queue<br/>filling up&#x22;]]
  C[&#x22;Slow consumer&#x22;]

  P -- &#x22;send (fast)&#x22; --> BUF
  BUF -- &#x22;deliver (slow)&#x22; --> C
  BUF -. &#x22;backpressure: slow down&#x22; .-> P

  class BUF queue
  class P,C client"
/>

*A buffer absorbs short bursts; when it fills, backpressure flows back to the producer so it eases off instead of overflowing.*

A durable queue is itself a form of backpressure-by-buffering: it absorbs bursts on disk so producers never block on slow consumers, and you drain the backlog by adding more competing workers. A broadcast (fire-and-forget) channel has no such buffer — a subscriber that cannot keep up simply misses messages.

### Visibility timeout and in-flight messages [#visibility-timeout-and-in-flight-messages]

There is a subtler flow-control problem hiding inside competing consumers: what happens to a message **while** a worker is processing it? If the destination handed the same message to a second worker, you would process it twice. If it deleted the message immediately on delivery, a crash mid-processing would lose it.

The standard answer is the **visibility timeout**. When a worker receives a message, the message is not deleted — it is hidden from other consumers for a bounded window and counts as **in-flight**. The worker has until the timeout to finish and acknowledge (ack), which deletes it. If the worker crashes or the timeout expires first, the message becomes visible again and is redelivered to another worker.

<Mermaid
  chart="stateDiagram-v2
  [*] --> Available
  Available --> InFlight: received (hidden)
  InFlight --> Done: ack within timeout
  InFlight --> Available: timeout / crash (redeliver)
  Done --> [*]"
/>

*Visibility timeout: a received message is in-flight and hidden; ack within the window deletes it, otherwise it reappears for another worker.*

The timeout is a balance. Too short, and a legitimately slow job gets redelivered (and processed twice) before it finishes. Too long, and a crashed worker's messages sit invisible for ages before anyone retries them. The limit on in-flight messages also caps real concurrency: a destination only lets so many messages be in-flight at once, which is itself a backpressure knob.

## Precise definitions [#precise-definitions]

* **Competing consumers (point-to-point):** a distribution model where multiple consumers read from one shared destination and each message is delivered to exactly one consumer. Throughput scales with the number of consumers.
* **fan-out (pub/sub):** a distribution model where each message is copied to every independent subscriber on a channel. Adding subscribers adds parallel copies, not shared load.
* **Consumer Group:** a named set of consumers on one channel that compete as a unit — each message goes to one member of the group, while every group on the channel receives its own copy.
* **backpressure:** flow control that signals a producer to slow down (or buffer, or shed) when consumers cannot keep up, preventing overflow and loss.
* **Visibility timeout:** the bounded window during which a received-but-unacknowledged message is hidden from other consumers and counts as in-flight; on expiry without an ack it is redelivered.
* **In-flight message:** a message that has been delivered to a consumer but not yet acknowledged — held, not deleted, so it can be redelivered if processing fails.

## Trade-offs [#trade-offs]

| Goal                            | Reach for                     | Why                                              | Watch out for                                                            |
| ------------------------------- | ----------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------ |
| Process more, faster            | Competing consumers / a group | Each message done once; add workers to scale     | One slow worker holds its in-flight messages until timeout               |
| React independently in N places | fan-out / separate groups     | Every consumer sees every message                | Adds load, not throughput — N copies of the work                         |
| Absorb bursts without dropping  | Durable queue (buffer)        | Disk soaks up the spike; drain with more workers | Backlog grows if consumers stay too slow — monitor depth                 |
| Don't overwhelm a slow consumer | Backpressure                  | Producer eases off instead of overflowing        | Fire-and-forget channels have no buffer — slow subscribers miss messages |
| Survive crashes mid-processing  | Visibility timeout + ack      | Unacked work reappears for another worker        | Wrong timeout → double-processing (too short) or stalls (too long)       |

<Callout type="warn">
  **Pitfall — "scaling" a broadcast.** Adding subscribers to a fire-and-forget channel does **not** share the load: every subscriber still receives every message, so you multiply the work instead of dividing it. To actually scale processing, put the consumers in the same group so they compete for messages.
</Callout>

<Callout type="warn">
  **Pitfall — designing for exactly-once consumers.** Visibility-timeout redelivery means a consumer can see the same message more than once (a slow job, a crash, an expired timeout). Make handlers **idempotent** so a redelivery is harmless rather than betting on never seeing a duplicate.
</Callout>

## In KubeMQ [#in-kubemq]

KubeMQ exposes both levers directly:

* **Fan-out vs competing consumers is one parameter.** On Events and Events Store, subscribers that pass the **same group name** compete (each message to one member); subscribers with **no group** (or different groups) each get a full copy. Same channel, different `group` argument.
* **Queues give you the buffer and the visibility timeout.** A queue durably stores messages, so producers never block on slow consumers — you scale by running more receivers, and each received message is hidden for its **visibility timeout** until you ack it, then redelivered if you don't.

The snippet below is the same load-balanced subscribe from the [Events Consumer Groups](/learn/events/tutorials/consumer-groups) tutorial: several consumers join one **group** on `order-events`, and KubeMQ delivers each event to exactly one of them. Drop the group name and the very same subscribers turn into fan-out.

<Tabs groupId="language" items="['Go','Python','Node.js','Java','C#','Kotlin','C++','Rust','Ruby','Elixir']">
  <Tab value="Go">
    ```go title="grouped_worker.go"
    sub, err := client.SubscribeToEvents(ctx, "order-events", "workers",
        kubemq.WithOnEvent(func(event *kubemq.Event) {
            fmt.Printf("Processing: %s\n", string(event.Body))
        }),
    )
    if err != nil {
        log.Fatal(err)
    }
    defer sub.Unsubscribe()
    // Members sharing group "workers" compete; pass "" for fan-out instead.
    ```
  </Tab>

  <Tab value="Python">
    ```python title="grouped_worker.py"
    from kubemq.pubsub import Client as PubSubClient
    from kubemq.pubsub import EventsSubscription, CancellationToken

    client = PubSubClient(address="localhost:50000")
    client.subscribe_to_events(
        subscription=EventsSubscription(
            channel="order-events",
            group="workers",  # same group -> competing consumers; omit for fan-out
            on_receive_event_callback=lambda e: print(f"Processing: {e.body.decode()}"),
        ),
        cancel=CancellationToken(),
    )
    ```
  </Tab>

  <Tab value="Node.js">
    ```javascript title="grouped_worker.js"
    const { KubeMQClient } = require("kubemq-js");

    const client = new KubeMQClient({ address: "localhost:50000" });
    client.subscribeToEvents({
      channel: "order-events",
      group: "workers", // same group -> competing consumers; omit for fan-out
      onEvent: (msg) =>
        console.log(`Processing: ${Buffer.from(msg.body).toString()}`),
      onError: (err) => console.error(err.message),
    });
    ```
  </Tab>

  <Tab value="Java">
    ```java title="GroupedWorker.java"
    PubSubClient client = PubSubClient.builder()
        .address("localhost:50000")
        .clientId("worker-1")
        .build();

    client.subscribeToEvents(EventsSubscription.builder()
        .channel("order-events")
        .group("workers") // same group -> competing consumers; omit for fan-out
        .onReceiveEventCallback(event ->
            System.out.printf("Processing: %s%n", new String(event.getBody())))
        .build());
    ```
  </Tab>

  <Tab value="C#">
    ```csharp title="GroupedWorker.cs"
    await using var client = new KubeMQClient(new KubeMQClientOptions());
    await client.ConnectAsync();

    await foreach (var msg in client.SubscribeToEventsAsync(
        new EventsSubscription { Channel = "order-events", Group = "workers" }))
    {
        // same Group -> competing consumers; leave Group unset for fan-out
        Console.WriteLine($"Processing: {Encoding.UTF8.GetString(msg.Body.Span)}");
    }
    ```
  </Tab>

  <Tab value="Kotlin">
    ```kotlin title="GroupedWorker.kt"
    val client = PubSubClient("localhost:50000")

    client.subscribeToEvents(
        channel = "order-events",
        group = "workers", // same group -> competing consumers; omit for fan-out
        onEvent = { event -> println("Processing: ${String(event.body)}") },
        onError = { err -> System.err.println(err.message) },
    )
    ```
  </Tab>

  <Tab value="C++">
    ```cpp title="grouped_worker.cpp"
    auto client = kubemq::PubSubClient("localhost:50000");

    // same group ("workers") -> competing consumers; pass "" for fan-out
    client.subscribeToEvents("order-events", "workers",
        [](const kubemq::Event& event) {
            std::cout << "Processing: " << event.body << std::endl;
        },
        [](const std::string& err) { std::cerr << err << std::endl; }
    );
    ```
  </Tab>

  <Tab value="Rust">
    ```rust title="grouped_worker.rs"
    let group = "workers"; // same group -> competing consumers; "" for fan-out
    let sub = client
        .subscribe_to_events(
            "order-events",
            group,
            |event| {
                Box::pin(async move {
                    println!("Processing: {}", String::from_utf8_lossy(&event.body));
                })
            },
            None,
        )
        .await?;
    ```
  </Tab>

  <Tab value="Ruby">
    ```ruby title="grouped_worker.rb"
    client = KubeMQ::PubSubClient.new(address: "localhost:50000", client_id: "worker-1")
    cancel = KubeMQ::CancellationToken.new

    # group: -> competing consumers; omit group: for fan-out
    sub = KubeMQ::PubSub::EventsSubscription.new(channel: "order-events", group: "workers")
    client.subscribe_to_events(sub, cancellation_token: cancel) do |event|
      puts "Processing: #{event.body}"
    end
    ```
  </Tab>

  <Tab value="Elixir">
    ```elixir title="grouped_worker.exs"
    {:ok, client} = KubeMQ.Client.start_link(address: "localhost:50000", client_id: "worker-1")

    # group: -> competing consumers; omit group: for fan-out
    {:ok, _sub} =
      KubeMQ.Client.subscribe_to_events(client, "order-events",
        group: "workers",
        on_event: fn event -> IO.puts("Processing: #{event.body}") end
      )
    ```
  </Tab>
</Tabs>

The same group switch applies to **Events Store** (durable, position-tracked groups) and to **Queues**, where competing receivers share the buffer and each delivery is governed by a visibility timeout you ack to clear.

### How KubeMQ does this [#how-kubemq-does-this]

<Cards>
  <Card title="Events Consumer Groups" href="/learn/events/tutorials/consumer-groups" description="Load-balance a real-time channel across a competing-consumer group." />

  <Card title="Scale Subscribers" href="/learn/events/how-to/scale-subscribers" description="Horizontal scaling strategies: groups, mixed group + monitor subscribers." />

  <Card title="Handle Slow Consumers" href="/learn/events/how-to/handle-slow-consumers" description="Keep a fast publisher from overwhelming subscribers that fall behind." />

  <Card title="Visibility Timeout" href="/learn/queues/how-to/visibility-timeout" description="In-flight messages, hiding windows, and redelivery on Queues." />

  <Card title="Extend Visibility" href="/learn/queues/how-to/extend-visibility" description="Hold a long-running job in-flight without it being redelivered." />

  <Card title="Retry with Backoff" href="/learn/queues/how-to/retry-with-backoff" description="Pace redelivery so a struggling consumer can recover." />

  <Card title="Events Store Consumer Groups" href="/learn/events-store/tutorials/consumer-groups" description="Durable, position-tracked competing consumers over persisted streams." />
</Cards>