# Consumer Groups (/connectors/kafka/concepts/consumer-groups)
## Overview [#overview]
The Kafka connector implements Kafka's **classic** consumer-group protocol — the same
`FindCoordinator` → `JoinGroup` → `SyncGroup` → `Heartbeat` → `LeaveGroup` sequence every
mainstream Kafka client library (`kafka-clients`, `librdkafka`, `franz-go`, `sarama`) already
speaks by default. A group of consumers subscribing to the same topic get their partitions
divided among them automatically, their progress recorded durably per group, and their
membership rebalanced whenever the group's shape changes. None of this needs a KubeMQ-specific
concept: `group.id` is the only setting a client sets, exactly as it would against real Kafka.
Kafka also defines a newer, broker-side **KIP-848** next-generation group protocol
(`ConsumerGroupHeartbeat`/`ConsumerGroupDescribe`). The connector does not advertise it — only
classic groups run today. A client configured with `group.protocol=consumer` stalls on connect;
set `group.protocol=classic` — still every mainstream client's default — to use the protocol
this page describes.
## The classic group protocol [#the-classic-group-protocol]
Five API calls carry the whole membership lifecycle:
* **FindCoordinator** locates the broker that owns a given group's state — on a standalone
deployment, trivially the node itself.
* **JoinGroup** is how a consumer enters the group: it names the group, the subscribed topics,
and the assignment strategies it supports. The coordinator collects every member's JoinGroup
and designates one of them the **group leader**. A first-time dynamic join without a member ID
is briefly turned back with `MEMBER_ID_REQUIRED` (error code 79) and asked to retry with the ID
the broker just minted — a two-round handshake that stops a network-retry storm from
registering duplicate phantom members. Static membership, below, is the one case that skips
this round-trip entirely.
* **SyncGroup** is where partitions actually get assigned. The elected leader — not the
coordinator — computes the assignment locally, using whichever strategy the group agreed on
(range, round-robin, sticky, and so on), and sends that full assignment back through its own
SyncGroup call. The coordinator then hands each *other* member its slice of that same
assignment when they call SyncGroup. Assignment is therefore **leader-authoritative**: the
broker distributes an assignment it did not compute.
* **Heartbeat** keeps a member's membership alive between rebalances; the coordinator times a
member out — and removes it — if heartbeats stop arriving within the group's session timeout.
* **LeaveGroup** is the explicit, immediate departure a client sends on a clean shutdown, so the
group doesn't have to wait out a session timeout to notice the member is gone.
Two more calls sit alongside this lifecycle rather than inside it: **OffsetCommit** and
**OffsetFetch** persist and retrieve a group's position, covered on its own below because it
survives across resets of the group protocol itself.
## Leader-authoritative assignment and generations [#leader-authoritative-assignment-and-generations]
Every completed rebalance bumps the group's **generation** — a monotonically increasing counter
that fences stale requests. A member that heartbeats or commits using a generation
number the coordinator no longer recognizes is answered with `ILLEGAL_GENERATION` (error code 22)
rather than silently accepted; this is what stops a member that missed a rebalance from acting on
an assignment that no longer applies. Because assignment itself is leader-computed (above), the
coordinator's own responsibility across a generation is comparatively narrow: collect membership,
forward the leader's assignment to everyone else, and track which generation is current.
## Durable per-group offsets [#durable-per-group-offsets]
A consumer group's committed position — for every topic-partition it consumes — is stored
**durably**, independent of the group's membership lifecycle. Committing an offset, whether
automatically on an interval or explicitly after processing a record, survives every consumer in
the group restarting, the group's coordinator failing over to another node, and the group itself
going empty and refilling later with a different set of members. See
[Consuming](/connectors/kafka/how-to/consuming) for the manual-vs-automatic commit
mechanics.
## Rebalancing [#rebalancing]
A rebalance reruns JoinGroup/SyncGroup across the whole group, and is triggered by anything that
changes what "correct assignment" means:
* A member joins or leaves — explicitly, via `LeaveGroup`, or implicitly, via a session-timeout.
* A subscribed topic's partition count changes. Growing a topic's partitions is an explicit,
operator-triggered action on KubeMQ — partitions only ever increase, never shrink or
auto-reshard — and doing so deliberately triggers every subscribed group to rebalance onto the
new partition count, rather than leaving some partitions unassigned.
A full rebalance briefly pauses processing for the whole group while JoinGroup/SyncGroup rerun —
the cost static membership (below) exists to avoid paying on every routine restart.
## Static membership (KIP-345) [#static-membership-kip-345]
A consumer that sets `group.instance.id` gets a **persistent identity** the coordinator
remembers across disconnects, instead of a fresh, disposable member ID every time it joins:
* **Identity, not just a request field.** The coordinator maps that instance ID to a specific
member ID once and reuses the mapping on every subsequent join from the same instance — the
client doesn't need to cache and resend a member ID itself.
* **Static join skips the extra round-trip.** A dynamic (non-static) member without a member ID
gets `MEMBER_ID_REQUIRED` back, as described above, and has to retry with the ID the broker just
handed it. A static member's very first `JoinGroup` — carrying `group.instance.id` — resolves
straight to its known member ID and is admitted immediately, with no forced retry.
* **A clean rejoin skips the rebalance entirely.** If the group is already stable, its leader
hasn't changed, and a static member rejoins with the same subscription it had before — the
common case: the process crashed and restarted, or reconnected after a network blip — the
coordinator answers at the **same generation** with no rebalance at all. Every other member in
the group is undisturbed. (One exception: if the rejoining instance is the group's current
leader, a full rebalance still runs — the fast path is taken only for non-leader members.)
* **A displaced or mismatched instance is fenced.** If a second connection shows up claiming an
instance ID that's already mapped to a *different* member ID than the one now presenting it —
the classic double-start-during-a-restart shape — the coordinator refuses it with
`FENCED_INSTANCE_ID` (error code 82) rather than quietly admitting a second writer under the
same identity.
Static membership is supported in full — every one of the four behaviors above holds across
`JoinGroup`, `SyncGroup`, `Heartbeat`, `OffsetCommit`, and `LeaveGroup`. It's the mechanism that
makes a rolling restart of a consumer fleet cheap: each pod restarts with the same
`group.instance.id` it had before, and the group skips a rebalance for the restart itself —
unless the restarting pod is the group's current leader, which still triggers one.
Kafka also has a second, queue-style consumption model — **share groups** (KIP-932) — where
records are individually acquired and acknowledged instead of partition-assigned. It's a
genuinely different model from everything on this page, not a variant of classic groups; see
[Share Groups](/connectors/kafka/how-to/share-groups) for the contrast.
## Related [#related]