# Partitions & Ordering (/connectors/kafka/concepts/partitions-and-ordering) Every Kafka topic on KubeMQ is split into one or more **partitions** — independent, ordered logs that parallelize produce and consume. A topic's partition count starts at 1, only ever grows, and is hard-capped at 256. Partition **assignment** — which partition a keyed record lands in — is decided entirely by the client, never by KubeMQ, which makes mixing client libraries on the same keyed topic a real footgun worth understanding before you rely on per-key ordering. ## Overview [#overview] A topic with `N` partitions is `N` independent ordered logs, each backed by its own Events Store channel (`kafka.~` for partition ≥ 1; partition 0 lives at `kafka.`). Producers and consumers parallelize across those `N` logs, but KubeMQ's ordering guarantee — like real Kafka's — only ever applies **within** a single partition. There is no cross-partition ordering, by design: that is exactly the trade KubeMQ's Kafka connector makes to let many producers and many consumers work a topic concurrently. ## Partition count: 1 to 256, increase-only [#partition-count-1-to-256-increase-only] A newly created topic defaults to a single partition (`NumPartitions` omitted, or set below 1, is treated as 1). From there, the only way to change the partition count is `CreatePartitions` — and it is deliberately **increase-only**: | Request | Result | | ------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `CreatePartitions` with `Count` > current count, ≤ 256 | Accepted — the partition count durably grows, and every consumer group already subscribed to the topic is triggered into a rebalance so members pick up the new partitions | | `CreatePartitions` with `Count` == current count | Rejected — `INVALID_PARTITIONS` (error code 37), "topic already has N partition(s)" | | `CreatePartitions` with `Count` \< current count | Rejected — `INVALID_PARTITIONS` (37), "partition count must be >= current" — **partitions never shrink** | | `CreatePartitions` with `Count` > 256 | Rejected — `INVALID_PARTITIONS` (37), "Count exceeds the maximum of 256" | The 256 ceiling is a hard, non-configurable constant in the connector — there is no setting that raises it. A `ValidateOnly` dry-run is honored for all of the above: it reports the same accept/reject outcome without making a durable change. This increase-only design is deliberate, not a missing feature. KubeMQ never auto-grows a topic's partition count in the background — every increase is an explicit, operator-triggered `CreatePartitions` call, so a partition-count change is always a visible event rather than a silent background re-shard. See [Limits & Rules](/connectors/kafka/reference/limits-and-rules) for the full numeric ceiling table and [Topic Mapping](/connectors/kafka/reference/topic-mapping) for exactly how partitions map onto Events Store channels. ## Client-side key hashing [#client-side-key-hashing] **KubeMQ never hashes a record key to choose a partition.** Every partition assignment for a keyed produce is decided **client-side** and relayed to KubeMQ opaquely — there is no server-side hashing code in the connector, ever. If a client sets an explicit partition (or uses a manual partitioner), that partition is honored verbatim; if it leaves partitioning to the library's default, the library's own `hash(key) % N` decides. This matches real Apache Kafka's own architecture, but it means the **partition a key lands in depends on which client library produced it** — different libraries ship different default partitioners: | Client family | Default partitioner | Examples | | ------------------------------ | ------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | Java `kafka-clients`, franz-go | murmur2-based hash | Java `KafkaProducer`; franz-go's `UniformBytesPartitioner`, which falls back to a murmur2 hasher when no custom hasher is set | | librdkafka and its bindings | CRC32 | `kcat`, `confluent-kafka` (Python), and other librdkafka-based clients | Java and franz-go agree with each other out of the box (same hash family), but a librdkafka-based client hashes the **same key** to a **different partition** than a Java or franz-go client would, on the same topic, with the same `N`. **Mixing producer libraries on the same keyed topic can split a key across partitions.** If one service produces with `kcat` or another librdkafka-based client, and another produces with Java or franz-go, the same key can land in two different partitions — silently. Per-key ordering only holds **within one client library's own partitioning scheme**; it is not guaranteed across two different ones. Before you rely on "same key, same partition" across services, verify every producer uses the same partitioner strategy (or route through a single producer library for that topic), or design consumers to reconcile order at the application layer with a monotonic per-key sequence embedded in the record itself. *The same key, hashed by two different client libraries' default partitioners, can land on two different partitions of the same topic — KubeMQ relays the client's partition choice opaquely and never re-hashes it.* ## The per-partition ordering guarantee [#the-per-partition-ordering-guarantee] A single partition is one ordered writer: every record a producer sends to that partition is appended in the order it arrived, and a consumer reading that partition in isolation sees records in exactly that order. A single-partition topic is therefore **totally ordered** — every record, across every producer, in one sequence. The moment a topic has `N > 1` partitions, ordering is guaranteed **only within each partition**; there is no ordering guarantee across partitions, and a consumer that needs a total order across the whole topic must read a single partition (or reconcile order itself using a per-key sequence in the record). Partitions are also KubeMQ's unit of consume parallelism: a consumer group can have at most one active consumer per partition at a time (see [Consumer Groups](/connectors/kafka/concepts/consumer-groups)), so growing `N` is how you add parallel consume capacity — at the cost of the ordering trade above. ### Growing a topic re-shards keys [#growing-a-topic-re-shards-keys] Because partition assignment is `hash(key) % N`, changing `N` changes the modulus every keyed partitioner uses — client-side murmur2, CRC32, or any custom scheme. A key that mapped to partition 1 under `N=3` can map to a completely different partition once the topic grows to `N=6`. Concretely: if all of a key's records went to partition 1 while `N=3`, the very next record for that same key, produced after a successful increase to `N=6`, might land on partition 4 instead — a consumer reading only partition 1 will not see the two batches as one continuous ordered stream. The practical takeaway: **a topic's per-key ordering guarantee holds only within one fixed-partition-count epoch** — from one partition count to the next `CreatePartitions` increase. This is exactly why `CreatePartitions` is increase-only and never automatic: an operator-triggered increase is a visible, deliberate boundary between ordering epochs, not something that happens silently underneath a running producer. If a topic's consumers depend on strict cross-epoch per-key order, either avoid growing `N` on that topic, or design the consumer to reconcile order at the application layer. One related, self-healing edge case: right after a leader-side partition-count increase in a clustered deployment, a follower node that has not yet caught up on replication can briefly answer `Metadata` with a stale, **lower** partition count than the true value. This mirrors real Apache Kafka's own controller-to-broker propagation window — it never advertises a count larger than the true one, so a client is never misdirected to a partition that does not exist, and the client's next metadata refresh self-heals it. ## Related [#related]