Release 3.6.0

Released September 28, 2026. A clustered next storage engine now spreads partition leadership across every node by default, instead of pinning it to one, so Kafka produce and fetch traffic is served by the whole cluster. The layout change behind it means a next-engine data directory from 3.5.5 or earlier cannot be opened — read Before you upgrade first. The operator moves to v3.5.0 and the Helm chart kubemq-next to 3.5.0.

Artifacts

Before you upgrade

• Breaking: every next-engine data directory needs a fresh volume. 3.6.0 writes and opens a new on-disk layout that makes spreading partition leadership possible, and it cannot open a directory a 3.5.5-or-earlier server wrote. There is no migration and no compatibility mode: a node started on an old directory refuses to boot with "this version needs a fresh data directory; delete the old one or start on a new volume," and nothing is migrated or deleted, even with Store.CleanStore=true (--clean / STORE_CLEAN_STORE). Drain or export any messages you still need first, then start every node on an empty directory or a new volume. Legacy-engine (STORE_ENGINE=legacy) data directories are not affected.
• Two Kafka fixes stay off until every node runs 3.6.0. CONNECTORS_KAFKA_SHARE_FEATURE_VERSION=3 and CONNECTORS_KAFKA_PRODUCE_CONDITIONAL_APPEND=true each write a replicated command a node on an older release cannot apply, so both default off and a rolling upgrade with default settings is safe. Once every node runs 3.6.0, set both on every node and restart the nodes one at a time. Gate 3 stops a share consumer from losing a producer batch's unacknowledged records when it acknowledges that batch across several requests, and stops a failed read from spending a delivery attempt it should not; conditional append stops a produce batch that committed after its leader stopped waiting for it, or that a former leader forwarded after losing leadership, from taking an offset another batch already holds.

Changes

• A clustered next engine spreads partition leadership across every node by default. A cluster now runs 8 shards with the leader balancer on, so each node leads a share of the partitions and serves their produce and fetch traffic; a standalone node runs 1 shard. Both stay configurable through STORE_NEXT_SHARDS (1–16; every node in a cluster must agree, and changing it means creating a new cluster) and STORE_NEXT_BALANCE_LEADERS (false pins every leader to one node, as before) — see Next storage engine settings. On Kubernetes, the operator's spec.store.nextShards and spec.store.nextBalanceLeaders fields, and the matching fields in the kubemq-next chart, are immutable once a cluster is created.
• Kafka group ids and transactional ids may contain any character, as in Apache Kafka. A route-separator character such as ; used to split an id's durable writes across the wrong internal channels, and a space or a handful of other characters were refused outright. Such ids are now stored under escaped internal channel names; every other id is unaffected.
• Kafka share groups (KIP-932) close out their remaining GA gaps. A share consumer that acknowledges one producer batch across several requests (because it polls fewer records per call than the batch holds) no longer loses the records it releases or never gets to; a failed read after an acquire no longer spends a delivery attempt; and compacted topics stay consistent across replicas. Both fixes ship behind the CONNECTORS_KAFKA_SHARE_FEATURE_VERSION=3 gate above.
• Kafka leader failover no longer crashes Java consumers. A partition that could not yet be served answered Fetch with a code Apache Kafka never sends, which the Java client (kafka-clients 4.3.1 and 4.4.0) treats as fatal and stops the consumer thread on. Fetch, ListOffsets, OffsetForLeaderEpoch, Produce and DeleteRecords now answer the retriable code Kafka uses in that state, and every client refreshes its metadata and retries instead of failing.
• Kafka consumer groups can keep their members across a coordinator failover (new setting, off by default). A moved coordinator used to force every member to rejoin with a new member id and jump the group's generation, which could turn a stale offset commit into a spurious error. CONNECTORS_KAFKA_CLASSIC_GROUP_MEMBERSHIP_DURABLE=true has the next coordinator reinstate each group's members, generation and assignments instead.
• A killed next node no longer loses an acknowledged write, and a killed follower restarts instead of crash-looping. Under the default Store.NextAckPolicy=fast, an acknowledged write is now always written to the operating system before the acknowledgment, with only its fsync staying in the background — see Ack policy. A follower that still lost acknowledged entries to a crash now restarts not ready and heals itself on the shard's next write, rather than panicking on every restart; a node left holding writes the rest of the cluster lost now refuses to serve them, with the cause on /ready, instead of stalling silently.
• A native Events Store group subscriber on a compacted Kafka topic no longer stalls. Once compaction removed a superseded record, every batch that included its sequence used to fail as a whole and hold up the retained records in it. The subscriber now delivers every retained record once and skips the removed sequences, the way a Kafka consumer of a compacted partition does.
• Coordinator state survives a failover that lands mid-roll. A Kafka consumer group's, transactional producer's or topic's replicated state log now waits for its periodic roll to finish before acknowledging a write during that roll, so a coordinator failover or restart at the wrong moment no longer rolls a committed offset, a transaction's state or a topic's configuration and partition count back to an older snapshot.
• A round of smaller Kafka correctness fixes: ListOffsets by timestamp answers from each record's own timestamp instead of when it was stored; a clustered topic reports every node holding its data as a replica, not only the leader; an idempotent producer's retry after a leader failover now reports the offset its batch was actually written at; acks=0 records a producer sent right before closing its connection are no longer dropped; and kubemq_kafka_produce_rejects_total gets its own series for a fenced or unknown producer instead of folding those into generic failures.
• /api/cluster-snapshot names the node raft says leads, instead of each node's own snapshot, which could be tens of seconds stale right after a failover.

Known issues

• No default connector image is published. Set spec.image on every KubemqConnector.