Connections & Observability
The Kafka connector's observability surface — the /api/kafka/* dashboard endpoints, Prometheus metric families, and the per-connection principal view.
Overview
This reference documents the Kafka connector's observability surface: the read-only dashboard
endpoints under /api/kafka/*, the Prometheus metric families, and the per-connection
view the dashboard renders for every live wire connection.
Dashboard endpoints
Every route below is GET-only and requires at least the read-only dashboard role when RBAC
auth is enabled (auth is off by default on a single-node install). List routes always answer
200 with an empty list — and "enabled": false — before the connector is turned on or wired;
detail routes answer 404 for an unknown ID.
| Endpoint | Returns |
|---|---|
/api/kafka/overview | the cluster-combined KPI band — connections, groups, consumers, topic/partition counts, lifetime produced/fetched messages and bytes, total consumer-group lag, and produce/fetch rates |
/api/kafka/connections | the node-local list of live wire connections |
/api/kafka/connections/:id | one connection's detail — see Per-connection view below |
/api/kafka/topics | the topic list, each with its partition list |
/api/kafka/topics/:topic | one topic's detail — per-partition high-watermark, log-start offset, and cleanup.policy |
/api/kafka/groups | the consumer-group list — protocol type, state, committed offsets, and lag |
/api/kafka/groups/:id | one group's detail |
/api/kafka/consumers | live group members, flattened across every group |
/api/kafka/charts?time_range=&time_zone= | the connector-wide produce/fetch throughput series the dashboard's chart polls |
The overview's partitions and topics counts are always the real per-topic partition-list
sums — the same data /api/kafka/topics returns — never a topic-count stand-in. Groups,
consumers, and connection counts are answered from the cluster-combined snapshot (summed across
live nodes). The /api/kafka/connections list, by contrast, is node-local — each wire
connection lives on whichever node accepted it.
Prometheus metrics
The connector registers seven kubemq_kafka_* metric families, scraped like every other
connector's metrics:
| Metric | Labels | Meaning |
|---|---|---|
kubemq_kafka_operations_total | operation | total Kafka connector operations (produce/fetch), by operation |
kubemq_kafka_operation_avg_latency_ms | operation | average latency in milliseconds, by operation |
kubemq_kafka_connections | — | current number of open Kafka wire connections |
kubemq_kafka_messages_total | op (produced|fetched) | total produced/fetched messages |
kubemq_kafka_message_bytes_total | op (produced|fetched) | total produced/fetched wire bytes |
kubemq_kafka_produce_rejects_total | error_code | total produce rejects, by Kafka error code — see Error Codes |
kubemq_kafka_consumer_group_lag | group, topic, partition | current lag (log-end-offset minus committed offset), per group/topic/partition |
kubemq_kafka_operations_total and kubemq_kafka_operation_avg_latency_ms are pull-derived from
a live snapshot at scrape time, so the exported counters already reflect any restart-restored
cumulative totals — no separate re-seeding step. A pre-existing, unrelated counter,
kubemq_kafka_undecodable_records_total, tracks malformed wire records and resets on restart; it
is not part of this family set.
Per-connection view
Each row in /api/kafka/connections and /api/kafka/connections/:id carries:
| Field | Meaning |
|---|---|
principal | the authenticated identity — empty string if the connection is anonymous |
security_protocol | PLAINTEXT | SSL | SASL_PLAINTEXT | SASL_SSL |
sasl_mechanism | empty, PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512 — empty when SASL isn't in use |
authed | whether the connection completed authentication |
source_ip | the client's source IP |
This is the fastest way to confirm which security posture a given client actually negotiated — useful when validating a SASL or mTLS rollout against Authentication.
Related
Capabilities
Every implemented Kafka API and its version range, from produce/fetch to transactions and share groups.
Architecture
The 9092/9093 wire-protocol listeners and how topics map to the underlying storage engine.
Authentication
SASL/PLAIN and SCRAM, OAUTHBEARER/OIDC, mTLS, and the ACL model this view's fields reflect.
Error Codes
The Kafka protocol error codes behind kubemq_kafka_produce_rejects_total.
Was this page helpful?
Configuration reference
The Kafka connector settings at a glance — the opt-in flag, the 9092/9093 ports, and TOML/environment/Docker examples, plus the full settings reference.
Topic Mapping
How Kafka topics, partitions, and offsets map to KubeMQ — the kafka.{topic} Events Store log, the ~{partition} suffix, and the offset-to-Sequence off-by-one.