Observability
Monitor KubeMQ end to end — Prometheus metrics, OpenTelemetry tracing, structured logs, an audit trail, the dashboard, and the management API on port 8080.
KubeMQ reports on itself across several complementary surfaces. An always-on Prometheus exporter, opt-in OpenTelemetry traces and metrics, structured JSON logs, a CloudEvents audit trail, a built-in web dashboard, and an HTTP/WebSocket management API all expose what the server is doing right now and what it has done. This section is the operator's home for monitoring KubeMQ in production — how to scrape metrics, wire up tracing, read the logs, query the audit log, and consume the management API.
Each surface answers a different question: Prometheus metrics give you the quantitative time series, tracing shows you the path of a single message across patterns, logs and the audit trail record discrete events, the dashboard renders the live picture, and the management API is the programmatic interface behind it all.
The observability surfaces
| Surface | What it gives you | Where it's served | Always-on? |
|---|---|---|---|
| Prometheus metrics | Counters, gauges, and histograms for every messaging pattern, queues, RPC latency, cluster health, and the agent platform | GET /metrics (:8080) | Yes |
| OpenTelemetry tracing | Distributed traces and OTel metrics over OTLP to Jaeger, Grafana Tempo, or Datadog | OTLP export to an external collector | No (opt-in) |
| Structured logging | JSON log lines on stdout — levels, the field set, and trace correlation | stdout | Yes |
| Audit logging | A CloudEvents v1.0 trail of auth, lifecycle, and data-plane error events | GET /api/v1/audit (:8080) | Yes |
| Built-in dashboard | A live web view of channels, clients, queues, the patterns, cluster topology, and agents | :8080 | Yes |
| Management API | The HTTP and WebSocket interface for health, stats, snapshots, audit query, and dashboard actions | :8080 | Yes |
Push vs pull
KubeMQ uses a pull-first model for metrics and a push-first model for traces, and the two can run at the same time. Logs are written to stdout and captured by your runtime.
| Signal | Model | Consumer |
|---|---|---|
| Prometheus metrics | Pull — Prometheus scrapes GET /metrics on demand | Prometheus / Grafana |
| Dashboard snapshots | Pull — the dashboard polls the snapshot endpoints | Built-in dashboard |
| OpenTelemetry traces | Push — the OTLP exporter pushes to a collector | Jaeger / Tempo / Datadog |
| OpenTelemetry metrics | Push — the meter provider exports periodically | OTLP-compatible metrics backend |
| Audit events | Pull — the REST query API | Operator / SIEM |
| Logs | Push — JSON written to stdout, captured by the runtime | Log aggregator (Loki, Splunk, etc.) |
The Prometheus exporter always runs — it also powers the built-in dashboard. OpenTelemetry is opt-in and is enabled through the telemetry settings.
Where each surface is served
The management API port (:8080) is the same in-process port that serves the dashboard,
the metrics endpoint, the stats and snapshot endpoints, and the audit query API. When
control-plane auth is enabled, every endpoint below requires at least the ReadOnly
role.
| Surface | Endpoint | Access (control-plane auth on) |
|---|---|---|
| Prometheus scrape | GET /metrics (:8080) | ReadOnly+ |
| Dashboard snapshots | GET /api/snapshot, GET /api/cluster-snapshot | ReadOnly+ |
| Audit query | GET /api/v1/audit, GET /api/v1/audit/stats | ReadOnly+ |
| Channel and client stats | GET /v1/stats/channels, GET /v1/stats/clients | ReadOnly+ |
| Live message monitor | WebSocket /api/monitor | ReadOnly+ |
| Real-time cluster stream | WebSocket /api/connection | ReadOnly+ |
| Billing summary | GET /billing | ReadOnly+ |
| OTLP export | OTLP gRPC (:4317) or HTTP (:4318) to an external collector | Collector-side |
| Application logs | stdout (JSON) | — |
Start here
Prometheus Metrics
Scrape the always-on exporter — every metric series for messaging, queues, RPC latency, cluster health, and the agent platform.
Distributed Tracing
Export OpenTelemetry traces and metrics over OTLP to Jaeger, Grafana Tempo, or Datadog.
Structured Logging
JSON logs on stdout — levels, runtime level changes, the field set, and trace correlation.
Audit Logging
The CloudEvents audit trail — the event catalog, retention settings, and the REST query API.
Management API
The HTTP and WebSocket management API on port 8080 — envelope, status codes, endpoints, and data models.
Configure it
Where the settings live
The raw configuration keys for telemetry, audit, and server-event notifications — with their Docker and Helm naming, defaults, and valid values — live in the Observability settings reference. That page is the source of truth for the settings; this section covers the concepts, how-to, and the full management API.
Was this page helpful?
Operate KubeMQ
Run KubeMQ in production with full visibility — metrics, distributed tracing, structured logging, and audit for monitoring the messaging engine.
Prometheus Metrics
Scrape KubeMQ's always-on Prometheus endpoint — every metric series for messaging, queues, RPC latency, cluster health, and the agent platform.