Licensing on Kubernetes

On Kubernetes the server enforces the license, exactly as it does everywhere else. The operator is a convenience: it moves the license from your KubemqCluster object into the pods, refuses to create a cluster whose license is obviously invalid, and keeps a copy of the last good lease so a rescheduled pod does not start from scratch. Kubernetes detection changes only the fingerprint source (the kube-system namespace UID) and how the installation describes itself in usage reports.

One installation = one cluster. All pods of a cluster share the same fingerprint, so a 3-replica KubemqCluster is one installation. The license's installation cap (max_instances) is, on Kubernetes, a cap on replicas.

With the operator

CRD fields

The KubemqCluster spec has four licensing fields. Set exactly one: the CRD carries a validation rule (exactly one license source is required: licenseKey, licenseFile, licenseKeySecretRef or licenseFileSecretRef), so an object with none or with more than one is rejected by the API server at admission, before the operator sees it.

The Secret forms are the recommended path: the key or file never appears on the KubemqCluster object, in kubectl get -o yaml, or in the Helm release values. At each reconcile the operator reads your Secret and copies the value into the pod-environment Secret it owns (named after the cluster), which the pods load through envFrom. Your Secret is only ever read, never written, and the resolved value is never stored on the KubemqCluster object. The licensing variables (KUBEMQ_LICENSE_*) and KUBEMQ_API_OPERATOR_TOKEN are reserved: setting them through spec.env is rejected with a ReconcileError condition (nothing is silently stripped).

apiVersion: next.kubemq.io/v1
kind: KubemqCluster
metadata:
  name: kubemq-next
  namespace: kubemq
spec:
  replicas: 3
  licenseKeySecretRef:
    name: kubemq-license
    key: key
  volume:
    size: 20Gi

The Helm chart exposes the same four values at the top level (licenseKey, licenseFile, licenseKeySecretRef, licenseFileSecretRef) — see Install with Helm.

What the operator checks

The operator verifies only an offline file (spec.licenseFile or spec.licenseFileSecretRef). A key carries nothing to verify locally — the server activates it — so a key-licensed cluster passes straight through. At every reconcile of a file-licensed cluster the operator checks the file's signature, its signing key id, that it is not a revocation, and that it is an offline license at all. It deliberately does not check expiry: the running server holds the file too and decides for itself when to stop.

The replica clamp

The operator can only clamp when it knows the cap: from an offline file, or from a cached lease in the lease cache Secret (below) whose license id matches status.license_id. A key-licensed cluster whose cache is still empty — the usual first create — has no cap the operator can see, so it creates the StatefulSet with spec.replicas as given and the server enforces the cap itself.

When the cap is known, the operator clamps the replica count down to max_instances in memory on every reconcile — the KubemqCluster object is never rewritten, and because the peer list it writes is a bootstrap record the same clamp yields the same peers every pass. The ReplicasClamped event is emitted only on the first create. The StatefulSet and the PodDisruptionBudget are both built from the effective count, so a spec.replicas raised past the cap on a live cluster simply keeps the StatefulSet at the cap. The reconcile is blocked only when the live StatefulSet already runs more replicas than the cap (a later license with a lower cap than the running count) and spec.replicas is still above it: the operator sets condition ReplicasOverLicenseCap=True, emits a ReplicasOverLicenseCap warning event, and leaves the StatefulSet as it is — it never shrinks a running cluster on its own. Setting spec.replicas to a value within the cap is your own scale-down and goes through.

The server enforces the cap independently. At boot it counts the statically configured members (the next engine's peer list, or the legacy route list plus one) and refuses to start when they exceed max_instances. In a live cluster on the next engine that joins by membership (the Cluster.Replication.Join setting) it also reads the voting members every minute; after 30 consecutive readings over the cap the node drains (#cluster-over-cap). Autoscaling the StatefulSet with a HorizontalPodAutoscaler is unsupported.

The lease cache Secret

For a key-licensed cluster (an offline file has no lease) the operator copies the server's newest lease into an operator-owned Secret named <cluster>-license-cache under the key lease, and injects that Secret into the pods as KUBEMQ_LICENSE_CACHED_LEASE (optional, so an empty Secret never blocks a pod).

The write-back runs at the end of a reconcile, from the first Ready pod, and calls GET /api/v1/license?include=lease on that pod's management port. It requires the management API's authentication to be enabled (spec.api.auth.enable): the operator generates, once per cluster, a full management-API key in the server's own shape (kmq_<keyid>_<secret>), stores it in its <cluster>-api-admin Secret under the key operator-token, and injects it into the pods as KUBEMQ_API_OPERATOR_TOKEN. The Secret is created before the StatefulSet, like the seed-admin password, and the token is never overwritten once set; when you bring your own spec.api.auth.adminSecretRef the operator-owned Secret holds the token alone. The server seeds a service account named kubemq-operator with the admin role from it on every boot, so rotating the token is a change of the Secret value followed by a pod restart. The operator authenticates with that token as a Bearer credential over the plain-HTTP management listener — no session login, no TLS prerequisite. When authentication is disabled the write-back is skipped and one LeaseWritebackSkipped event says so (this event exists only for that case); every other failure is logged and skipped, and never fails the reconcile. The call is rate-limited: no request is made while the cached lease still has more than 2 hours left, and once it is due at most one attempt is made every 30 minutes per cluster, whatever else re-triggers the reconcile (a candidate cluster is requeued at that cadence so the cache follows the server's hourly refresh even when nothing else changes). A lease is written only when it is newer than the cached one.

Upgrading an existing cluster with authentication enabled to an operator that has this write-back adds KUBEMQ_API_OPERATOR_TOKEN to the pod template, so those pods roll once on the operator upgrade.

The server uses that value only when its own cache file does not exist — the case of a pod rescheduled onto a node with a fresh volume. Instead of a first boot, which needs the licensing service to be reachable at that moment, the pod starts from the cached lease and refreshes right away. The server verifies the lease's signature and fingerprint before adopting it, and adopts it up to its expiry plus grace (with a 5-minute leeway), logging #grace-countdown when the lease itself has already expired; the operator, for its part, clears the Secret as housekeeping only when the cached lease is past its expiry. A cached lease that fails verification for any other reason (an untrusted or unknown signing key, a token of the wrong kind) is kept in place and not used, with a log line: the failure may be the operator's own trust bundle lagging a key rotation, and the server decides for itself what it adopts. The operator also records the license id it trusts in status.license_id (the file's, or the first lease it cached) and refuses a written-back lease whose id differs from the cached one (warning event LeaseIdentityMismatch), so a pod cannot swap the cluster's license from the inside. The operator fingerprints the resolved license input in status.license_source_hash (a hash, never the value); when it changes the license was deliberately replaced, so status.license_id and the cached lease are cleared (event LicenseSourceChanged) and the next lease the server reports becomes the new identity instead of a mismatch.

Without the write-back (management-API authentication disabled) everything else — the gate, the replica cap, the persistent-volume cache — still works; a pod rescheduled onto a fresh volume simply does a first boot instead of starting from the cached lease.

Two things this Secret is not:

• It is not your licensing Secret. The operator never writes to the Secret you reference in licenseKeySecretRef / licenseFileSecretRef, which may be GitOps-managed.
• It is not part of the pod template. Only its contents change, so a lease refresh never rolls the StatefulSet.

Status

kubectl get kubemqclusters.next.kubemq.io -n kubemq prints the License-type, License-To and License-Expire columns, and -o yaml shows status.license_id, status.license_source_hash and the two licensing conditions the operator sets, LicenseInvalid and ReplicasOverLicenseCap. The three columns and status.license_id are filled only from an offline license file or from a cached lease in the lease cache Secret — and the lease write-back needs management-API authentication (api.auth.enable). On a key-licensed cluster with authentication off, the usual first install, the operator emits LeaseWritebackSkipped and all four stay empty; that is expected, not a licensing failure — ask the server (below) for the live state. kubectl describe lists the events (LicenseInvalid, ReplicasClamped, ReplicasOverLicenseCap, LicenseSourceChanged, LeaseWritebackSkipped, LeaseIdentityMismatch). For the live state — lease expiry, refresh outcome, grace countdown, revocation count — ask the server (or run kmq license):

kubectl port-forward -n kubemq svc/kubemq-next-api 8080:8080
curl -s http://localhost:8080/api/v1/license | jq

The response is documented on Use a license key.

Without the operator

You can run the server image in your own StatefulSet. Licensing works the same way; you supply what the operator would otherwise provide.

RBAC for the fingerprint

The server's fingerprint on Kubernetes is the UID of the kube-system namespace, read with a plain HTTPS GET /api/v1/namespaces/kube-system against the API server using the pod's projected ServiceAccount token. That read needs get on namespaces for the single resource name kube-system.

The Helm chart creates a ServiceAccount named kubemq-cluster-next, a ClusterRole named kubemq-cluster-next-<namespace>-license with exactly that permission, and a ClusterRoleBinding named kubemq-cluster-next-<namespace>-license-crb (both scoped to the release namespace, so two releases in different namespaces do not collide). Without the chart, create the equivalent yourself (this is the operator repo's deploy/next/rbac.yaml):

apiVersion: v1
kind: ServiceAccount
metadata:
  name: kubemq-cluster-next
  namespace: kubemq
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kubemq-cluster-next-license
rules:
  - apiGroups: [""]
    resources: ["namespaces"]
    resourceNames: ["kube-system"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubemq-cluster-next-kubemq-license-crb
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubemq-cluster-next-license
subjects:
  - kind: ServiceAccount
    name: kubemq-cluster-next
    namespace: kubemq

Set serviceAccountName: kubemq-cluster-next on the pod template.

What happens without the permission

The server still starts. It falls back to a random id persisted at <store>/license/instance.id, logs a boot warning naming the missing permission (#missing-kube-system-permission), and reports fingerprint_source: "persisted" in GET /api/v1/license. The consequences:

• Each pod has its own store, so each pod is a separate installation — a 3-replica cluster counts as 3 installations against your cap instead of 1.
• An offline file issued for the cluster's kube-system UID does not match the fallback id, so the server refuses to start with #fingerprint-mismatch.
• A pod that loses its volume gets a new fingerprint and a new installation row.

Grant the permission; do not work around it.

PersistentVolumeClaim requirement

The lease cache, the persisted instance id and the usage cursor all live under the store path. Without a PersistentVolumeClaim every pod restart is a first boot that must reach the licensing service, and the fail-open window described in the overview does not exist. Use a volumeClaimTemplate mounted at /kubemq/store; the server logs #store-not-persistent at boot when the store path is not a mount.

Inject the license

Use a Secret with secretKeyRef, as shown on Use a license key (key) and Air-gapped installations (file). Do not put a license in a ConfigMap or a plain value:.