Running Celery on KubeMQ inside Kubernetes gives you a single in-cluster broker that scales with your task queue instead of an external Redis or RabbitMQ StatefulSet. This guide walks through deploying the KubeMQ broker, the Celery worker Deployment, health probes, and KEDA autoscaling driven by KubeMQ queue depth — all using the manifests shipped in the kubemq-celery repository.

Prerequisites

A Kubernetes cluster and kubectl access to it
A Celery app image built with kubemq-celery installed (see Configuration)
Helm, if you plan to install KEDA for queue-depth autoscaling

Architecture

A Celery worker Deployment (N replicas) connects to a KubeMQ StatefulSet over gRPC on port 50000. KEDA watches the KubeMQ queue depth and scales the Deployment up or down, while the shared HTTP server (REST/health) on port 9090 and the Management API dashboard on port 8080 expose broker-level health and metrics.

Workers reach the broker through its Kubernetes Service DNS name (kubemq.default.svc:50000), so no external load balancer or NodePort is needed for internal traffic. The kubemq:// URL scheme is registered by import kubemq_celery — see Getting Started if you have not wired up the app yet.

Deploy the KubeMQ Broker

Quick deploy

The fastest path is the hosted manifest, which creates a KubeMQ StatefulSet in the default namespace with the gRPC service on port 50000, the shared HTTP server (REST/health) on port 9090, the Management API dashboard on port 8080, and persistent storage:

Quick deploy

kubectl apply -f https://get.kubemq.io/deploy

Custom 3-replica StatefulSet

For production, deploy a 3-replica HA cluster with explicit resource limits, TCP health probes, and a license token sourced from a Secret. The accompanying Service exposes the gRPC port (50000), the shared HTTP server (REST/health) on 9090, and the Management API dashboard on 8080:

kubemq-statefulset.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: kubemq
  namespace: default
spec:
  serviceName: kubemq
  replicas: 3  # HA cluster
  selector:
    matchLabels:
      app: kubemq
  template:
    metadata:
      labels:
        app: kubemq
    spec:
      containers:
        - name: kubemq
          image: europe-docker.pkg.dev/kubemq/images/kubemq:next
          ports:
            - containerPort: 50000
              name: grpc
            - containerPort: 9090
              name: http
            - containerPort: 8080
              name: dashboard
          env:
            - name: KUBEMQ_TOKEN
              valueFrom:
                secretKeyRef:
                  name: kubemq-license
                  key: token
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: 1000m
              memory: 1Gi
          readinessProbe:
            tcpSocket:
              port: 50000
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            tcpSocket:
              port: 50000
            initialDelaySeconds: 15
            periodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
  name: kubemq
  namespace: default
spec:
  selector:
    app: kubemq
  ports:
    - name: grpc
      port: 50000
      targetPort: 50000
    - name: http
      port: 9090
      targetPort: 9090
    - name: dashboard
      port: 8080
      targetPort: 8080
  type: ClusterIP

Once applied, the broker is reachable cluster-internally at kubemq.default.svc:50000.

The KubeMQ broker is the only service that needs to be running before workers start. Workers will not pass their readiness probe until they can reach it — see Health Checks below.

Deploy Celery Workers

The worker container runs the standard Celery command and reads its broker and result-backend addresses from environment variables. The readiness and liveness probes both use celery inspect ping, which exercises the transport's connection to KubeMQ.

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: celery-worker
  namespace: default
  labels:
    app: celery-worker
    component: worker
spec:
  replicas: 3
  selector:
    matchLabels:
      app: celery-worker
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0  # zero-downtime rolling updates
  template:
    metadata:
      labels:
        app: celery-worker
        component: worker
    spec:
      terminationGracePeriodSeconds: 120  # allow tasks to complete
      containers:
        - name: worker
          image: myapp:latest
          command:
            - celery
            - -A
            - myapp
            - worker
            - --loglevel=info
            - --concurrency=4
          env:
            - name: CELERY_BROKER_URL
              value: "kubemq://kubemq.default.svc:50000"
            - name: CELERY_RESULT_BACKEND
              value: "kubemq://kubemq.default.svc:50000"
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: 1000m
              memory: 512Mi
          # Readiness: worker is ready when it can reach the broker
          readinessProbe:
            exec:
              command:
                - celery
                - -A
                - myapp
                - inspect
                - ping
                - --timeout=5
            initialDelaySeconds: 30
            periodSeconds: 30
            timeoutSeconds: 10
            failureThreshold: 3
          # Liveness: worker is alive when it can still reach the broker
          livenessProbe:
            exec:
              command:
                - celery
                - -A
                - myapp
                - inspect
                - ping
                - --timeout=5
            initialDelaySeconds: 60
            periodSeconds: 60
            timeoutSeconds: 10
            failureThreshold: 3
          # Graceful shutdown: send SIGTERM, worker finishes current tasks
          lifecycle:
            preStop:
              exec:
                command: ["celery", "-A", "myapp", "control", "shutdown"]

A few details worth calling out:

maxUnavailable: 0 with maxSurge: 1 gives zero-downtime rolling updates — a new worker becomes ready before an old one is removed.
terminationGracePeriodSeconds: 120 plus the preStop celery control shutdown hook lets in-flight tasks drain instead of being killed mid-execution.
The probe --timeout=5 keeps inspect ping from hanging when the broker is unreachable, so Kubernetes can react quickly.

Your Celery app should read the same environment variables so the image works identically in Docker, Compose, and Kubernetes:

myapp.py

import os
import kubemq_celery  # registers the kubemq:// transport
from celery import Celery

app = Celery("myapp")
app.conf.update(
    broker_url=os.environ.get("CELERY_BROKER_URL", "kubemq://localhost:50000"),
    result_backend=os.environ.get("CELERY_RESULT_BACKEND", "kubemq://localhost:50000"),
    worker_prefetch_multiplier=1,
    task_acks_late=False,
)

Build the Worker Image

The repository ships a uv-based Dockerfile on top of python:3.12-slim. It installs dependencies from pyproject.toml into the system environment and adds a HEALTHCHECK that imports the transport package:

Dockerfile

FROM python:3.12-slim

WORKDIR /app

# Install uv and dependencies
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
COPY pyproject.toml .
RUN uv pip install --system --no-cache .

# Copy application code
COPY src/ src/
COPY examples/ examples/

ENV PYTHONPATH=/app/src:/app/examples

# Health check
HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
    CMD python -c "import kubemq_celery; print('ok')" || exit 1

CMD ["celery", "-A", "basic_task", "worker", "--loglevel=info"]

For local development, the provided docker-compose.yaml runs a complete stack: the KubeMQ broker, two workers on different queues (celery default and high-priority), and a Beat scheduler. The broker's health check gates worker startup via depends_on: condition: service_healthy:

docker-compose.yaml

services:
  # KubeMQ broker
  kubemq:
    image: europe-docker.pkg.dev/kubemq/images/kubemq:next
    ports:
      - "50000:50000"  # gRPC
      - "9090:9090"    # Shared HTTP server (REST/health)
      - "8080:8080"    # Management API dashboard
    environment:
      - KUBEMQ_LOG_LEVEL=info
    healthcheck:
      test: ["CMD", "wget", "--spider", "-q", "http://localhost:9090/health"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 10s

  # Worker 1: default queue
  worker-default:
    build:
      context: ../..
      dockerfile: examples/kubernetes/Dockerfile
    command: celery -A basic_task worker --loglevel=info -Q celery --concurrency=2
    environment:
      - CELERY_BROKER_URL=kubemq://kubemq:50000
      - CELERY_RESULT_BACKEND=kubemq://kubemq:50000
    depends_on:
      kubemq:
        condition: service_healthy
    restart: unless-stopped

  # Worker 2: high-priority queue
  worker-priority:
    build:
      context: ../..
      dockerfile: examples/kubernetes/Dockerfile
    command: celery -A basic_task worker --loglevel=info -Q high-priority --concurrency=2
    environment:
      - CELERY_BROKER_URL=kubemq://kubemq:50000
      - CELERY_RESULT_BACKEND=kubemq://kubemq:50000
    depends_on:
      kubemq:
        condition: service_healthy
    restart: unless-stopped

  # Beat scheduler
  beat:
    build:
      context: ../..
      dockerfile: examples/kubernetes/Dockerfile
    command: celery -A basic_task beat --loglevel=info
    environment:
      - CELERY_BROKER_URL=kubemq://kubemq:50000
      - CELERY_RESULT_BACKEND=kubemq://kubemq:50000
    depends_on:
      kubemq:
        condition: service_healthy
    restart: unless-stopped

Run the stack

cd examples/kubernetes
docker compose up -d

Service Discovery

Workers connect to the broker through the Kubernetes Service DNS name. The URL takes the form kubemq://<service-name>.<namespace>.svc:<port>; pick the variant that matches where your workers run:

Scenario	Broker URL
Same namespace	`kubemq://kubemq:50000`
Explicit namespace	`kubemq://kubemq.default.svc:50000`
FQDN	`kubemq://kubemq.default.svc.cluster.local:50000`
With auth	`kubemq://:my-token@kubemq.default.svc:50000`
With TLS	`kubemq+tls://kubemq.default.svc:50000`

When workers live in the same namespace as the broker, the short kubemq://kubemq:50000 form is enough. Use the explicit-namespace or FQDN forms for cross-namespace traffic, and switch to kubemq+tls:// when gRPC encryption is required.

KEDA Autoscaling

KEDA scales the worker Deployment on KubeMQ queue depth rather than CPU or memory, which is far more responsive for task-queue workloads than the standard HPA.

Install KEDA

Install KEDA via Helm

helm repo add kedacore https://kedacore.github.io/charts
helm install keda kedacore/keda --namespace keda --create-namespace

ScaledObject for queue depth

The kubemq trigger polls the broker for the depth of a channel and scales the target Deployment when it exceeds queueLength. The example below polls every 10 seconds and adds workers once more than 10 messages are queued on the celery channel:

keda-scaler.yaml

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: celery-worker-scaler
  namespace: default
  labels:
    app: celery-worker
    component: autoscaling
spec:
  scaleTargetRef:
    name: celery-worker          # must match Deployment name
  minReplicaCount: 1             # always keep at least 1 worker
  maxReplicaCount: 20            # maximum workers
  pollingInterval: 10            # check queue depth every 10 seconds
  cooldownPeriod: 60             # wait 60s before scaling down
  triggers:
    - type: kubemq
      metadata:
        # KubeMQ broker address (cluster-internal)
        address: "kubemq.default.svc:50000"
        # Queue channel to monitor (Celery default queue)
        channel: "celery"
        # Scale threshold: add workers when queue depth > 10
        queueLength: "10"

Multi-queue scaling

When you route tasks across priority queues, create one ScaledObject per queue, each targeting its own worker Deployment with thresholds tuned to its priority. A high-priority queue stays warm with minReplicaCount: 2 and scales early at queueLength: 5, while a low-priority queue scales to zero (minReplicaCount: 0) and tolerates a deeper backlog (queueLength: 50):

keda-scaler-multi-queue.yaml

# High-priority queue -- aggressive scaling
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: celery-high-priority-scaler
  namespace: default
spec:
  scaleTargetRef:
    name: celery-worker-high     # separate Deployment for high-priority
  minReplicaCount: 2             # always keep 2 for high-priority
  maxReplicaCount: 50
  pollingInterval: 5             # check more frequently
  cooldownPeriod: 30             # scale down faster
  triggers:
    - type: kubemq
      metadata:
        address: "kubemq.default.svc:50000"
        channel: "high-priority"
        queueLength: "5"         # lower threshold for high-priority
---
# Low-priority queue -- conservative scaling
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: celery-low-priority-scaler
  namespace: default
spec:
  scaleTargetRef:
    name: celery-worker-low
  minReplicaCount: 0             # scale to zero when idle
  maxReplicaCount: 10
  pollingInterval: 30            # check less frequently
  cooldownPeriod: 300            # wait 5 minutes before scaling down
  triggers:
    - type: kubemq
      metadata:
        address: "kubemq.default.svc:50000"
        channel: "low-priority"
        queueLength: "50"        # higher threshold for low-priority

Scale-to-zero (minReplicaCount: 0) only fires when the queue is empty for the full cooldownPeriod. Pair it with a longer pollingInterval to avoid flapping on bursty low-priority traffic.

Resource Recommendations

Size the broker by sustained message throughput, and size workers by their concurrency and whether the workload is I/O- or CPU-bound.

KubeMQ broker

Load	CPU Request	CPU Limit	Memory Request	Memory Limit	Replicas
Low (< 100 msg/s)	250m	500m	256Mi	512Mi	1
Medium (100–1000 msg/s)	500m	1000m	512Mi	1Gi	3
High (> 1000 msg/s)	1000m	2000m	1Gi	2Gi	3

Celery workers

Concurrency	CPU Request	CPU Limit	Memory Request	Memory Limit
1 (I/O bound)	100m	500m	128Mi	256Mi
4 (default)	250m	1000m	256Mi	512Mi
8 (CPU bound)	500m	2000m	512Mi	1Gi

Health Checks

Worker health

The transport implements verify_connection(), which pings the KubeMQ broker. Celery's built-in inspect commands rely on it, so the same calls used in the Deployment probes also work for manual diagnostics:

Worker diagnostics

# Check if workers are alive
celery -A myapp inspect ping

# List active tasks
celery -A myapp inspect active

# Check worker stats
celery -A myapp inspect stats

Broker health

KubeMQ accepts TCP probes on its gRPC port. Use these on the broker StatefulSet:

Broker probes

readinessProbe:
  tcpSocket:
    port: 50000
  initialDelaySeconds: 5
  periodSeconds: 10
livenessProbe:
  tcpSocket:
    port: 50000
  initialDelaySeconds: 15
  periodSeconds: 20

Dashboard

The KubeMQ Management API dashboard provides broker-level monitoring — queue depth, message rates, and channel statistics — on port 8080 (the shared HTTP server on 9090 serves REST and the /health probe). Port-forward to reach the dashboard locally:

Open the dashboard

# Port-forward to access the Management API dashboard locally
kubectl port-forward svc/kubemq 8080:8080

# Open http://localhost:8080 in your browser

The dashboard complements Flower's task-level view: Flower shows tasks and workers, while the KubeMQ dashboard shows the channels backing them.

Production Checklist

Broker availability — deploy the KubeMQ broker with >= 3 replicas for HA and configure persistent volume claims for its data.

Security — set a KubeMQ authentication token on both the broker and the workers, and enable TLS for gRPC connections with kubemq+tls://.

Resilience — set worker resource requests and limits, configure readiness and liveness probes, and add a dead letter queue for failed messages.

Scaling — configure a KEDA ScaledObject for queue-depth autoscaling and match worker --concurrency to the workload type (I/O vs CPU bound).

Operations — deploy Flower for task monitoring, set result_expires to <= 86400 (24 hours), and apply network policies that restrict broker access to worker namespaces.

Kubernetes Deployment & KEDA Autoscaling

Prerequisites

Architecture

Deploy the KubeMQ Broker

Quick deploy

Custom 3-replica StatefulSet

Deploy Celery Workers

Build the Worker Image

Service Discovery

KEDA Autoscaling

Install KEDA

ScaledObject for queue depth

Multi-queue scaling

Resource Recommendations

KubeMQ broker

Celery workers

Health Checks

Worker health

Broker health

Dashboard

Production Checklist

Configuration

Migration

FastAPI Task Dispatch with KEDA

On this page