# Kubernetes Deployment & KEDA Autoscaling (/integrations/celery/how-to/kubernetes)



Running Celery on KubeMQ inside Kubernetes gives you a single in-cluster broker that scales with your task queue instead of an external Redis or RabbitMQ StatefulSet. This guide walks through deploying the KubeMQ broker, the Celery worker Deployment, health probes, and [KEDA](https://keda.sh/) autoscaling driven by KubeMQ queue depth — all using the manifests shipped in the `kubemq-celery` repository.

## Prerequisites [#prerequisites]

* A Kubernetes cluster and `kubectl` access to it
* A Celery app image built with `kubemq-celery` installed (see [Configuration](/integrations/celery/how-to/configuration))
* Helm, if you plan to install [KEDA](https://keda.sh/) for queue-depth autoscaling

## Architecture [#architecture]

A Celery worker Deployment (N replicas) connects to a KubeMQ StatefulSet over gRPC on port `50000`. KEDA watches the KubeMQ queue depth and scales the Deployment up or down, while the shared HTTP server (REST/health) on port `9090` and the Management API dashboard on port `8080` expose broker-level health and metrics.

<Mermaid
  chart="flowchart LR
    Worker[&#x22;Celery Worker Deployment<br/>(N replicas)&#x22;] -->|&#x22;gRPC :50000&#x22;| KMQ[&#x22;KubeMQ StatefulSet<br/>(HA cluster)&#x22;]
    KMQ -->|&#x22;gRPC :50000&#x22;| Worker
    KEDA[&#x22;KEDA ScaledObject<br/>(queue-depth autoscale)&#x22;] -.->|&#x22;scale replicas&#x22;| Worker
    KEDA -.->|&#x22;poll queue depth&#x22;| KMQ
    Dash[&#x22;KubeMQ Management API<br/>Dashboard :8080&#x22;] --- KMQ
    HTTP[&#x22;Shared HTTP Server<br/>REST/health :9090&#x22;] --- KMQ"
/>

Workers reach the broker through its Kubernetes Service DNS name (`kubemq.default.svc:50000`), so no external load balancer or NodePort is needed for internal traffic. The `kubemq://` URL scheme is registered by `import kubemq_celery` — see [Getting Started](/integrations/celery/tutorials/getting-started) if you have not wired up the app yet.

## Deploy the KubeMQ Broker [#deploy-the-kubemq-broker]

### Quick deploy [#quick-deploy]

The fastest path is the hosted manifest, which creates a KubeMQ StatefulSet in the `default` namespace with the gRPC service on port `50000`, the shared HTTP server (REST/health) on port `9090`, the Management API dashboard on port `8080`, and persistent storage:

```bash title="Quick deploy"
kubectl apply -f https://get.kubemq.io/deploy
```

### Custom 3-replica StatefulSet [#custom-3-replica-statefulset]

For production, deploy a 3-replica HA cluster with explicit resource limits, TCP health probes, and a license token sourced from a Secret. The accompanying Service exposes the gRPC port (`50000`), the shared HTTP server (REST/health) on `9090`, and the Management API dashboard on `8080`:

```yaml title="kubemq-statefulset.yaml"
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: kubemq
  namespace: default
spec:
  serviceName: kubemq
  replicas: 3  # HA cluster
  selector:
    matchLabels:
      app: kubemq
  template:
    metadata:
      labels:
        app: kubemq
    spec:
      containers:
        - name: kubemq
          image: europe-docker.pkg.dev/kubemq/images/kubemq:next
          ports:
            - containerPort: 50000
              name: grpc
            - containerPort: 9090
              name: http
            - containerPort: 8080
              name: dashboard
          env:
            - name: KUBEMQ_TOKEN
              valueFrom:
                secretKeyRef:
                  name: kubemq-license
                  key: token
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: 1000m
              memory: 1Gi
          readinessProbe:
            tcpSocket:
              port: 50000
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            tcpSocket:
              port: 50000
            initialDelaySeconds: 15
            periodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
  name: kubemq
  namespace: default
spec:
  selector:
    app: kubemq
  ports:
    - name: grpc
      port: 50000
      targetPort: 50000
    - name: http
      port: 9090
      targetPort: 9090
    - name: dashboard
      port: 8080
      targetPort: 8080
  type: ClusterIP
```

Once applied, the broker is reachable cluster-internally at `kubemq.default.svc:50000`.

<Callout type="info">
  The KubeMQ broker is the only service that needs to be running before workers start. Workers will not pass their readiness probe until they can reach it — see [Health Checks](#health-checks) below.
</Callout>

## Deploy Celery Workers [#deploy-celery-workers]

The worker container runs the standard Celery command and reads its broker and result-backend addresses from environment variables. The readiness and liveness probes both use `celery inspect ping`, which exercises the transport's connection to KubeMQ.

```yaml title="deployment.yaml"
apiVersion: apps/v1
kind: Deployment
metadata:
  name: celery-worker
  namespace: default
  labels:
    app: celery-worker
    component: worker
spec:
  replicas: 3
  selector:
    matchLabels:
      app: celery-worker
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0  # zero-downtime rolling updates
  template:
    metadata:
      labels:
        app: celery-worker
        component: worker
    spec:
      terminationGracePeriodSeconds: 120  # allow tasks to complete
      containers:
        - name: worker
          image: myapp:latest
          command:
            - celery
            - -A
            - myapp
            - worker
            - --loglevel=info
            - --concurrency=4
          env:
            - name: CELERY_BROKER_URL
              value: "kubemq://kubemq.default.svc:50000"
            - name: CELERY_RESULT_BACKEND
              value: "kubemq://kubemq.default.svc:50000"
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: 1000m
              memory: 512Mi
          # Readiness: worker is ready when it can reach the broker
          readinessProbe:
            exec:
              command:
                - celery
                - -A
                - myapp
                - inspect
                - ping
                - --timeout=5
            initialDelaySeconds: 30
            periodSeconds: 30
            timeoutSeconds: 10
            failureThreshold: 3
          # Liveness: worker is alive when it can still reach the broker
          livenessProbe:
            exec:
              command:
                - celery
                - -A
                - myapp
                - inspect
                - ping
                - --timeout=5
            initialDelaySeconds: 60
            periodSeconds: 60
            timeoutSeconds: 10
            failureThreshold: 3
          # Graceful shutdown: send SIGTERM, worker finishes current tasks
          lifecycle:
            preStop:
              exec:
                command: ["celery", "-A", "myapp", "control", "shutdown"]
```

A few details worth calling out:

* `maxUnavailable: 0` with `maxSurge: 1` gives zero-downtime rolling updates — a new worker becomes ready before an old one is removed.
* `terminationGracePeriodSeconds: 120` plus the `preStop` `celery control shutdown` hook lets in-flight tasks drain instead of being killed mid-execution.
* The probe `--timeout=5` keeps `inspect ping` from hanging when the broker is unreachable, so Kubernetes can react quickly.

Your Celery app should read the same environment variables so the image works identically in Docker, Compose, and Kubernetes:

```python title="myapp.py"
import os
import kubemq_celery  # registers the kubemq:// transport
from celery import Celery

app = Celery("myapp")
app.conf.update(
    broker_url=os.environ.get("CELERY_BROKER_URL", "kubemq://localhost:50000"),
    result_backend=os.environ.get("CELERY_RESULT_BACKEND", "kubemq://localhost:50000"),
    worker_prefetch_multiplier=1,
    task_acks_late=False,
)
```

## Build the Worker Image [#build-the-worker-image]

The repository ships a `uv`-based Dockerfile on top of `python:3.12-slim`. It installs dependencies from `pyproject.toml` into the system environment and adds a `HEALTHCHECK` that imports the transport package:

```dockerfile title="Dockerfile"
FROM python:3.12-slim

WORKDIR /app

# Install uv and dependencies
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
COPY pyproject.toml .
RUN uv pip install --system --no-cache .

# Copy application code
COPY src/ src/
COPY examples/ examples/

ENV PYTHONPATH=/app/src:/app/examples

# Health check
HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
    CMD python -c "import kubemq_celery; print('ok')" || exit 1

CMD ["celery", "-A", "basic_task", "worker", "--loglevel=info"]
```

For local development, the provided `docker-compose.yaml` runs a complete stack: the KubeMQ broker, two workers on different queues (`celery` default and `high-priority`), and a Beat scheduler. The broker's health check gates worker startup via `depends_on: condition: service_healthy`:

```yaml title="docker-compose.yaml"
services:
  # KubeMQ broker
  kubemq:
    image: europe-docker.pkg.dev/kubemq/images/kubemq:next
    ports:
      - "50000:50000"  # gRPC
      - "9090:9090"    # Shared HTTP server (REST/health)
      - "8080:8080"    # Management API dashboard
    environment:
      - KUBEMQ_LOG_LEVEL=info
    healthcheck:
      test: ["CMD", "wget", "--spider", "-q", "http://localhost:9090/health"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 10s

  # Worker 1: default queue
  worker-default:
    build:
      context: ../..
      dockerfile: examples/kubernetes/Dockerfile
    command: celery -A basic_task worker --loglevel=info -Q celery --concurrency=2
    environment:
      - CELERY_BROKER_URL=kubemq://kubemq:50000
      - CELERY_RESULT_BACKEND=kubemq://kubemq:50000
    depends_on:
      kubemq:
        condition: service_healthy
    restart: unless-stopped

  # Worker 2: high-priority queue
  worker-priority:
    build:
      context: ../..
      dockerfile: examples/kubernetes/Dockerfile
    command: celery -A basic_task worker --loglevel=info -Q high-priority --concurrency=2
    environment:
      - CELERY_BROKER_URL=kubemq://kubemq:50000
      - CELERY_RESULT_BACKEND=kubemq://kubemq:50000
    depends_on:
      kubemq:
        condition: service_healthy
    restart: unless-stopped

  # Beat scheduler
  beat:
    build:
      context: ../..
      dockerfile: examples/kubernetes/Dockerfile
    command: celery -A basic_task beat --loglevel=info
    environment:
      - CELERY_BROKER_URL=kubemq://kubemq:50000
      - CELERY_RESULT_BACKEND=kubemq://kubemq:50000
    depends_on:
      kubemq:
        condition: service_healthy
    restart: unless-stopped
```

```bash title="Run the stack"
cd examples/kubernetes
docker compose up -d
```

## Service Discovery [#service-discovery]

Workers connect to the broker through the Kubernetes Service DNS name. The URL takes the form `kubemq://<service-name>.<namespace>.svc:<port>`; pick the variant that matches where your workers run:

| Scenario           | Broker URL                                        |
| ------------------ | ------------------------------------------------- |
| Same namespace     | `kubemq://kubemq:50000`                           |
| Explicit namespace | `kubemq://kubemq.default.svc:50000`               |
| FQDN               | `kubemq://kubemq.default.svc.cluster.local:50000` |
| With auth          | `kubemq://:my-token@kubemq.default.svc:50000`     |
| With TLS           | `kubemq+tls://kubemq.default.svc:50000`           |

When workers live in the same namespace as the broker, the short `kubemq://kubemq:50000` form is enough. Use the explicit-namespace or FQDN forms for cross-namespace traffic, and switch to `kubemq+tls://` when gRPC encryption is required.

## KEDA Autoscaling [#keda-autoscaling]

KEDA scales the worker Deployment on KubeMQ queue depth rather than CPU or memory, which is far more responsive for task-queue workloads than the standard HPA.

### Install KEDA [#install-keda]

```bash title="Install KEDA via Helm"
helm repo add kedacore https://kedacore.github.io/charts
helm install keda kedacore/keda --namespace keda --create-namespace
```

### ScaledObject for queue depth [#scaledobject-for-queue-depth]

The `kubemq` trigger polls the broker for the depth of a channel and scales the target Deployment when it exceeds `queueLength`. The example below polls every 10 seconds and adds workers once more than 10 messages are queued on the `celery` channel:

```yaml title="keda-scaler.yaml"
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: celery-worker-scaler
  namespace: default
  labels:
    app: celery-worker
    component: autoscaling
spec:
  scaleTargetRef:
    name: celery-worker          # must match Deployment name
  minReplicaCount: 1             # always keep at least 1 worker
  maxReplicaCount: 20            # maximum workers
  pollingInterval: 10            # check queue depth every 10 seconds
  cooldownPeriod: 60             # wait 60s before scaling down
  triggers:
    - type: kubemq
      metadata:
        # KubeMQ broker address (cluster-internal)
        address: "kubemq.default.svc:50000"
        # Queue channel to monitor (Celery default queue)
        channel: "celery"
        # Scale threshold: add workers when queue depth > 10
        queueLength: "10"
```

### Multi-queue scaling [#multi-queue-scaling]

When you route tasks across priority queues, create one `ScaledObject` per queue, each targeting its own worker Deployment with thresholds tuned to its priority. A high-priority queue stays warm with `minReplicaCount: 2` and scales early at `queueLength: 5`, while a low-priority queue scales to zero (`minReplicaCount: 0`) and tolerates a deeper backlog (`queueLength: 50`):

```yaml title="keda-scaler-multi-queue.yaml"
# High-priority queue -- aggressive scaling
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: celery-high-priority-scaler
  namespace: default
spec:
  scaleTargetRef:
    name: celery-worker-high     # separate Deployment for high-priority
  minReplicaCount: 2             # always keep 2 for high-priority
  maxReplicaCount: 50
  pollingInterval: 5             # check more frequently
  cooldownPeriod: 30             # scale down faster
  triggers:
    - type: kubemq
      metadata:
        address: "kubemq.default.svc:50000"
        channel: "high-priority"
        queueLength: "5"         # lower threshold for high-priority
---
# Low-priority queue -- conservative scaling
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: celery-low-priority-scaler
  namespace: default
spec:
  scaleTargetRef:
    name: celery-worker-low
  minReplicaCount: 0             # scale to zero when idle
  maxReplicaCount: 10
  pollingInterval: 30            # check less frequently
  cooldownPeriod: 300            # wait 5 minutes before scaling down
  triggers:
    - type: kubemq
      metadata:
        address: "kubemq.default.svc:50000"
        channel: "low-priority"
        queueLength: "50"        # higher threshold for low-priority
```

<Callout type="info">
  Scale-to-zero (`minReplicaCount: 0`) only fires when the queue is empty for the full `cooldownPeriod`. Pair it with a longer `pollingInterval` to avoid flapping on bursty low-priority traffic.
</Callout>

## Resource Recommendations [#resource-recommendations]

Size the broker by sustained message throughput, and size workers by their concurrency and whether the workload is I/O- or CPU-bound.

### KubeMQ broker [#kubemq-broker]

| Load                    | CPU Request | CPU Limit | Memory Request | Memory Limit | Replicas |
| ----------------------- | ----------- | --------- | -------------- | ------------ | -------- |
| Low (\< 100 msg/s)      | 250m        | 500m      | 256Mi          | 512Mi        | 1        |
| Medium (100–1000 msg/s) | 500m        | 1000m     | 512Mi          | 1Gi          | 3        |
| High (> 1000 msg/s)     | 1000m       | 2000m     | 1Gi            | 2Gi          | 3        |

### Celery workers [#celery-workers]

| Concurrency   | CPU Request | CPU Limit | Memory Request | Memory Limit |
| ------------- | ----------- | --------- | -------------- | ------------ |
| 1 (I/O bound) | 100m        | 500m      | 128Mi          | 256Mi        |
| 4 (default)   | 250m        | 1000m     | 256Mi          | 512Mi        |
| 8 (CPU bound) | 500m        | 2000m     | 512Mi          | 1Gi          |

## Health Checks [#health-checks]

### Worker health [#worker-health]

The transport implements `verify_connection()`, which pings the KubeMQ broker. Celery's built-in inspect commands rely on it, so the same calls used in the Deployment probes also work for manual diagnostics:

```bash title="Worker diagnostics"
# Check if workers are alive
celery -A myapp inspect ping

# List active tasks
celery -A myapp inspect active

# Check worker stats
celery -A myapp inspect stats
```

### Broker health [#broker-health]

KubeMQ accepts TCP probes on its gRPC port. Use these on the broker StatefulSet:

```yaml title="Broker probes"
readinessProbe:
  tcpSocket:
    port: 50000
  initialDelaySeconds: 5
  periodSeconds: 10
livenessProbe:
  tcpSocket:
    port: 50000
  initialDelaySeconds: 15
  periodSeconds: 20
```

### Dashboard [#dashboard]

The KubeMQ Management API dashboard provides broker-level monitoring — queue depth, message rates, and channel statistics — on port `8080` (the shared HTTP server on `9090` serves REST and the `/health` probe). Port-forward to reach the dashboard locally:

```bash title="Open the dashboard"
# Port-forward to access the Management API dashboard locally
kubectl port-forward svc/kubemq 8080:8080

# Open http://localhost:8080 in your browser
```

The dashboard complements [Flower](https://flower.readthedocs.io/)'s task-level view: Flower shows tasks and workers, while the KubeMQ dashboard shows the channels backing them.

## Production Checklist [#production-checklist]

<Steps>
  <Step>
    **Broker availability** — deploy the KubeMQ broker with `>= 3` replicas for HA and configure persistent volume claims for its data.
  </Step>

  <Step>
    **Security** — set a KubeMQ authentication token on both the broker and the workers, and enable TLS for gRPC connections with `kubemq+tls://`.
  </Step>

  <Step>
    **Resilience** — set worker resource requests and limits, configure readiness and liveness probes, and add a dead letter queue for failed messages.
  </Step>

  <Step>
    **Scaling** — configure a KEDA `ScaledObject` for queue-depth autoscaling and match worker `--concurrency` to the workload type (I/O vs CPU bound).
  </Step>

  <Step>
    **Operations** — deploy Flower for task monitoring, set `result_expires` to `<= 86400` (24 hours), and apply network policies that restrict broker access to worker namespaces.
  </Step>
</Steps>

## Related [#related]

<Cards>
  <Card title="Configuration" href="/integrations/celery/how-to/configuration" description="Broker transport options, TLS/mTLS, async transport, and result backend settings." />

  <Card title="Migration" href="/integrations/celery/how-to/migration" description="Move an existing Celery app from Redis or RabbitMQ to KubeMQ." />

  <Card title="FastAPI Task Dispatch with KEDA" href="/integrations/celery/scenarios/fastapi-tasks-kubernetes" description="A FastAPI service dispatching Celery tasks to KubeMQ-backed workers, autoscaled by queue depth." />
</Cards>