Celery

kubemq-celery is a KubeMQ transport and queue-peek result backend for Celery task queues. It lets you run Celery on KubeMQ with a one-line configuration change — broker="kubemq://localhost:50000" — making KubeMQ the only Kubernetes-native, in-cluster Celery broker. The transport plugs into Kombu (Celery's messaging layer) and registers the kubemq:// URL scheme, so your existing tasks, workers, and tooling keep working unchanged.

New to the idea? See what is an integration for how SDK-level integrations differ from the server-side connectors.

Supported versions

Why KubeMQ over Redis/RabbitMQ?

KubeMQ replaces visibility-timeout heuristics and broker plugins with native gRPC primitives, and runs as a first-class workload inside your Kubernetes cluster.

• No visibility-timeout bugs — messages are explicitly acknowledged or rejected over a gRPC stream, instead of relying on a timeout window that causes duplicates when set too low and stalls reprocessing when set too high.
• Native delayed delivery — countdown and eta map directly to KubeMQ's delay_in_seconds with zero polling overhead and no broker plugin.
• Kubernetes-native — an in-cluster broker with auto-clustering, gRPC keep-alive for long-lived connections, and KEDA-driven autoscaling, instead of an external StatefulSet or an Erlang-backed service.

Install

Install from PyPI with pip or uv:

# pip
pip install kubemq-celery

# uv (recommended)
uv add kubemq-celery

Switching an existing app is a one-line change — kubemq-celery is a drop-in replacement for Redis or RabbitMQ:

import kubemq_celery  # registers the kubemq:// transport
from celery import Celery

app = Celery("myapp", broker="kubemq://localhost:50000")

@app.task
def add(x, y):
    return x + y

Start a worker and send a task:

celery -A tasks worker --loglevel=info

from tasks import add

add.delay(4, 6)

For the full walkthrough — including a local Docker broker and the result backend — see Getting Started.

Features

Architecture

A Celery worker talks to KubeMQ through Kombu's virtual transport layer. kubemq-celery implements a Channel that extends Kombu's virtual.Channel (with supports_fanout = True) and maps Kombu's storage primitives onto KubeMQ's gRPC API on port 50000. Task messages travel over KubeMQ Queues (with native ack/nack and delay_in_seconds), while pidbox and monitoring traffic — Flower, celery inspect, celery control — use KubeMQ Events fanout. When the optional result backend is enabled, results are written as Queue messages and read back with a non-destructive peek, so multiple callers can fetch the same result without consuming it.

A Celery worker speaks kubemq:// through Kombu; tasks and peek-read results ride KubeMQ Queues, while pidbox and monitoring traffic fan out over KubeMQ Events.

Usage

Quick Links

New to KubeMQ? Start with the KubeMQ Getting Started guide for core concepts like Queues and Events before wiring up Celery.