Scale to Zero with Push Mode

For spiky workloads — GPU inference workers, nightly batch jobs, on-demand task processors — running a consumer continuously wastes money when the queue is empty. The goal of scale-to-zero is to run zero replicas while idle and spin a consumer up the instant work arrives, then drain it back to zero once the queue clears. The KubeMQ KEDA external scaler supports this by reporting the Waiting message count of a queue channel and letting KEDA decide when to activate and how many replicas to run.

Prerequisites

• KEDA installed in the cluster and the KubeMQ external scaler deployed (see Getting Started)
• A KubeMQ broker reachable from the scaler, and a Deployment whose replica count KEDA is allowed to manage

Poll Mode vs. Push Mode

The scaler supports two KEDA trigger types, and the difference matters most when scaling from zero.

In poll mode, the soonest KEDA can notice the first message is on the next pollingInterval tick. In push mode, the scaler holds the stream open and pushes an active signal as soon as it detects waiting work, so the workload wakes up sooner without forcing a tiny pollingInterval across the whole metric pipeline.

The Scale-to-Zero ScaledObject

This is the scaled-object-scale-to-zero.yaml example from the scaler repo. It uses external-push, idles at zero replicas, and activates on the first waiting message.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: scale-to-zero-scaler
spec:
  scaleTargetRef:
    name: batch-processor
  pollingInterval: 10
  cooldownPeriod: 120
  minReplicaCount: 0
  maxReplicaCount: 5
  triggers:
    - type: external-push
      metadata:
        scalerAddress: <SCALER_SERVICE>.<NAMESPACE>.svc.cluster.local:9090
        kubemqAddress: kubemq.default.svc.cluster.local:50000
        queueName: batch-jobs
        targetWaiting: "1"
        activationTargetWaiting: "0"

Key fields:

• minReplicaCount: 0 — the workload idles at zero replicas.
• maxReplicaCount: 5 — the ceiling once work arrives.
• targetWaiting: "1" — one waiting message per replica, so KEDA adds a replica for roughly every queued message up to the max.
• activationTargetWaiting: "0" — activate as soon as Waiting exceeds 0.

The Activation Threshold

Scaling from zero is a two-stage decision in KEDA: first activate (go from 0 to 1), then scale based on the metric. The activation stage is governed by activationTargetWaiting.

activationTargetWaiting is the minimum Waiting count that activates scaling. It defaults to 0, which means the scaler reports active as soon as Waiting is strictly greater than 0 — the very first message wakes the workload. In the scaler, IsActive and the push stream both compute the active flag the same way:

return &pb.IsActiveResponse{
    Result: float64(waiting) > meta.ActivationTargetWaiting,
}, nil

The default is applied in metadata parsing, so omitting the field is equivalent to setting "0":

activationTargetWaiting := 0.0
if v := metadata["activationTargetWaiting"]; v != "" {
    parsed, err := strconv.ParseFloat(v, 64)
    if err != nil {
        return nil, status.Error(codes.InvalidArgument, "invalid activationTargetWaiting: must be a number")
    }
    if parsed < 0 || math.IsNaN(parsed) || math.IsInf(parsed, 0) {
        return nil, status.Error(codes.InvalidArgument, "activationTargetWaiting must be a finite non-negative number")
    }
    activationTargetWaiting = parsed
}

The comparison is strictly greater-than, so with the default 0 a single message (Waiting == 1) activates, but activationTargetWaiting: "1" would require Waiting >= 2 before waking up.

How StreamIsActive Pushes Updates

When KEDA opens the external-push stream, the scaler does the work below. It sends the active status once immediately, then re-polls on a fixed 5-second interval and pushes each update. Transient poll failures are logged and skipped rather than tearing down the stream:

const streamIsActiveInterval = 5 * time.Second

func (s *ExternalScaler) StreamIsActive(ref *pb.ScaledObjectRef, stream grpc.ServerStreamingServer[pb.IsActiveResponse]) error {
    meta, err := ParseScalerMetadata(ref.ScalerMetadata)
    if err != nil {
        return err
    }

    waiting, err := s.getWaiting(stream.Context(), meta)
    if err != nil {
        return mapKubeMQError(err)
    }
    if sendErr := stream.Send(&pb.IsActiveResponse{
        Result: float64(waiting) > meta.ActivationTargetWaiting,
    }); sendErr != nil {
        return sendErr
    }

    ticker := time.NewTicker(s.streamInterval)
    defer ticker.Stop()

    for {
        select {
        case <-stream.Context().Done():
            return nil
        case <-ticker.C:
            waiting, err := s.getWaiting(stream.Context(), meta)
            if err != nil {
                s.logger.Warn("StreamIsActive poll failed",
                    "kubemq_address", meta.KubeMQAddress,
                    "queue_name", meta.QueueName,
                    "error", err,
                )
                continue
            }
            if sendErr := stream.Send(&pb.IsActiveResponse{
                Result: float64(waiting) > meta.ActivationTargetWaiting,
            }); sendErr != nil {
                return sendErr
            }
        }
    }
}

The fixed streamIsActiveInterval of 5 seconds — independent of the ScaledObject's pollingInterval — is what makes scale-from-zero responsive in push mode. A transient KubeMQ error during a tick logs a warning and continues the loop, so a brief broker hiccup will not collapse the stream or strand the workload.

Tuning Notes

• cooldownPeriod controls how long the workload stays up after the queue drains before scaling back to zero. The example uses 120 seconds; lower it to scale down sooner, raise it to absorb bursty traffic and avoid thrashing.
• activationTargetWaiting > 0 prevents waking the workload on a single stray message. Set it to a small number (for example "1" so Waiting >= 2 activates) when an occasional one-off message should not pay the cost of a cold start.
• pollingInterval still matters in push mode. Push mode accelerates the activation decision, but KEDA continues to read the metric via GetMetrics on the pollingInterval cadence to do the replica math (how many replicas between 1 and maxReplicaCount). Keep it reasonable so replica counts track queue depth even while the stream drives activation.

Adding a Fallback Safety Net

When minReplicaCount is 0, an unreachable scaler can leave the workload stuck at zero with no consumer to drain the queue. KEDA's fallback block guards against this: if the scaler returns errors past a failure threshold, KEDA pins the workload to a fixed replica count instead. The scaled-object-ml-inference.yaml example wires this up for a GPU inference worker:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: ml-inference-scaler
spec:
  scaleTargetRef:
    name: gpu-inference-worker
  pollingInterval: 5
  cooldownPeriod: 300
  minReplicaCount: 0
  maxReplicaCount: 20
  fallback:
    failureThreshold: 3
    replicas: 2
  triggers:
    - type: external
      metadata:
        scalerAddress: <SCALER_SERVICE>.<NAMESPACE>.svc.cluster.local:9090
        kubemqAddress: kubemq.default.svc.cluster.local:50000
        queueName: inference-requests
        targetWaiting: "5"
        activationTargetWaiting: "1"

Here, after 3 consecutive scaler failures KEDA holds the workload at 2 replicas rather than scaling to zero, so requests keep draining even when the scaler cannot be reached. The scaler maps KubeMQ failures (connection refused, auth failure, timeout) to gRPC error codes precisely so KEDA's fallback logic can trigger on them. You can add the same fallback block to a push-mode ScaledObject.

Verify

kubectl apply -f scaled-object-scale-to-zero.yaml
kubectl get deploy batch-processor -w

kubectl get scaledobject scale-to-zero-scaler

Related

• Autoscale a Queue Consumer for the end-to-end poll-mode setup.
• Concepts for the external vs external-push model and the activation gate.
• ScaledObject metadata for the full trigger metadata table.
• gRPC RPCs for IsActive, StreamIsActive, GetMetricSpec, and GetMetrics.
• Queues for the core queue concept the Waiting count comes from.