Message filtering

A subscription may carry a filter so it only receives messages whose attributes match. The connector implements a hand-written CEL-subset (a strict subset of Google's filter expression language) — attributes-only, compiled once at create-time, and applied at publish fan-out.

Where the filter runs

A filter is set on CreateSubscription and is compiled once and immutable thereafter (UpdateSubscription cannot change filter). It is applied at publish fan-out: when a Publish writes the topic log and fans copies out to each subscription, a message that does not match a subscription's filter is never enqueued for that subscription (it is effectively auto-acked for it). The topic log gcp.{topic} itself is unfiltered — the filter only governs which subscription queues receive a copy. See Publishing and Channel mapping.

Supported syntax

Filters operate on the message's attributes map only (not data, not the ordering key):

attributes:KEY                       -- attribute KEY exists  (KEY may be quoted: attributes:"k")
attributes.KEY = "v"                 -- equality
attributes.KEY != "v"                -- inequality
hasPrefix(attributes.KEY, "p")       -- value has the prefix "p"
AND  OR  NOT  -                      -- boolean operators (NOT and unary - both negate)
( … )                                -- parentheses for grouping

Rules:

• Attributes-only. There is no data-based filtering and no numeric / comparison operators beyond = / != / hasPrefix.
• ≤ 256 characters. A filter expression longer than 256 chars is rejected.
• Immutable. Compiled at CreateSubscription; cannot be changed by UpdateSubscription.
• Malformed → INVALID_ARGUMENT. A filter that fails to parse is rejected at create-time, not silently ignored.

Examples

A filter is just a field on the subscription config:

from google.cloud import pubsub_v1

subscriber = pubsub_v1.SubscriberClient()    # honours PUBSUB_EMULATOR_HOST
subscriber.create_subscription(request={
    "name": subscriber.subscription_path("my-project", "sub-eu"),
    "topic": "projects/my-project/topics/orders",
    "filter": 'attributes.region = "eu"',    # compiled once; immutable
})

How attributes map

Message attributes round-trip as KubeMQ message tags. The connector also carries three reserved tags (_pubsub_message_id, _pubsub_publish_time, _pubsub_ordering_key); filters operate on the user attributes, not the reserved tags. See Publishing.

The fan-out pattern

Filtering is the basis of the fan-out pattern: attach N subscriptions to one topic, each with its own filter, and a single Publish is fanned out only to the subscriptions whose filter matches. See Fan-out.

Error quick reference

Related