Getting Started
Connect a Google Cloud Pub/Sub SDK to KubeMQ on gRPC port 8085 via PUBSUB_EMULATOR_HOST, then run a publish, pull, and acknowledge round-trip in minutes.
Get a message flowing through the KubeMQ Google Cloud Pub/Sub connector in minutes. You point
a standard Pub/Sub SDK at the connector's gRPC endpoint, create a topic and a subscription,
publish a message, and pull it back — all over the genuine Pub/Sub v1 wire protocol, with no
emulator to install and no KubeMQ SDK. The only change versus a real-GCP app is one environment
variable: PUBSUB_EMULATOR_HOST.
Prerequisites
- A running kubemq-server with the Pub/Sub connector enabled and reachable on gRPC port 8085. The connector is opt-in (disabled by default) — see the enable step below.
- One of the first-party Google Cloud Pub/Sub clients below for your language. There is no KubeMQ SDK; you use the official Google client with only the emulator host set.
- No credentials. When
PUBSUB_EMULATOR_HOSTis set, the SDK clears its Google credentials, skips Google auth, and dials insecure gRPC — exactly as against Google's local emulator.
Enable the connector
The GCP Pub/Sub connector is disabled by default — a stock kubemq-server does not bind gRPC port 8085 until you turn it on. Enable it with its enable variable:
docker run -d \ --name kubemq \ -p 8085:8085 \ -p 50000:50000 \ -e KUBEMQ_TOKEN=YOUR_LICENSE_KEY \ -e CONNECTORS_GCP_ENABLE=true \ europe-docker.pkg.dev/kubemq/images/kubemq:nextThe enable variable is CONNECTORS_GCP_ENABLE. A stock server does not serve Pub/Sub
until you set this to true. For Kubernetes, set spec.gcp.enabled: true in the
KubemqCluster CR.
Connect the SDK
Every official Pub/Sub client library and gcloud honour the standard PUBSUB_EMULATOR_HOST
environment variable. Export it (and any project id) before running your app:
export PUBSUB_EMULATOR_HOST=localhost:8085 # connector default gRPC port; SDK uses the insecure path
export PUBSUB_PROJECT_ID=my-project # any id; the project segment is parsed but ignored
# Some clients and gcloud also read this alias:
# export GOOGLE_CLOUD_PROJECT=my-projectThe project id is parsed but ignored. The connector is single-tenant (like the emulator),
so resource ids are global across projects — topic orders is always the Events Store log
gcp.orders regardless of the project segment. Any project id works.
How it works
CreateTopic("orders") maps the topic to the KubeMQ Events Store log gcp.orders;
CreateSubscription("sub-orders") maps the subscription to the Queue channel
gcp.sub.sub-orders. Publish writes once to the topic log through the message broker, then
fans out one Queue copy per subscription; Pull returns the message plus an opaque ack_id,
and Acknowledge removes it from the subscription queue.
The topic orders maps to the Events Store log gcp.orders; a publish fans out one copy to the subscription queue gcp.sub.sub-orders, and the pull returns it with a node-local ack_id.
Steps
Point the SDK at the connector
Build a standard Google Cloud Pub/Sub client. Most clients auto-detect the emulator from
PUBSUB_EMULATOR_HOST (default localhost:8085); three need a one-line opt-in — C# sets
EmulatorDetection.EmulatorOnly, Ruby passes emulator_host:, and Java points a
plaintext ManagedChannel at the host with NoCredentialsProvider.
import (
"context"
"os"
"cloud.google.com/go/pubsub"
)
// pubsub.NewClient auto-reads PUBSUB_EMULATOR_HOST and dials insecurely — no flag.
client, err := pubsub.NewClient(ctx, os.Getenv("PUBSUB_PROJECT_ID"))from google.cloud import pubsub_v1
# Both clients honour PUBSUB_EMULATOR_HOST automatically.
publisher = pubsub_v1.PublisherClient()
subscriber = pubsub_v1.SubscriberClient()import com.google.api.gax.core.NoCredentialsProvider;
import com.google.api.gax.grpc.GrpcTransportChannel;
import com.google.api.gax.rpc.FixedTransportChannelProvider;
import com.google.api.gax.rpc.TransportChannelProvider;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
String emulatorHost = System.getenv().getOrDefault("PUBSUB_EMULATOR_HOST", "localhost:8085");
// Java needs the emulator host wired explicitly: a plaintext channel + no credentials.
ManagedChannel channel = ManagedChannelBuilder.forTarget(emulatorHost).usePlaintext().build();
TransportChannelProvider channelProvider =
FixedTransportChannelProvider.create(GrpcTransportChannel.create(channel));
NoCredentialsProvider noCreds = NoCredentialsProvider.create();import { PubSub, v1 } from "@google-cloud/pubsub";
const projectId = process.env["PUBSUB_PROJECT_ID"] ?? "my-project";
// The high-level client reads PUBSUB_EMULATOR_HOST; reuse its options for the v1 clients.
const baseOptions = new PubSub({ projectId }).options;
const options = { ...baseOptions, port: baseOptions.port === undefined ? undefined : Number(baseOptions.port) };
const publisher = new v1.PublisherClient(options);
const subscriber = new v1.SubscriberClient(options);using Google.Api.Gax;
using Google.Cloud.PubSub.V1;
// The .NET client does NOT auto-detect the emulator — set EmulatorOnly explicitly.
var publisher = await new PublisherServiceApiClientBuilder
{
EmulatorDetection = EmulatorDetection.EmulatorOnly,
}.BuildAsync();
var subscriber = await new SubscriberServiceApiClientBuilder
{
EmulatorDetection = EmulatorDetection.EmulatorOnly,
}.BuildAsync();require "google/cloud/pubsub"
# Ruby needs emulator_host passed explicitly (it does not always read the env var).
pubsub = Google::Cloud::PubSub.new(
project_id: ENV["PUBSUB_PROJECT_ID"] || "my-project",
emulator_host: ENV["PUBSUB_EMULATOR_HOST"] || "localhost:8085"
)Create a topic and publish
CreateTopic("orders") registers the topic and maps it to the Events Store log gcp.orders;
CreateSubscription("sub-orders") maps to the Queue channel gcp.sub.sub-orders. Publish
writes the message once to the topic log and returns a server-assigned message id.
topic, _ := client.CreateTopic(ctx, "orders") // -> gcp.orders
sub, _ := client.CreateSubscription(ctx, "sub-orders", pubsub.SubscriptionConfig{
Topic: topic,
AckDeadline: 10 * time.Second, // connector default; valid range 10..600s.
}) // -> gcp.sub.sub-orders
id, _ := topic.Publish(ctx, &pubsub.Message{Data: []byte("hello kubemq")}).Get(ctx)
fmt.Println("published:", id)topic_path = publisher.topic_path(proj, "orders") # -> gcp.orders
sub_path = subscriber.subscription_path(proj, "sub-orders") # -> gcp.sub.sub-orders
publisher.create_topic(request={"name": topic_path})
subscriber.create_subscription(request={"name": sub_path, "topic": topic_path})
future = publisher.publish(topic_path, b"hello kubemq")
print("published:", future.result(timeout=15))TopicName topic = TopicName.of(projectId, "orders"); // -> gcp.orders
SubscriptionName sub = SubscriptionName.of(projectId, "sub-orders"); // -> gcp.sub.sub-orders
topicAdmin.createTopic(topic);
subAdmin.createSubscription(sub, topic, PushConfig.getDefaultInstance(), 10);
topicAdmin.publish(PublishRequest.newBuilder()
.setTopic(topic.toString())
.addMessages(PubsubMessage.newBuilder()
.setData(ByteString.copyFromUtf8("hello kubemq")).build())
.build());const topic = publisher.projectTopicsPath(projectId, "orders"); // -> gcp.orders
const sub = subscriber.subscriptionPath(projectId, "sub-orders"); // -> gcp.sub.sub-orders
await publisher.createTopic({ name: topic });
await subscriber.createSubscription({ name: sub, topic, ackDeadlineSeconds: 10 });
const [published] = await publisher.publish({
topic,
messages: [{ data: Buffer.from("hello kubemq") }],
});
console.log("published:", published.messageIds?.[0]);var topicName = TopicName.FromProjectTopic(projectId, "orders"); // -> gcp.orders
var subName = SubscriptionName.FromProjectSubscription(projectId, "sub-orders"); // -> gcp.sub.sub-orders
await publisher.CreateTopicAsync(topicName);
await subscriber.CreateSubscriptionAsync(subName, topicName, pushConfig: null, ackDeadlineSeconds: 10);
var published = await publisher.PublishAsync(topicName, new[]
{
new PubsubMessage { Data = ByteString.CopyFromUtf8("hello kubemq") },
});
Console.WriteLine($"published: {published.MessageIds[0]}");topic_path = pubsub.topic_path("orders") # -> gcp.orders
sub_path = pubsub.subscription_path("sub-orders") # -> gcp.sub.sub-orders
topic = pubsub.topic_admin.create_topic(name: topic_path)
pubsub.subscription_admin.create_subscription(name: sub_path, topic: topic_path, ack_deadline_seconds: 10)
msg = pubsub.publisher(topic.name).publish("hello kubemq")
puts "published: #{msg.message_id}"Pull and acknowledge
Pull returns the message plus an opaque ack_id, holding it under an ack-deadline lease.
Acknowledge(ack_id) removes it from the subscription queue. A successful run prints the body
you published.
recvCtx, cancel := context.WithTimeout(ctx, 15*time.Second)
defer cancel()
sub.Receive(recvCtx, func(_ context.Context, m *pubsub.Message) {
fmt.Printf("received: %q\n", string(m.Data))
m.Ack() // Acknowledge by ack_id under its lease.
cancel()
})resp = subscriber.pull(request={"subscription": sub_path, "max_messages": 1}, timeout=20)
msg = resp.received_messages[0]
print("received:", msg.message.data.decode())
subscriber.acknowledge(request={"subscription": sub_path, "ack_ids": [msg.ack_id]})PullResponse resp = subStub.pullCallable().call(PullRequest.newBuilder()
.setSubscription(sub.toString()).setMaxMessages(1).build());
ReceivedMessage got = resp.getReceivedMessages(0);
System.out.printf("received: %s%n", got.getMessage().getData().toStringUtf8());
subStub.acknowledgeCallable().call(AcknowledgeRequest.newBuilder()
.setSubscription(sub.toString()).addAckIds(got.getAckId()).build());const [pull] = await subscriber.pull({ subscription: sub, maxMessages: 1 });
const received = pull.receivedMessages![0];
console.log("received:", Buffer.from(received.message!.data as Uint8Array).toString("utf8"));
await subscriber.acknowledge({ subscription: sub, ackIds: [received.ackId!] });var pull = await subscriber.PullAsync(subName, maxMessages: 1);
var received = pull.ReceivedMessages[0];
Console.WriteLine($"received: {received.Message.Data.ToStringUtf8()}");
await subscriber.AcknowledgeAsync(subName, new[] { received.AckId });subscriber = pubsub.subscriber(sub_path)
rcv = subscriber.pull(immediate: false, max: 1).first
puts "received: #{rcv.data.inspect}"
rcv.acknowledge!The ack_id is node-local under StreamingPull — a lease minted on one node is invalid on
another. In a clustered deployment, pin a subscription's StreamingPull traffic to one node
(session-affinity load balancer), or accept at-least-once delivery across nodes. See
Subscribing.
Next steps
Configuration
The opt-in CONNECTORS_GCP_ENABLE variable, gRPC port 8085, and the thirteen CONNECTORS_GCP_* settings.
Architecture
The gRPC emulator listener, the 38-RPC surface, and how topics and subscriptions map to KubeMQ primitives.
Publish & Subscribe
The core topic-to-subscription round-trip with complete multi-language code.
Connectivity & emulator mode
The PUBSUB_EMULATOR_HOST drop-in, per-language emulator opt-in, and the insecure-gRPC posture.
Was this page helpful?
Cross-Protocol Interop
Publish with a Google Pub/Sub SDK and consume natively with a KubeMQ client on the Events Store log gcp.{topic}, carrying the reserved _pubsub_* tags across.
Connectivity and emulator mode
How a Google Pub/Sub SDK reaches the connector — the PUBSUB_EMULATOR_HOST drop-in, per-language emulator opt-in, insecure gRPC, and the cluster caveat.