KubeMQ
ConnectorsGoogle Cloud Pub/Sub

Google Cloud Pub/Sub

Point an unmodified Google Cloud Pub/Sub app at KubeMQ by setting PUBSUB_EMULATOR_HOST and run topics and subscriptions over the embedded gRPC connector.

Point your Google Cloud Pub/Sub application at KubeMQ by setting one environment variablePUBSUB_EMULATOR_HOST. The Google Cloud Pub/Sub connector is a built-in, wire-protocol bridge inside kubemq-server that speaks the genuine Pub/Sub v1 gRPC services on a dedicated gRPC listener (default port 8085, the Pub/Sub emulator convention). Any standard, unmodified Pub/Sub client — the Go, Python, Java, Node.js, C#, and Ruby first-party Google clients, plus gcloud pubsub — talks to KubeMQ exactly as it would to Google's local emulator, with no code changes, no library swap, and no KubeMQ SDK.

What is the Pub/Sub connector

The connector is a single gRPC listener that implements the real Pub/Sub v1 wire protocol — 38 RPCs across four services:

  • google.pubsub.v1.Publisher (9 RPCs) — topics and publish.
  • google.pubsub.v1.Subscriber (16 RPCs) — subscriptions, pull, streaming pull, ack, snapshots, and seek.
  • google.pubsub.v1.SchemaService (10 RPCs) — Avro and Protobuf schemas.
  • google.iam.v1.IAMPolicy (3 permissive stubs) — emulator-parity IAM.

Every official Pub/Sub client library honours the PUBSUB_EMULATOR_HOST environment variable: when it is set, the SDK clears credentials, skips Google auth, and dials insecure gRPC — exactly as it would against Google's local emulator. The connector is the emulator — there is no separate emulator to install, no LocalStack, and no boot-the-server step beyond running kubemq-server.

Two KubeMQ primitives back the model: a topic maps onto a native KubeMQ Events Store log gcp.{topic} (the authoritative, replayable source of truth), and each subscription maps onto a native Queue channel gcp.sub.{subscription}. A publish is written once to the topic log, then fanned out to one queue copy per subscription — so Pub/Sub producers and native gRPC/REST consumers interoperate on the same messages.

The Pub/Sub connector is opt-in (disabled by default). A stock kubemq-server does not bind gRPC port 8085 until you enable it with CONNECTORS_GCP_ENABLE=true (Docker) or spec.gcp.enabled: true (Kubernetes). See Getting started.

How it works

A Pub/Sub SDK client dials the connector's gRPC endpoint. A Publish is written once to the topic's Events Store log gcp.{topic} through the message broker, then fanned out to one Queue copy per subscription gcp.sub.{subscription} (applying each subscription's filter). Pull / StreamingPull lease each delivered message under an ack-deadline, and Acknowledge removes it from the subscription queue.

A publish writes once to the Events Store log gcp.{topic} and fans out one Queue copy per subscription on gcp.sub.{subscription}, all backed by the message broker; consumers pull each message under an ack-deadline lease.

Ports & protocol surface

PortTransportProtocolNotes
8085Insecure gRPCPub/Sub v1 gRPC (emulator mode)The Pub/Sub emulator convention. Opt-in — bound only when CONNECTORS_GCP_ENABLE=true. Must differ from the gRPC/REST/HTTP and AWS-connector ports. No auth, no TLS — exactly like Google's local emulator.
gRPC over TLSPub/Sub v1 gRPCTLS is provided by the server-wide Security block — there is no Pub/Sub-specific TLS option. The default emulator path is unencrypted; do not expose port 8085 to untrusted networks.

The connector is gRPC only — there is no REST/JSON v1 surface (no grpc-gateway). Clients and tools that only speak the Pub/Sub REST API will not work; use a gRPC client library or gcloud (which uses gRPC for the emulator). See Architecture for the dispatch detail.

Send a message

The example below runs the full round-trip — CreateTopicCreateSubscriptionPublishPullAcknowledge — over a stock Google Cloud Pub/Sub client. The only change versus a real-GCP app is the emulator endpoint: each client reads PUBSUB_EMULATOR_HOST (default localhost:8085) and a project id from PUBSUB_PROJECT_ID (any value — the project segment is parsed but ignored).

package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"sync"
	"time"

	"cloud.google.com/go/pubsub"
)

func projectID() string {
	if v := os.Getenv("PUBSUB_PROJECT_ID"); v != "" {
		return v
	}
	return "my-project" // any id; the project segment is parsed but ignored.
}

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
	defer cancel()

	// PUBSUB_EMULATOR_HOST (default localhost:8085) routes the official client at
	// the connector over insecure gRPC with credentials cleared — no code change.
	client, err := pubsub.NewClient(ctx, projectID())
	if err != nil {
		log.Fatalf("NewClient: %v", err)
	}
	defer client.Close()

	// CreateTopic "orders" maps to Events Store log "gcp.orders".
	topic, err := client.CreateTopic(ctx, "orders")
	if err != nil {
		log.Fatalf("CreateTopic: %v", err)
	}
	defer topic.Stop()

	// CreateSubscription maps to Queue channel "gcp.sub.sub-orders".
	sub, err := client.CreateSubscription(ctx, "sub-orders", pubsub.SubscriptionConfig{
		Topic:       topic,
		AckDeadline: 10 * time.Second, // connector default; valid range 10..600s.
	})
	if err != nil {
		log.Fatalf("CreateSubscription: %v", err)
	}

	id, err := topic.Publish(ctx, &pubsub.Message{Data: []byte("hello from cloud.google.com/go/pubsub")}).Get(ctx)
	if err != nil {
		log.Fatalf("Publish: %v", err)
	}
	fmt.Printf("published: %s\n", id)

	recvCtx, recvCancel := context.WithTimeout(ctx, 15*time.Second)
	defer recvCancel()
	var once sync.Once
	err = sub.Receive(recvCtx, func(_ context.Context, m *pubsub.Message) {
		fmt.Printf("received: %q\n", string(m.Data))
		m.Ack() // Acknowledge by ack_id under its lease.
		once.Do(recvCancel)
	})
	if err != nil && recvCtx.Err() == nil {
		log.Fatalf("Receive: %v", err)
	}
}
import os

from google.cloud import pubsub_v1


def project_id() -> str:
    # Any id works — the project segment is parsed but ignored.
    return os.environ.get("PUBSUB_PROJECT_ID", "my-project")


def main() -> None:
    # Both clients honour PUBSUB_EMULATOR_HOST (default localhost:8085): they clear
    # credentials, skip Google auth, and dial insecure gRPC.
    publisher = pubsub_v1.PublisherClient()
    subscriber = pubsub_v1.SubscriberClient()

    proj = project_id()
    topic_path = publisher.topic_path(proj, "orders")          # -> gcp.orders
    sub_path = subscriber.subscription_path(proj, "sub-orders")  # -> gcp.sub.sub-orders

    publisher.create_topic(request={"name": topic_path})
    subscriber.create_subscription(request={"name": sub_path, "topic": topic_path})

    future = publisher.publish(topic_path, b"hello from google-cloud-pubsub")
    print(f"published: {future.result(timeout=15)}")

    resp = subscriber.pull(request={"subscription": sub_path, "max_messages": 1}, timeout=20)
    msg = resp.received_messages[0]
    print(f"received: {msg.message.data.decode()!r}")

    # Acknowledge by ack_id; the message leaves the subscription queue.
    subscriber.acknowledge(request={"subscription": sub_path, "ack_ids": [msg.ack_id]})
    subscriber.close()


if __name__ == "__main__":
    main()
import com.google.api.gax.core.NoCredentialsProvider;
import com.google.api.gax.grpc.GrpcTransportChannel;
import com.google.api.gax.rpc.FixedTransportChannelProvider;
import com.google.api.gax.rpc.TransportChannelProvider;
import com.google.cloud.pubsub.v1.SubscriptionAdminClient;
import com.google.cloud.pubsub.v1.SubscriptionAdminSettings;
import com.google.cloud.pubsub.v1.TopicAdminClient;
import com.google.cloud.pubsub.v1.TopicAdminSettings;
import com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub;
import com.google.cloud.pubsub.v1.stub.SubscriberStubSettings;
import com.google.protobuf.ByteString;
import com.google.pubsub.v1.AcknowledgeRequest;
import com.google.pubsub.v1.PublishRequest;
import com.google.pubsub.v1.PubsubMessage;
import com.google.pubsub.v1.PullRequest;
import com.google.pubsub.v1.PullResponse;
import com.google.pubsub.v1.PushConfig;
import com.google.pubsub.v1.ReceivedMessage;
import com.google.pubsub.v1.SubscriptionName;
import com.google.pubsub.v1.TopicName;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;

public final class Main {
    public static void main(String[] args) throws Exception {
        String emulatorHost = System.getenv().getOrDefault("PUBSUB_EMULATOR_HOST", "localhost:8085");
        String projectId = System.getenv().getOrDefault("PUBSUB_PROJECT_ID", "my-project");

        // The Go/Python/Node clients auto-detect the emulator; Java points a plaintext
        // gRPC channel at the host explicitly with NoCredentialsProvider.
        ManagedChannel channel = ManagedChannelBuilder.forTarget(emulatorHost).usePlaintext().build();
        TransportChannelProvider channelProvider =
                FixedTransportChannelProvider.create(GrpcTransportChannel.create(channel));
        NoCredentialsProvider noCreds = NoCredentialsProvider.create();

        TopicName topic = TopicName.of(projectId, "orders");                 // -> gcp.orders
        SubscriptionName sub = SubscriptionName.of(projectId, "sub-orders"); // -> gcp.sub.sub-orders

        try (TopicAdminClient topicAdmin = TopicAdminClient.create(TopicAdminSettings.newBuilder()
                .setTransportChannelProvider(channelProvider).setCredentialsProvider(noCreds).build());
                SubscriptionAdminClient subAdmin = SubscriptionAdminClient.create(
                        SubscriptionAdminSettings.newBuilder()
                                .setTransportChannelProvider(channelProvider)
                                .setCredentialsProvider(noCreds).build());
                GrpcSubscriberStub subStub = GrpcSubscriberStub.create(SubscriberStubSettings.newBuilder()
                        .setTransportChannelProvider(channelProvider)
                        .setCredentialsProvider(noCreds).build())) {

            topicAdmin.createTopic(topic);
            subAdmin.createSubscription(sub, topic, PushConfig.getDefaultInstance(), 10);

            topicAdmin.publish(PublishRequest.newBuilder()
                    .setTopic(topic.toString())
                    .addMessages(PubsubMessage.newBuilder()
                            .setData(ByteString.copyFromUtf8("hello from google-cloud-pubsub")).build())
                    .build());

            PullResponse resp = subStub.pullCallable().call(PullRequest.newBuilder()
                    .setSubscription(sub.toString()).setMaxMessages(1).build());
            ReceivedMessage got = resp.getReceivedMessages(0);
            System.out.printf("received: %s%n", got.getMessage().getData().toStringUtf8());

            // Acknowledge by ack_id.
            subStub.acknowledgeCallable().call(AcknowledgeRequest.newBuilder()
                    .setSubscription(sub.toString()).addAckIds(got.getAckId()).build());
        } finally {
            channel.shutdown();
        }
    }
}
import { PubSub, v1 } from "@google-cloud/pubsub";

const projectId = process.env["PUBSUB_PROJECT_ID"] ?? "my-project";

// The high-level PubSub client reads PUBSUB_EMULATOR_HOST (default localhost:8085)
// and resolves the insecure emulator transport; reuse its options for the v1 clients.
const baseOptions = new PubSub({ projectId }).options;
const options = { ...baseOptions, port: baseOptions.port === undefined ? undefined : Number(baseOptions.port) };
const publisher = new v1.PublisherClient(options);
const subscriber = new v1.SubscriberClient(options);

async function main(): Promise<void> {
  const topic = publisher.projectTopicsPath(projectId, "orders");          // -> gcp.orders
  const sub = subscriber.subscriptionPath(projectId, "sub-orders");        // -> gcp.sub.sub-orders

  await publisher.createTopic({ name: topic });
  await subscriber.createSubscription({ name: sub, topic, ackDeadlineSeconds: 10 });

  const [published] = await publisher.publish({
    topic,
    messages: [{ data: Buffer.from("hello from @google-cloud/pubsub") }],
  });
  console.log(`published: ${published.messageIds?.[0]}`);

  const [pull] = await subscriber.pull({ subscription: sub, maxMessages: 1 });
  const received = pull.receivedMessages![0];
  console.log(`received: ${Buffer.from(received.message!.data as Uint8Array).toString("utf8")}`);

  // Acknowledge by ack_id.
  await subscriber.acknowledge({ subscription: sub, ackIds: [received.ackId!] });
}

main().catch((err) => {
  console.error(err);
  process.exit(1);
});
using Google.Api.Gax;
using Google.Cloud.PubSub.V1;
using Google.Protobuf;

var projectId = Environment.GetEnvironmentVariable("PUBSUB_PROJECT_ID") ?? "my-project";
var topicName = TopicName.FromProjectTopic(projectId, "orders");           // -> gcp.orders
var subName = SubscriptionName.FromProjectSubscription(projectId, "sub-orders"); // -> gcp.sub.sub-orders

// The .NET client does NOT auto-detect the emulator — set EmulatorOnly so each
// client reads PUBSUB_EMULATOR_HOST (default localhost:8085) and dials insecurely.
var publisher = await new PublisherServiceApiClientBuilder
{
    EmulatorDetection = EmulatorDetection.EmulatorOnly,
}.BuildAsync();
var subscriber = await new SubscriberServiceApiClientBuilder
{
    EmulatorDetection = EmulatorDetection.EmulatorOnly,
}.BuildAsync();

await publisher.CreateTopicAsync(topicName);
await subscriber.CreateSubscriptionAsync(subName, topicName, pushConfig: null, ackDeadlineSeconds: 10);

var publishResponse = await publisher.PublishAsync(topicName, new[]
{
    new PubsubMessage { Data = ByteString.CopyFromUtf8("hello from Google.Cloud.PubSub.V1") },
});
Console.WriteLine($"published: {publishResponse.MessageIds[0]}");

var pull = await subscriber.PullAsync(subName, maxMessages: 1);
var received = pull.ReceivedMessages[0];
Console.WriteLine($"received: {received.Message.Data.ToStringUtf8()}");

// Acknowledge by ack_id.
await subscriber.AcknowledgeAsync(subName, new[] { received.AckId });
# frozen_string_literal: true

require "google/cloud/pubsub"

project_id = ENV["PUBSUB_PROJECT_ID"] || "my-project"
# Ruby needs the emulator host passed explicitly (it does not always read the env var).
emulator_host = ENV["PUBSUB_EMULATOR_HOST"] || "localhost:8085"

pubsub = Google::Cloud::PubSub.new(project_id: project_id, emulator_host: emulator_host)
topic_admin = pubsub.topic_admin
sub_admin   = pubsub.subscription_admin

topic_path = pubsub.topic_path("orders")            # -> gcp.orders
sub_path   = pubsub.subscription_path("sub-orders") # -> gcp.sub.sub-orders

topic = topic_admin.create_topic(name: topic_path)
sub_admin.create_subscription(name: sub_path, topic: topic_path, ack_deadline_seconds: 10)

publisher = pubsub.publisher(topic.name)
msg = publisher.publish("hello from google-cloud-pubsub")
puts "published: #{msg.message_id}"

subscriber = pubsub.subscriber(sub_path)
received = subscriber.pull(immediate: false, max: 1)
rcv = received.first
puts "received: #{rcv.data.inspect}"

# Acknowledge by ack_id; the message leaves the subscription queue.
rcv.acknowledge!

Supported languages

The connector speaks the genuine Pub/Sub v1 wire protocol, so any first-party Google Cloud Pub/Sub client works — you only set PUBSUB_EMULATOR_HOST. There is no KubeMQ SDK, no proto bindings, and no published package; the examples pin one official Google client per language.

LanguageGoogle Cloud Pub/Sub clientEmulator construction
Gocloud.google.com/go/pubsubpubsub.NewClient(ctx, projectID) — auto-detects PUBSUB_EMULATOR_HOST, dials insecurely
Pythongoogle-cloud-pubsub (via uv)pubsub_v1.PublisherClient() — honours PUBSUB_EMULATOR_HOST
Javacom.google.cloud:google-cloud-pubsub (BOM)plaintext ManagedChannel to the emulator host + NoCredentialsProvider
JavaScript / TypeScript@google-cloud/pubsubnew PubSub({ projectId }) — auto-detects the emulator; run via tsx
C# / .NETGoogle.Cloud.PubSub.V1 (.NET 8)…Builder { EmulatorDetection = EmulatorDetection.EmulatorOnly }.Build()
Rubygoogle-cloud-pubsubGoogle::Cloud::PubSub.new(project_id:, emulator_host:)

There is no Rust tab — Google ships no first-party Pub/Sub client for Rust. The connector's verified example suite is the six languages above. Most clients auto-detect the emulator from PUBSUB_EMULATOR_HOST; C# needs EmulatorDetection.EmulatorOnly, Ruby needs an explicit emulator_host:, and Java points a plaintext channel at the host — see Connectivity & emulator mode.

Next steps

Was this page helpful?

On this page