Skip to main content

Setting Up a Pub/Sub Topic

This guide walks you through creating a managed Pub/Sub messaging topic and connecting it to your applications for event-driven communication.

Pub/Sub topics enable asynchronous, loosely-coupled communication between microservices. This is ideal for background processing, event streaming, and cross-service notifications. Infrastream manages all underlying infrastructure, including encryption, regional data persistence, and IAM bindings.

Prerequisites​

  • You must have at least one Application manifest that will publish to or subscribe from the topic.
  • You need to know the identity of your project: organization, organizational-unit, environment, and project.

Step 1: Create a New Manifest File for the Topic​

Create a new YAML file for your Pub/Sub topic inside your infrastream-manifests Git repository.

A common convention is to place topics in a pubsub subdirectory: ../project/{project-name}/pubsub/{topic-name}.yaml

Step 2: Define Your PubSub Manifest​

Open the file and define the topic configuration:

apiVersion: lowops.manifests.v1
kind: PubSub
metadata:
  name: order-events
  project: payment-gateway
  environment: production
  organizational-unit: retail-banking
  organization: fincorp
spec:
  description: "Events emitted when an order is created, updated, or completed"
  encrypted: true                          # CMEK encryption for message payloads
  allowedPersistenceRegions:               # Restrict where messages are stored
    - us-central1
    - us-east1
  messageRetentionDurationSeconds: 604800  # 7 days retention (in seconds)

Key fields:

  • encrypted: true provisions a dedicated KMS key ring and crypto key for envelope encryption
  • allowedPersistenceRegions enforces a message storage policy, restricting where data can physically reside
  • messageRetentionDurationSeconds configures the backlog retention window for unacknowledged messages

Step 3: Grant Your Applications Access to the Topic​

By default, no application can publish to or subscribe from the topic. You must explicitly configure access in each application's manifest.

Publishing to the topic​

In the publisher application's manifest:

# In your Application manifest (e.g., order-service.yaml)
# ... apiVersion, kind, metadata ...
spec:
  # ... other application settings ...

  accessControl:
    pubsub:
      publishTo:
        - order-events          # Grants roles/pubsub.publisher

Subscribing to the topic​

In the subscriber application's manifest:

# In your Application manifest (e.g., analytics-worker.yaml)
# ... apiVersion, kind, metadata ...
spec:
  # ... other application settings ...

  accessControl:
    pubsub:
      subscribeTo:
        - order-events          # Grants roles/pubsub.subscriber

Tip: An application can both publish and subscribe to topics. Just include both publishTo and subscribeTo lists in the same pubsub block.

Step 4: Commit, Review, and Merge​

Commit the new PubSub manifest and any updated Application manifests in a single pull request.

After your PR is reviewed and approved, merge it. The platform will automatically:

  1. Provision the Pub/Sub topic with the specified encryption and retention settings.
  2. Create the least-privilege IAM bindings (roles/pubsub.publisher and/or roles/pubsub.subscriber) on each application's service account.
  3. Restart affected applications with the new permissions.