TradeStream: Financial Services Platform on Infrastream

End-to-End Architecture & Deployment Guide

1. Executive Summary

Field	Detail
Project	TradeStream EU
Client	FinCorp Global (Tier-1 Investment Bank)
Partner	Stellar Technologies
Platform	Infrastream
Objective	Deploy a cloud-native, high-frequency trading analysis platform compliant with European financial regulations (GDPR/MiFID II). The system must ingest real-time market data, detect fraud using ML, and provide a secure dashboard for traders.
Compute	Google Cloud Run (Microservices), Compute Engine (Redpanda Cluster)
Data	AlloyDB (Ledger), Redpanda (Streaming), Cloud Storage (Reports)
Security	Identity-Aware Proxy (IAP), VPC Service Controls, Secret Manager
CI/CD	Infrastream Build Definitions & Release Tracks

2. Architecture Overview

This diagram illustrates the flow of data and the relationship between Infrastream manifests and the deployed infrastructure.

Architecture Diagram

3. Organization & Environment Setup

We begin by defining the organizational structure to ensure all resources are correctly tagged, billed, and governed. A specific unit for "Financial Services" is created to enforce stricter compliance policies.

OrganizationalUnit Manifest

# FILE: ./financial-services.yaml
apiVersion: lowops.manifests.v1
kind: OrganizationalUnit
metadata:
  name: financial-services
  organization: stellar-tech
spec:
  description: |
    Unit dedicated to banking, trading, and insurance clients. 
    Enforces strict compliance, audit logging, and data residency 
    requirements (GDPR/MiFID II).
  permissions:
    administrators:
      groups:
      - fin-compliance-leads
    contributors:
      groups:
      - trade-engineers

Environment Manifest

Next, we define the Production environment where live trading data will flow, setting a high logging frequency for auditability.

# FILE: ./environment/production/environment.yaml
apiVersion: lowops.manifests.v1
kind: Environment
metadata:
  name: production
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  description: |
    Production environment for TradeStream EU. High availability, 
    multi-zone, strict egress controls.
  permissions:
    administrators:
      groups:
        - sre-leads
    viewers:
      groups:
        - auditors
        - l1-support
  network:
    logs:
      interval: INTERVAL_1_MIN # High frequency logging for audit
      sampling: 1.0

4. The Project & Security Boundary

This manifest defines the Google Cloud Project. Crucially, the allowedEgress field implements a "Zero Trust Egress" policy. The application can only communicate with the London Stock Exchange and our email provider.

# FILE: ./environment/production/project/tradestream-eu/tradestream-eu.yaml
apiVersion: lowops.manifests.v1
kind: Project
metadata:
  name: tradestream-eu
  environment: production
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  description: |
    TradeStream Europe. Real-time market analysis and fraud detection.
    Region: europe-west3 (Frankfurt) for data residency.
  region: europe-west3
  allowedEgress:
    - api.london-stock-exchange.com
    - api.sendgrid.com
  hibernation:
    hibernate: false # Financial systems run 24/7
  maintenance:
    start: 2025-02-08T02:00:00Z
    end: 2025-02-08T04:00:00Z
    exclusions: []

5. Source Code & Build Pipeline

5.1 GitHub Repositories

We define the repositories. The AI Assistant is enabled for the ingestor repo to help review complex C++/Go code.

Ingestor Repository:

# FILE: ./github-repository/ts-ingestor-repo.yaml
apiVersion: lowops.manifests.v1
kind: GithubRepository
metadata:
  name: ts-ingestor-repo
  github-connection: default
  organization: stellar-tech
spec:
  description: High-performance market data ingestor (Go/C++).
  strategy: GITHUB_FLOW
  aiAssistant:
    pullRequest:
      enabled: true # AI helps review complex concurrency code
  configuration:
    wiki: true
    projects: true
    issues: true
  codeOwners:
    '*':
      - trade-engineers
  permissions:
    administrators:
      groups:
        - sre-leads
    contributors:
      groups:
        - trade-engineers

Trader UI Repository:

# FILE: ./github-repository/ts-trader-ui-repo.yaml
apiVersion: lowops.manifests.v1
kind: GithubRepository
metadata:
  name: ts-trader-ui-repo
  github-connection: default
  organization: stellar-tech
spec:
  description: React-based Trader Dashboard.
  strategy: GITHUB_FLOW
  pullRequest:
    template: default
  permissions:
    administrators:
      groups:
        - front-end-leads
    contributors:
      groups:
        - front-end-developers

5.2 Build Definitions

These manifests define the CI pipeline. The ingestor uses a Go pipeline with strict security scanning, while the UI uses a standard React/Node pipeline.

Ingestor Build:

# FILE: ./build-definition/ts-ingestor-build.yaml
apiVersion: lowops.manifests.v1
kind: BuildDefinition
metadata:
  name: ts-ingestor-build
  github-repository: ts-ingestor-repo
  github-connection: default
  organization: stellar-tech
spec:
  type: GO # Optimized for high throughput
  stages:
    - analysis
    - test
    - security-scan # Critical for financial software
    - build
    - containerize
  go:
    version: "1.21"
  containerize:
    - name: ts-ingestor
      registries:
        - docker-private # Internal secure registry

Trader UI Build:

# FILE: ./build-definition/ts-trader-ui-build.yaml
apiVersion: lowops.manifests.v1
kind: BuildDefinition
metadata:
  name: ts-trader-ui-build
  github-repository: ts-trader-ui-repo
  github-connection: default
  organization: stellar-tech
spec:
  type: REACT
  stages:
    - analysis
    - test
    - build
    - publish
    - containerize
  node:
    version: "20"
  publish:
    registry: npm-private
  containerize:
    - name: ts-trader-ui
      registries:
        - docker-private

6. Data Layer (Stateful Resources)

The platform uses AlloyDB for the transactional ledger (ACID compliant) and a Redpanda cluster on VMs for the high-volume stream of market ticks.

AlloyDB Database

# FILE: ./environment/production/project/tradestream-eu/database/trade-ledger.yaml
apiVersion: lowops.manifests.v1
kind: Database
metadata:
  name: trade-ledger
  project: tradestream-eu
  environment: production
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  description: Primary AlloyDB cluster for trade records and user auth.
  cpuCount: 8
  clusterSize: 2 # High Availability
  configuration:
    maxConnections: 5000
    googleDbAdvisor: true # Used for in-database anomaly scoring
  backupConfig:
    quantityBasedRetention: 30 # 30 Days retention

Redpanda Cluster (Virtual Machine)

# FILE: ./environment/production/project/tradestream-eu/virtual-machine/redpanda-cluster.yaml
apiVersion: lowops.manifests.v1
kind: VirtualMachine
metadata:
  name: redpanda-cluster
  project: tradestream-eu
  environment: production
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  description: |
    Stateful Redpanda (Kafka) cluster for market data ingestion (3 nodes).
  meshStrategy: SIDECAR
  stateful:
    enabled: true
    clusterSize: 3
    allowTcp:
      - 9092 # Kafka API
      - 9644 # Admin API
    health:
      port: 9644
      protocol: http
      path: /v1/status/ready
      checkIntervalSec: 30
      unhealthyThreshold: 3
      healthyThreshold: 1
      timeoutSec: 5
  ports:
    9644: http
  configuration:
    source: redpanda-v23
    volumeMounts:
      /var/lib/redpanda/data:
        diskConfig:
          sizeGb: 500
          type: pd-ssd
        mountOptions:
          - rw
          - noatime
          - xfs
  machineType: c2-standard-4 # Compute optimized for streaming

7. Storage & Messaging

We define storage for regulatory reports and a Pub/Sub topic to decouple the fraud detection and alerting systems.

Cloud Storage Bucket

# FILE: ./environment/production/project/tradestream-eu/bucket/regulatory-reports.yaml
apiVersion: lowops.manifests.v1
kind: Bucket
metadata:
  organization: stellar-tech
  organizational-unit: financial-services
  environment: production
  project: tradestream-eu
  name: regulatory-reports
spec:
  storageClass: NEARLINE
  public: false
  description: Encrypted bucket for MiFID II compliance reports.
  processors:
    obscenityScanner: false

Pub/Sub Topic

# FILE: ./environment/production/project/tradestream-eu/pub-sub/fraud-alerts.yaml
apiVersion: lowops.manifests.v1
kind: PubSub
metadata:
  organization: stellar-tech
  organizational-unit: financial-services
  environment: production
  project: tradestream-eu
  name: fraud-alerts
spec:
  messageRetentionDurationSeconds: 604800
  notifications:
    - deploymentConfig: ts-notification-svc
      path: webhook/fraud-alert

8. Secrets Management

The ingestor's API key for the London Stock Exchange is stored securely in Secret Manager.

# FILE: ./environment/production/project/tradestream-eu/secret/lse-api-key.yaml
apiVersion: lowops.manifests.v1
kind: Secret
metadata:
  name: lse-api-key
  project: tradestream-eu
  environment: production
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  description: API Key for London Stock Exchange data feed.

9. Microservices (Deployment Configs)

We define the runtime configuration for the services: the ts-ingestor and the ts-trader-ui frontend dashboard.

Market Data Ingestor

# FILE: ./environment/production/project/tradestream-eu/deployment-config/ts-ingestor.yaml
apiVersion: lowops.manifests.v1
kind: DeploymentConfig
metadata:
  name: ts-ingestor
  organization: stellar-tech
  organizational-unit: financial-services
  environment: production
  project: tradestream-eu
spec:
  description: |
    Connects to Stock Exchange API, normalizes data, pushes to Redpanda.
  version: "1.0.0"
  container:
    env:
      - name: KAFKA_BROKERS
        value: redpanda-cluster.tradestream-eu.production.financial-services.stellar-tech.internal:9092
      - name: LOG_LEVEL
        value: warn
  health:
    port: 8080
    protocol: http
    path: /health
  scaling:
    min: 2
    max: 10
  ports:
    8080: http

Trader Dashboard UI

# FILE: ./environment/production/project/tradestream-eu/deployment-config/ts-trader-ui.yaml
apiVersion: lowops.manifests.v1
kind: DeploymentConfig
metadata:
  name: ts-trader-ui
  organization: stellar-tech
  organizational-unit: financial-services
  environment: production
  project: tradestream-eu
spec:
  description: React Frontend for Traders.
  version: "1.0.0"
  health:
    port: 80
    protocol: http
    path: /
  scaling:
    min: 2
  volumes:
    /usr/share/nginx/html/config:
      files:
        config.json:
          content: |
            {
              "apiUrl": "https://api.tradestream.stellar-tech.com",
              "refreshRate": 500,
              "currency": "EUR"
            }
  ports:
    80: http

10. Observability & Monitoring

A Monitoring dashboard and Alerting rules are defined to ensure high availability. The SRE and compliance teams are notified via Slack and Email on critical events.

Monitoring Dashboard

# FILE: ./environment/production/project/tradestream-eu/monitoring/trade-ops.yaml
apiVersion: lowops.manifests.v1
kind: Monitoring
metadata:
  name: trade-ops
  project: tradestream-eu
  environment: production
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  enabled: true
  description: "Main operations dashboard for TradeStream EU"

Alerting Configuration

# FILE: ./environment/production/project/tradestream-eu/monitoring/alerting/critical-alerts.yaml
apiVersion: lowops.manifests.v1
kind: Alerting
metadata:
  name: critical-alerts
  parent: trade-ops
  project: tradestream-eu
  environment: production
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  notifications:
    email:
      recipients:
        - sre-oncall@stellar-tech.com
        - compliance@fincorp.com
    slack:
      channel: "#fin-ops-critical"
      apiUrl: "https://hooks.slack.com/services/..."

11. Load Balancing & Authentication (IAP)

The Trader UI is securely exposed to the internet via a Global Load Balancer, enforcing Identity-Aware Proxy (IAP) for zero-trust access.

PublicIngress: Creates the Global Load Balancer
IdentityProvider: Configures the login screen (Microsoft SSO)
HttpRoute: Routes traffic to the UI service

The iapPermissions block in the Ingress manifest ensures that only authorized users can access the application.

Public Ingress (Load Balancer)

# FILE: ./environment/production/project/tradestream-eu/public-ingress/trade-ingress.yaml
apiVersion: lowops.manifests.v1
kind: PublicIngress
metadata:
  name: trade-ingress
  project: tradestream-eu
  environment: production
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  description: |
    Global Load Balancer for TradeStream.
    Enforces Identity-Aware Proxy (IAP) for zero-trust access.
  domain: tradestream.stellar-tech.com
  iapPermissions:
    groups:
      - authorized-traders
      - compliance-officers
      - sre-leads

Identity Provider (SSO)

# FILE: ./environment/production/project/tradestream-eu/public-ingress/trader-login.yaml
apiVersion: lowops.manifests.v1
kind: IdentityProvider
metadata:
  name: trader-login
  public-ingress: trade-ingress
  project: tradestream-eu
  environment: production
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  description: "Corporate SSO for FinCorp Traders"
  displayName: "FinCorp Secure Login"
  logoUrl: "https://fincorp.com/logo.png"
  mode: REDIRECT
  providers:
    microsoft: true # Corporate Active Directory login
    google: false
    password: false

HTTP Route

# FILE: ./environment/production/project/tradestream-eu/public-ingress/http-route/ui-route.yaml
apiVersion: lowops.manifests.v1
kind: HttpRoute
metadata:
  organization: stellar-tech
  organizational-unit: financial-services
  project: tradestream-eu
  environment: production
  public-ingress: trade-ingress
  name: ui-route
spec:
  description: Route for Trader UI
  rules:
    - matches:
        - prefixMatch: /
      action:
        destinations:
          - deploymentConfig: ts-trader-ui
            port: 80

12. Release Management

Finally, we tie everything together with a Release Track. This defines how code moves from the repository to the production environment.

Release Track

# FILE: ./release-track/fin-track.yaml
apiVersion: lowops.manifests.v1
kind: ReleaseTrack
metadata:
  name: fin-track
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  description: Strict release track for financial applications.
  releaseStages:
    - environments:
        - production
      stakeholders:
        groups:
          - fin-compliance-leads
          - sre-leads

Application Set

# FILE: ./release-track/application-set/core-trading.yaml
apiVersion: lowops.manifests.v1
kind: ApplicationSet
metadata:
  name: core-trading
  release-track: fin-track
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  description: Core high-frequency trading components.

Application: Ingestor

# FILE: ./release-track/application-set/application/ts-ingestor.yaml
apiVersion: lowops.manifests.v1
kind: Application
metadata:
  name: ts-ingestor
  application-set: core-trading
  release-track: fin-track
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  description: TradeStream Ingestor Application
  project: tradestream-eu
  meshStrategy: SIDECAR
  target: CLOUD_RUN
  defaultVersion: latest
  source: ts-ingestor-build # Links to the BuildDefinition
  container: ts-ingestor
  accessControl:
    secrets:
      lse-api-key:
        envVar: LSE_API_KEY

Application: Trader UI

# FILE: ./release-track/application-set/application/ts-trader-ui.yaml
apiVersion: lowops.manifests.v1
kind: Application
metadata:
  name: ts-trader-ui
  application-set: core-trading
  release-track: fin-track
  organizational-unit: financial-services
  organization: stellar-tech
spec:
  description: Trader Dashboard Frontend
  project: tradestream-eu
  target: CLOUD_RUN
  defaultVersion: latest
  source: ts-trader-ui-build # Links to the BuildDefinition
  container: ts-trader-ui

Key Infrastream Features Demonstrated

This use case showcases several critical Infrastream capabilities for financial services:

1. Regulatory Compliance by Design

GDPR/MiFID II compliance through data residency controls (europe-west3)
Complete audit trail via Git history
High-frequency logging (1-minute intervals) for regulatory requirements
Automated backup retention (30 days)

2. Zero Trust Security

Egress controls limiting external communication
Identity-Aware Proxy (IAP) for authenticated access
Secret Manager integration for API keys
Service mesh with sidecar proxy pattern

3. High Availability & Performance

Multi-zone AlloyDB cluster (clusterSize: 2)
Autoscaling Cloud Run services (min: 2, max: 10)
Dedicated Redpanda cluster for high-throughput streaming
Health checks and monitoring at every layer

4. GitOps-Driven Deployment

Infrastructure as Code for all components
Automated CI/CD via BuildDefinitions
Controlled release process via Release Tracks
Version-controlled configuration management

5. Cost & Access Governance

Organizational Unit segregation for financial services
Group-based access control
Resource tagging and cost allocation
Maintenance window scheduling

Business Value

Metric	Impact
Time to Production	Deploy complete trading platform in hours instead of months
Compliance	GDPR/MiFID II compliance built-in, not bolted on
Security Posture	Zero Trust architecture with provable audit trails
Operational Overhead	Platform manages 90% of infrastructure complexity
Disaster Recovery	30-day retention with automated backups
Cost Transparency	Complete visibility into infrastructure costs by organizational unit

End-to-End Architecture & Deployment Guide​

1. Executive Summary​

2. Architecture Overview​

3. Organization & Environment Setup​

OrganizationalUnit Manifest​

Environment Manifest​

4. The Project & Security Boundary​

5. Source Code & Build Pipeline​

5.1 GitHub Repositories​

5.2 Build Definitions​

6. Data Layer (Stateful Resources)​

AlloyDB Database​

Redpanda Cluster (Virtual Machine)​

7. Storage & Messaging​

Cloud Storage Bucket​

Pub/Sub Topic​

8. Secrets Management​

9. Microservices (Deployment Configs)​

Market Data Ingestor​

Trader Dashboard UI​

10. Observability & Monitoring​

Monitoring Dashboard​

Alerting Configuration​

11. Load Balancing & Authentication (IAP)​

Public Ingress (Load Balancer)​

Identity Provider (SSO)​

HTTP Route​

12. Release Management​

Release Track​

Application Set​

Application: Ingestor​

Application: Trader UI​

Key Infrastream Features Demonstrated​

1. Regulatory Compliance by Design​

2. Zero Trust Security​

3. High Availability & Performance​

4. GitOps-Driven Deployment​

5. Cost & Access Governance​

Business Value​