Architectural Overview
Infrastream runs its own Go engine. The engine reads your manifests and calls the Google Cloud APIs directly. No separate tool and no locked state file stand between them. Many teams can therefore deploy at the same time.
A git push starts a run. The engine parses the manifests into a graph, resolves Promises in dependency order and calls Google Cloud APIs. It writes the ledger in Spanner and streams run events to the Portal.
The diagram follows the data of one run: the manifests, the graph, the resources, the state and the portal. It does not show the phases in the table below. Steps 1 and 2 happen in the Clone and Scan phases. Steps 3 to 6 happen in the Reconcile, Plan and Apply phases. Steps 7 and 8 are the output of the run.
Engine run
A Git push to the manifest repository starts a run. A run has eight phases:
| Phase | What it does |
|---|---|
| Clone | Gets the manifest repository at the commit. |
| Scan | Parses the YAML files into typed manifests. |
| Default | Adds the child resources that each manifest implies. |
| Validate | Runs structural and semantic checks. |
| Compute | Fills in manifest values and generates CI workflows. |
| Reconcile | Merges the desired state with the Spanner graph. |
| Plan | Calculates the changes to the state. |
| Apply | Runs the changes at the same time, in dependency order. |
The engine runs Default and Validate for each manifest during the scan, in dependency order. It runs Reconcile at the start of the Plan or Apply run, after Compute.
After a CD promotion, the engine processes only the manifests that the commit changed.
JIT graph and promises
The engine parses all manifests into one directed acyclic graph in memory. When resource A needs a value from resource B, A holds a typed Promise for that value. The engine sorts the graph by dependency and runs independent resources at the same time. When B finishes, it fulfills the promise and A continues. One pass resolves the whole topology. No string interpolation passes values between resources.
A MetaGroup is a virtual node that stands for a set of resources. The engine adds LOGICAL edges from a MetaGroup to its dependents. Use this to finish an organizational unit and its policies before any of its environments or projects start. No placeholder resource appears in Google Cloud.
State in Spanner
The state is an append-only ledger in Cloud Spanner. See Core Concepts for the model. Three design choices matter at scale:
- Bit-reversed run IDs. They spread writes across Spanner and prevent write hotspots when many runs start at once.
- Temporal reconstruction. Run IDs are not sequential. The engine groups rows by the start time of each run with
ARRAY_AGGto rebuild the state of the graph at a past moment. - Vector embeddings. The ledger stores
StateEmbeddingvectors from Vertex AI. They support semantic search and blast-radius analysis in the database.
Live updates
While the engine runs, it writes RunEvent changes to Go channels. The portal reads these channels and sends them to browsers over a gRPC server stream. The graph in your browser shows node states (PENDING, CREATE, STALLED) without a page reload.
Compute abstraction
An application does not stay tied to the compute service it first used. Change the spec.target field of an Application or ExternalApplication to move it, with no change to the code. Cloud Run and Compute Engine VMs work today. Support for Google Kubernetes Engine (GKE) is in active development and not yet generally available.
For each target, the engine sets up the network, the service mesh, load balancers, the IAM rules of the application identity, and logs and metrics. See Migrate a Service to Cloud Run.