Inventory, bundles, and storage
Terraform keeps everything it knows in one place: a state file. vect keeps what it knows in several, because it has to remember more. It has to remember the code it applied (so it can tear it down after the code is gone), the values that flowed between executors, the history of every apply, and the state of each Terraform root underneath. This guide explains what each of those is, where it lives, and how long it stays.
The short version is a table. Each row is something vect stores, and the Terraform concept it’s closest to.
| What | Called | Closest Terraform idea |
|---|---|---|
| The code that was applied | a bundle, held in a content-addressed store (CAS) | none: Terraform reads your working directory every time |
| What vect applied, per executor | the Inventory: versioned records and a deployment pointer history | the list of resources in a state file |
| Values flowing between executors | the value store, encrypted per workspace | terraform output and remote state data sources |
The engine’s own state for a terraform executor | a plain OpenTofu/Terraform state file in the state store | the state file, unchanged |
We’ll use a small project to look at each one. It is one terraform executor, in the example folder docs/site/examples/topics/storage/local in the vect repository, run with the same kind of server configuration as in Part 3. Commands read the server configuration from --config or VECT_CONFIG, so run export VECT_CONFIG=server.vect first, as in Part 3. Without it, even vect bundle stops with Cannot resolve the opentofu version, because the Terraform version default lives in that file. Everything below ran against a local Temporal dev server. Your IDs, digests, and timestamps will differ.
The example folder ships a .vectignore. This page starts without it, so delete that file before you begin; we put it back under Digests and explain it under Keeping files out.
Where things are stored
A workspace has four stores, and each one has a default backend and one alternative:
| Store | Holds | Default | Alternative |
|---|---|---|---|
state | engine state files for terraform executors | afero | s3 |
inventory | records, versions, deployment pointers, leases | afero | dynamodb |
values | encrypted cty values, and the references to them | afero | s3 |
cas | the workspace’s own encrypted blobs (saved plans, plan files, run logs) | afero | s3 |
afero here means plain files, through the same filesystem abstraction vect uses everywhere. With no stores block, every store is a directory under <data_dir>/<workspace>, and data_dir defaults to .vect/data. That is what the tutorial used. After one vect apply, the example project looks like this (trimmed):
.vect/
data/
bundles/ the server-wide bundle store
blobs/<sha256> bundle units, assemblies, manifests
meta/<sha256> who may read each blob
publications.json
cache/ unpacked units, read-only, used by workers
downloads/ verified engine and provider downloads, reused by later bundles
dev/ the "dev" workspace's stores
blobs/ meta/ its CAS (encrypted objects)
values/ the value store
ZGV2/ the Inventory ("ZGV2" is "dev", base64 encoded)
dev/terraform.pet.tfstate the state store
keys/
dev.key/ the local key provider's key fileA default you can swap: to put a workspace’s stores in AWS, add a stores block with one block per kind. Any kind you leave out stays on local files.
workspace "prod" {
…
stores {
state {
backend = "s3"
region = "us-east-1"
name = "vect-prod-state"
}
inventory {
backend = "dynamodb"
region = "us-east-1"
name = "vect-prod-inventory"
}
values {
backend = "s3"
region = "us-east-1"
name = "vect-prod-values"
}
cas {
backend = "s3"
region = "us-east-1"
name = "vect-prod-cas"
}
}
}vect needs a name (the bucket or table) for these stores until automatic naming from a store_naming template lands; the template is specified but not implemented yet. The complete file is aws.server.vect in the example folder, and it loads. We haven’t shown any of the AWS backends running, since that needs an AWS account. The repository’s tests run the store contracts against in-memory fakes of DynamoDB and S3, but treat the AWS setups as untried by this guide.
Why swap them? Local files are fine for one person on one machine. Once several runners, or CI and your laptop, work against the same workspace, they need a shared Inventory and a shared CAS. Inventory writes are conditional, so two runners can never both commit the same version of a record or hold the same lease; a store has to be able to do compare-and-set for that to hold, which is why the alternatives are DynamoDB and S3 rather than “any database” (STOR-REQ-1).
The bundle store
Bundles are different from the four kinds above. They contain no secrets and don’t belong to any one workspace: one CI build is meant to be planned into every workspace. So bundle units, assemblies, and manifests live in a single server-wide bundle store shared by all workspaces (CONF-SRV-4). Each workspace’s own cas store then holds only that workspace’s encrypted objects.
Without configuration it’s local files under <data_dir>/bundles, as in the tree above. To move it, add a bundles block to server:
server {
name = "prod"
bundles {
backend = "s3"
region = "us-east-1"
name = "vect-bundles-prod"
}
}With s3, name is required. With afero, an optional path replaces the default location.
Bundles: the code, packaged once
A Terraform run reads your working directory, so what runs depends on what’s checked out and what terraform init downloads that day. vect separates those. vect bundle packages the configuration once, and runs use the package.
vect bundlesource digest: sha256:b426548b29ab8f7b545aaf78784693b5ca4a88b6a8741f3bce6983f5bccd11e8
bundle digest: sha256:1f6298ffdb0ca2439087319793a6e2916d25346d2a05670c258c75c7d42dd96c
publication: pub_000001(vect plan and vect apply do this for you when you don’t pass --bundle, which is why the tutorial never ran it.)
A bundle is a small manifest that lists units, and each unit is stored once, named by the SHA-256 of its contents. For the example project the manifest lists five:
- the stack directory (your
.hclfiles), - the local Terraform module directory,
roots/pet, mounted at its project-relative path, - the
hashicorp/randomprovider package for this platform, with its version taken from the lock file, - the engine binary, here OpenTofu 1.10.6, and
- the root’s
.terraform.lock.hcl.
Besides units, each terraform executor gets an assembly: a manifest of the root’s entry point, its mounts, its exact provider versions, and its engine and version. The assembly’s digest is what Inventory records on each version of the executor, which is how vect later knows exactly what ran.
releases.hashicorp.com to run your configuration, and external modules are fetched at bundle time and rewritten to local mounts. The network is needed when you bundle, not when you run. The tradeoff is that bundles are large: the unpacked provider and engine for this tiny project make the bundle store 88 MB.Digests
Because units are addressed by content, bundling is idempotent. Run it again with nothing changed and you get the same digests:
Now put back the example’s .vectignore (the section below explains it):
source digest: sha256:ac77b0827a84a7cb6175edd0cf8064bf3a0e51cf0dbc6f5caf270619a73944e1
bundle digest: sha256:d5be5c930fadacbe1205e55d400d76635b1da26e84ad8fce84646e556e27485d
publication: pub_000003source digest: sha256:ac77b0827a84a7cb6175edd0cf8064bf3a0e51cf0dbc6f5caf270619a73944e1
bundle digest: sha256:d5be5c930fadacbe1205e55d400d76635b1da26e84ad8fce84646e556e27485d
publication: pub_000004(These differ from the first run only because we’d added a .vectignore in between; see below.) The bundle digest is what --bundle takes. The source digest is a hash of just the project files that were read while bundling, so a change to a sibling root you didn’t use doesn’t change it. Units already in the store aren’t uploaded again.
Each upload also creates a publication: a record of who published this bundle, when, and (when CI provides them) the commit and a source position in a branch. Publishing the same bundle twice gives two publications for one bundle digest. Publications live next to the manifest, outside the digested body, which is how the same code built from two commits can still be recognised as the same bundle. The spec says more about how publications are ranked and used to order runs in bundling.md.
Keeping files out: .vectignore
A .vectignore file at the project root, in gitignore syntax, keeps files out of the bundle. vect already skips .git, .terraform, .vect, lock files, and *.facts.hcl. Our example adds:
server.vect
*.server.vect
notes/
*.logIgnored files aren’t packed and aren’t part of the source digest. Before we added server.vect to the file, the stack unit held main.hcl plus all three server configuration files in the folder; afterward it held only main.hcl. Ignoring the server configuration is a good habit, since it names your Temporal endpoint and key paths, and none of that belongs in a published artifact.
The worker cache
Workers unpack a unit once into a local cache, .vect/data/cache in our example, and executors read from it through a read-only filesystem. On disk the files really are read-only (0444, or 0555 when executable), which makes it safe for concurrent runs to share. The directories stay owner-writable (0755), so rm -rf .vect works without a chmod. Each Terraform working directory gets a provider mirror that points at the cached provider units, so providers are never copied or downloaded per run.
The CAS
The content-addressed store is the generic mechanism under all of this: immutable blobs, named by digest, where a write whose bytes don’t match its digest is refused. Two CAS instances are in play. The server-wide bundle store holds the unencrypted bundle blobs. Each workspace’s cas store holds that workspace’s saved plans, plan files, and run logs, which are encrypted with the workspace key before they’re written. Next to every blob is a small metadata file recording which workspaces may read it:
{"workspaces":[],"encrypted":false} a bundle unit: any workspace
{"workspaces":["dev"],"encrypted":true} a plan: only "dev"A download request only serves a blob that belongs to the workspace it names, and encrypted blobs are never served through a presigned URL: the server decrypts and streams them.
Inventory: what vect applied
Inventory is the closest thing vect has to a state file, but it records that and why, not the resource attributes. It holds one record per executor instance, and each record has an append-only list of versions. Each version notes which run committed it, which bundle and assembly the code came from, references to the inputs and outputs, and when. Look at the example after an apply that changed the pet’s length from 2 to 3:
vect inventory terraform.petterraform.pet version 2 converged
VERSION RUN TYPE BUNDLE COMMITTED
1 01a1e26a-b566-711d-8fd3-81423eb4a5e2 terraform sha256:1f6298ffdb0ca2439087319793a6e2916d25346d2a05670c258c75c7d42dd96c 2026-11-16T03:33:29Z
2 01a1e26a-ea0c-77f7-8a4f-2cd176b2d7dd terraform sha256:0f6ecb6a9c18ec630aeb0f6f5da248a623e64a8cbdda561a90cef8eb7d30efb2 2026-11-16T03:33:41ZVersion 2 points at a different bundle than version 1, because the code changed. vect inventory with no argument lists every record and the deployment pointer history. Every apply that commits something appends a pointer that says which bundle was applied, to what scope, and when, which makes it the log of what each environment has run:
vect inventoryADDRESS VERSION STATUS
terraform.pet 2 converged
POINTER BUNDLE SCOPE APPLIED METADATA
1 sha256:1f6298ffdb0ca2439087319793a6e2916d25346d2a05670c258c75c7d42dd96c full 2026-11-16T03:33:29Z
2 sha256:0f6ecb6a9c18ec630aeb0f6f5da248a623e64a8cbdda561a90cef8eb7d30efb2 full 2026-11-16T03:33:41ZRecords are plain JSON files in the afero backend (under ZGV2/ver, ZGV2/head, and so on in our tree), and they hold references to values, not the values themselves. The same record also carries operational markers: an in_flight marker written before a change and cleared after, a failure entry, and a recovery_required mark when a failure happens after a change started. Those are what let a later run recover without you reconciling anything by hand. Leases, which stop two runners from working on the same node, live in the same store.
Tombstones, and destroying from records alone
Destroying a node doesn’t delete its record. It appends a tombstone version, which is why the earlier history is still there. And because each version names its assembly in the CAS, vect can plan a teardown without the configuration. To see that, we moved roots/pet away, replaced main.hcl with a single output, and ran a plan (we leave out the engine’s own lines):
vect planterraform.pet: running
terraform.pet: planned orphan: block no longer exists
plan of workspace "dev": succeeded (run 01a1e26a-eedc-71cc-b65b-72fae70829fc)
terraform.pet destroy orphan: block no longer exists
approval requests:
apr_727585d1581d39bbb1aa9f07 destroy terraform.pet::random_password.token destroy terraform.pet::random_password.token
apr_a772397df5e1b52f31c7c0c5 destroy terraform.pet::random_pet.this destroy terraform.pet::random_pet.thisThere was no random_pet anywhere in the working directory. vect restored the historical sources from the CAS using the digests recorded in the record, and planned a destroy from them. After vect apply, the record shows the tombstone:
terraform.pet version 3 destroyed
VERSION RUN TYPE BUNDLE COMMITTED
…
3 01a1e26a-f187-7a12-9bbd-9c2b3a47a7f8 terraform (destroyed) sha256:d1e69bfdffbbe03e3cf4a73fa92cd07002c7f867092d047bad7a8e966e9d8473 2026-11-16T03:33:42Zremoved blocks and orphan teardown safe. See the storage specification (STOR-INV-6, STOR-INV-7).The value store, and keys
Values that move between executors (inputs, outputs, pinned facts, and stack and root outputs) don’t sit in the Inventory record. They go into the value store, and the record holds an opaque reference such as {"id":"6f6cb6c1..."}. Values round-trip with their types, nulls, unknowns, and sensitive marks intact.
At rest, each value is encrypted with a key that belongs to its workspace, so credentials for one workspace can’t decrypt another’s values. A stored value looks like this:
{"key_version":"1","sealed":"…"}The key comes from the workspace’s key_provider. The tutorial’s local_file provider keeps a key file at the path you give it; aws_kms takes a key_id and asks KMS for a data key bound to the workspace. Each stored object records the key version that encrypted it, so a rotation leaves existing values readable and encrypts new ones with the new version. Re-encrypting old values isn’t something vect does.
local_file is for local use, and it’s honest about it: the key sits in .vect/keys, a directory beside the data it protects. Anyone who can read your project folder can read both. For anything shared, use key_provider "aws_kms". Like the S3 and DynamoDB stores, we haven’t shown KMS running here.Engine state: the plaintext part
Each terraform executor keeps an ordinary OpenTofu or Terraform state file, and that file isn’t encrypted by vect. In the example it’s .vect/data/dev/dev/terraform.pet.tfstate, and it contains exactly what you’d expect from a state file:
{"version":4,"terraform_version":"1.10.6","serial":1,"lineage":"f93fc623-86aa-b8b1-b2f6-a816dd6da375","outputs":{"name":{"value":"humble-silkworm","type":"string"},"token":{"value":"nHkRF2bV\u003cX-Z","type":"string","sensitive":true}},"resources":
…That is the sensitive token, in cleartext, because that’s how Terraform state works. vect encrypts what it stores (values, saved plans, plan files, run logs), but it doesn’t wrap the engine’s state. If you move the state store to S3, protect that bucket the way you’d protect a Terraform state bucket: server-side encryption, tight access, versioning.
The practical difference from Terraform is that you don’t configure a backend block for this. vect owns where the state lives (the state store), keeps one state object per executor address, and tracks it through Inventory. After a destroy, the state file stays behind, empty, alongside a .backup.
Retention and garbage collection
History and content-addressed blobs accumulate. Two things bound them.
Retention decides which Inventory history to keep. Per record it keeps the current version, the last converged version, the last 10 versions, and everything newer than 30 days. A record whose current version is a tombstone older than that window is removed entirely. Deployment pointers follow the same rule, except that vect always keeps the latest pointer with outputs. You can tune it per workspace:
workspace "dev" {
…
retention {
versions = 1
max_age = "1s"
grace = "1s"
}
}Those are absurdly small values, chosen so a short demo has something to collect. grace is how long an unreferenced blob must have sat untouched before collection may delete it; the default is 7 days.
Garbage collection deletes CAS blobs and value references that nothing still points at. vect gc marks everything reachable from every workspace that shares the store (retained versions and pointers, the current bundle, unfinished runs, unexpired plans), then removes what’s left and older than the grace period. Like plan before apply, it reports first and deletes only when you pass --apply:
To have something to collect, we set the project up as follows, in a fresh copy of the example with VECT_CONFIG=gc.server.vect. Apply it once without its .vectignore:
rm .vectignore
vect applyBring the .vectignore back, publish the bundle twice without applying it, then change the pet’s length from 2 to 3 and apply again:
server.vect
*.server.vect
notes/
*.logvect bundle
vect bundle
perl -pi -e 's/length = 2/length = 3/' roots/pet/main.tf
vect apply --auto-approveNow ask for a collection:
vect gcdry run for workspace dev: nothing is deleted without --apply
retention: would delete 1 versions, 0 records, 1 deployment pointers
STORE KIND ID SIZE AGE
cas cas …
…
values value …
…
bundles cas sha256:d5be5c930fadacbe1205e55d400d76635b1da26e84ad8fce84646e556e27485d 1346 …
…
would delete … entries, … bytesThose rows come from a run where we applied twice (length 2, then 3) and then collected, with the tiny retention above. (Trimmed to a few of the dozen or so rows. Your digests, sizes, ages, and totals will differ, and the entry count can differ by a row or two between runs.) Bundle d5be5c... is the one we published twice and never applied; the other bundles row is version 1’s bundle, which retention just pruned. We ran this collection before the orphan teardown shown earlier, while terraform.pet was still converged. With the stock settings, the same command reported “would delete 0 entries”, because nothing was older than the grace period. After vect gc --apply the output ends with a line like deleted N entries, M bytes, version 1 disappears from vect inventory terraform.pet (only version 2 remains), and the next vect apply still ran cleanly. (The two counts vary from run to run.) The outcome depends on where the record is. If you had destroyed the executor first, its current version would be a tombstone older than max_age, and retention would delete the whole record instead of just old versions.
A few safeguards matter if you share stores between workspaces. Each workspace writes a manifest of its own roots into the stores it shares, vect gc marks from the union of them, and it refuses to sweep if any registered workspace’s manifest is missing or stale. Deletes are conditional on the age gc observed, so a writer that refreshes a blob at the same moment wins. vect gc --retire-workspace <name> removes a workspace’s manifest when you’re done with it. It refuses while the workspace’s stores still hold entries:
vect: failed_precondition: gc: the workspace's stores are not empty: "dev" holds 3 CAS entries and 4 values; pass --abandon to retire it anywayThe counts depend on what your stores hold at that moment (another walk-through printed 5 CAS entries and 7 values). Pass --abandon (with --apply) to retire it anyway. --format json gives scripts a stable report. On S3, the bucket must have versioning enabled, because GC deletes exact object versions (STOR-GC-3; internal/cas/s3/versions.go).
What’s not here
This guide leaves out the S3, DynamoDB, and KMS setups running for real (no AWS account), and the store_naming template, which isn’t implemented yet. It also doesn’t cover how leases and in-flight markers are used during a run, which belongs with the run engine, or vect bootstrap, which isn’t implemented. The full requirements behind this page are the storage, bundling, and server configuration specifications.