Skip to content
Change and reuse

Change detection and reuse

Terraform answers “what changed?” the same way every time: it refreshes every resource, builds a plan for the whole configuration, and shows you the difference. That’s simple and it’s honest, and it gets slower in proportion to how much you manage, not in proportion to how much you changed. vect splits the question in two. Evaluating a configuration is cheap, so it does that for everything on every run. Planning a node is expensive (an OpenTofu process, provider calls, an API round trip), so it plans only the nodes whose inputs actually differ from what it last applied.

This guide explains how vect decides a node changed, what it reruns downstream when one does, how task executors are reused instead of rerun, and how a long-lived server can start runs on its own and keep overlapping runs from stepping on each other. The examples live under docs/site/examples/topics/change/ in the vect repository. Run IDs and pet names in the output will differ from yours.

What “unchanged” means

At the start of every run, vect evaluates every node in scope with the run’s pinned facts and the committed outputs of its upstreams. For each node it computes an input fingerprint: a digest of the node’s evaluated, non-ephemeral inputs. It then compares four things with the node’s current version in Inventory: the executor type, its revision (the version of the executor implementation), the assembly (the manifest of a terraform root’s entry point, mounts, exact provider versions, and engine, described in the storage guide), and that input fingerprint. For a task executor (more on those below), the trigger fingerprint joins the list. If they all match, the node is unchanged. vect skips it without calling the executor’s Plan at all (FLEET-AFF-2 and FLEET-AFF-3). For a terraform root the comparison uses a code digest of the assembly’s files, which ignores comments and spacing outside strings and heredocs. Editing only a comment in a module shared by many roots therefore skips every one of them, with the reason inputs unchanged; code differs only in comments or spacing, and writes no new version; changing a string, a heredoc, a description, or the lock file still plans (BNDL-ASM-6).

Here’s a four-node project, docs/site/examples/topics/change/reuse. A random pet name feeds a label node, which feeds a service node. A fourth node, docs, depends on nothing.

main.hcl
global "pet_length" {
  type  = number
  value = 2
}

global "note" {
  type  = string
  value = "first draft"
}

executor "terraform" "seed" {
  source = "./roots/seed"
  inputs = {
    length = pet_length
  }
}

executor "terraform" "label" {
  source = "./roots/label"
  inputs = {
    seed = terraform.seed.pet
    note = note
  }
}

executor "terraform" "service" {
  source = "./roots/service"
  inputs = {
    label = terraform.label.label
  }
}

executor "terraform" "docs" {
  source = "./roots/docs"
}

The label root records note in a resource but never puts it in an output. That detail matters in a moment. The first apply runs everything, as you’d expect. (The tool also streams OpenTofu’s own output for each node. We’ve trimmed it here to the summary.)

$ vect apply
…
apply of workspace "dev": succeeded (run 01a1e26a-ec7e-7b23-bfdc-ad2223390a75)
  terraform.docs     applied (run)  will run
  terraform.label    applied (run)  will run
  terraform.seed     applied (run)  will run
  terraform.service  applied (run)  will run
deployment pointer 1 appended

Now apply again, changing nothing:

$ vect apply
…
apply of workspace "dev": succeeded (run 01a1e26b-27f6-7c1d-86ed-ad564b9db33d)
  terraform.docs     unchanged  inputs unchanged
  terraform.label    unchanged  inputs unchanged
  terraform.seed     unchanged  inputs unchanged
  terraform.service  unchanged  inputs unchanged

No tofu init, no provider calls, no plan. Every node’s fingerprint matched its recorded version, so the run was a few state reads and a comparison. Compare that with terraform apply on an unchanged configuration, which refreshes every resource to tell you “No changes.”

Coming from Terraform: the price of that speed is that “unchanged” is a statement about inputs, not about the world. If someone edits a resource by hand, a node whose inputs haven’t moved is still skipped; Terraform’s refresh would have noticed. vect gives you two ways to look. A node you name with --target always re-plans, and a loop’s forced full run (below) re-plans everything on a timer. The difference shows in the reason column. This plan targets one node, and the reason is “no changes”, the answer of a real Plan call, not “inputs unchanged”:

$ vect plan --target terraform.service
…
plan of workspace "dev": succeeded (run 01a1e26b-3fbd-7b81-89e6-528ddad2a25f)
scope: --target terraform.service (upstream: inventory)
  terraform.service  no-op  no changes

What reruns downstream

A node that changed makes its consumers potentially affected, because their inputs may now differ. vect doesn’t assume they do. As each upstream commits, each consumer is evaluated again against the new outputs, and if its fingerprint still matches it’s skipped. That’s early cutoff: the change stops where it stops having an effect (FLEET-AFF-4 and FLEET-AFF-5).

Change note, which label records but never outputs:

$ vect apply --facts newnote.facts.hcl
…
apply of workspace "dev": succeeded (run 01a1e26b-561a-7697-8cb7-3209d8df40fb)
  terraform.docs     unchanged         inputs unchanged
  terraform.label    applied (update)  will update
  terraform.seed     unchanged         inputs unchanged
  terraform.service  unchanged         inputs unchanged
deployment pointer 2 appended

label was directly affected, so it re-planned and applied. service was only potentially affected, and label’s outputs came out identical, so service was skipped. A tree of consumers three levels deep would have been skipped just the same. seed and docs aren’t downstream of the change at all.

One more case: a terraform root whose only change is an output (no resource changes) is applied too, as an update. vect commits a new version for it, and the next run finds the inputs unchanged and skips it. (The label root in this example also holds a resource that depends on seed, so an upstream change reaches it through a resource, but it doesn’t have to.)

A change that does reach an output behaves the other way. Making the pet name longer changes seed’s output, which changes label’s input, which changes service’s. Before applying, you can ask vect to predict this with a plan, and the nodes whose inputs depend on something that doesn’t exist yet are reported as deferred:

$ vect plan --facts longer.facts.hcl
…
plan of workspace "dev": succeeded (run 01a1e26b-6d57-7c92-91f3-e1ce5c339a3d)
  terraform.docs     no-op   inputs unchanged
  terraform.label    defer   unknown inputs: inputs.seed
  terraform.seed     update  will update, destroying 1 resource (see approval requests)
  terraform.service  defer   unknown inputs: inputs.label
approval requests:
  apr_2cab9ec5ca9f4e8a05ebac13  destroy  terraform.seed::random_pet.seed  destroy terraform.seed::random_pet.seed

A plain vect plan can’t know the new pet name until seed applies, so it can’t say whether label and service change. apply finds out by running seed first and re-evaluating. This is the same “plan each node just before applying it” behavior described in How runs work.

Asking before you apply: vect affected

For a published bundle, vect affected --bundle <digest> answers “what would a run of this change touch?” It evaluates the bundle against the workspace’s Inventory and facts, exactly as an admitted run would, and reports the directly affected nodes with a reason, then everything potentially affected. It takes no leases and writes nothing, so it’s safe to run from CI on every pull request.

$ vect bundle
source digest: sha256:a608f78db8c294f97f265f67c0716f331f188759161d11b6ac9d3f7a3fc0f890
bundle digest: sha256:de2c5f77261de385b07735592c7f79eb6addae2357ae2f4d3ce80c43c5b5a9dd
publication: pub_000006
$ vect affected --bundle sha256:de2c5f77… --facts longer.facts.hcl --facts newnote.facts.hcl
workspace "dev": affected
  whole workspace
    directly affected:
      terraform.seed  inputs changed
    potentially affected:
      terraform.label
      terraform.service

(We shortened the digest in the command.) The prediction held. Applying with the same facts updated seed, label, and service, and left docs alone. --format json gives the same report for machines, including orphans, locked nodes, and catch-up waits that this tiny project doesn’t have (FLEET-IMPACT-1 to FLEET-IMPACT-6). Note that --spec-patch isn’t supported yet.

Tasks are reused, not re-planned

Terraform roots have state, so “did it change?” has a natural answer. A task has none. It runs a thing (an ECS task, a Temporal workflow, a Temporal activity), and when it finishes there’s no resource to compare against. If vect re-ran every task on every run, a database migration would execute each time anyone touched the repository.

So the task executors (aws_ecs_task, temporal_activity, temporal_workflow) are memoized, durably, in Inventory. A task’s trigger says what counts as “a different task”: by default, its type, its revision, and its evaluated non-ephemeral inputs; you can opt out of the inputs and add your own keepers, much like terraform_data’s triggers_replace.

main.hcl
executor "terraform" "database" {
  source = "./roots/database"
}

# A task executor: it runs once, then is reused until its trigger changes.
executor "temporal_workflow" "migrate" {
  workflow   = "Migrate"
  task_queue = "migrations"
  input      = { database = terraform.database.name }

  trigger = {
    use_inputs = false
    keepers    = { schema_version = "42" }
  }
}

executor "terraform" "service" {
  source = "./roots/service"
  inputs = {
    migrated = temporal_workflow.migrate.result
  }
}

With use_inputs = false, the migration doesn’t re-run when database’s output changes. It re-runs when schema_version does. (With the default use_inputs = true, the workflow’s own input is part of the fingerprint too.) When the trigger matches the recorded version, Plan returns a reuse action that projects the recorded result, so service still receives it as an input but the workflow never starts (STOR-MEMO-1 to STOR-MEMO-4).

Two properties make this safer than a cache. It lives in the Inventory record, not in a worker’s memory, so restarting a worker or reloading the stores doesn’t forget it. And it’s per-address: a task isn’t reused when its previous attempt is in-flight, uncertain, or marked recovery-required, so a half-finished migration is never mistaken for a finished one. A forced full run (below) re-plans a completed task but still reuses its matching result.

You can watch this with only a local Temporal dev server. The example folder has a tiny worker, worker/main.go, that serves a Migrate workflow on the migrations queue and prints a line whenever it starts. It’s excluded from the bundle by .vectignore. The repository’s docs site is its own Go module, so give the worker a module of its own, then run it, with TEMPORAL_ADDRESS pointing at your dev server (default localhost:7233):

cd worker
go mod init example.test/worker && go mod tidy
go run .

With the worker running, apply from the example folder in another terminal, with VECT_CONFIG pointing at the example’s server.vect (export VECT_CONFIG=$PWD/server.vect). The first apply runs the workflow. The second finds a matching trigger and doesn’t:

$ vect apply
…
apply of workspace "dev": succeeded (run 01a1e26b-9729-73dc-8d6b-7f3d8d592994)
  temporal_workflow.migrate  applied (run)  will run
  terraform.database         applied (run)  will run
  terraform.service          applied (run)  will run
deployment pointer 1 appended
$ vect apply
…
apply of workspace "dev": succeeded (run 01a1e26b-a6d1-7ad1-8c0c-4347ae7b6cd6)
  temporal_workflow.migrate  unchanged  inputs unchanged
  terraform.database         unchanged  inputs unchanged
  terraform.service          unchanged  inputs unchanged

The worker printed Migrate started, input=map[database:orders-db] for the first apply and nothing for the second. Now rename the database in roots/database/main.tf (orders-db to orders-db-2), which changes database’s output. Because the trigger ignores inputs, the migration is reused:

$ vect apply
…
apply of workspace "dev": succeeded (run 01a1e26b-b3f3-77b0-aba1-3be49424eb94)
  temporal_workflow.migrate  applied (no-op)   no changes; recording new assembly
  terraform.database         applied (update)  will update
  terraform.service          unchanged         inputs unchanged
deployment pointer 2 appended

The migrate row reads applied (no-op), no changes; recording new assembly. The workflow didn’t run, because its trigger ignores inputs. But migrate’s input fingerprint did change (database went from orders-db to orders-db-2), so vect appends a new version that records the new identity without applying anything, and the next run finds the node unchanged (FLEET-AFF-3, STOR-INV-8). The reason text says “assembly” generically; for a task executor, which has no assembly, vect records the new input digest while keeping the inputs the task last ran with. The worker printed nothing. Finally, bump schema_version from "42" to "43" in main.hcl:

$ vect apply
…
apply of workspace "dev": succeeded (run 01a1e26b-bf5d-74d2-849d-e618051daec5)
  temporal_workflow.migrate  applied (run)     will run
  terraform.database         unchanged         inputs unchanged
  terraform.service          applied (update)  will update
deployment pointer 3 appended

The worker logged Migrate started, input=map[database:orders-db-2], and service updated because the new result flowed into its input. The aws_ecs_task flavor works the same way but needs AWS, so we don’t run it here.

A default you can swap: trigger is { use_inputs = true, keepers = {} } when you don’t write it, which reruns a task whenever any of its inputs change. Set use_inputs = false when the inputs are context rather than cause, and put what should rerun it in keepers. Ephemeral values never go into a fingerprint (EXEC-EPH-6).

Loops: runs that start themselves

Everything so far started with you typing a command. On a server, vect can also decide to run. A loop reconciles one (workspace, scope) continuously, where a scope is a set of stack addresses such as stack.ctl. A loop is one long-running Temporal workflow hosted by vect serve. You declare loops in the server configuration, next to the workspace they belong to:

serve.vect
# Two loops, disjoint scopes, together covering the whole graph.
loop "dev" "ctl" {
  scopes         = ["stack.ctl"]
  lineage        = "change@main"
  full_run_every = "168h"
}

loop "dev" "dat" {
  scopes         = ["stack.dat"]
  lineage        = "change@main"
  catch_up_wait  = "30m"
}

The project, docs/site/examples/topics/change/loops, has a slow ctl stack and a busy dat stack that reads ctl’s subnet output. Start the server and each loop shows up as a running workflow, one per scope, in the same Temporal namespace as your runs:

$ vect serve
vect: serving on 127.0.0.1:8443
$ temporal workflow list --query 'ExecutionStatus="Running"'
  Status          WorkflowId                 Type            StartTime
  Running  vect/dev/loop/dat/change  vect.LoopWorkflow.v1  10 seconds ago
  Running  vect/dev/loop/ctl/change  vect.LoopWorkflow.v1  10 seconds ago

Loops have rules. A workspace’s loops have disjoint scopes, and a node inside two of them is a configuration error. Runs from different loops can execute at the same time. Only one runner holds a loop’s ownership lease, so a second vect serve configured with the same scope stays idle until the first one’s lease expires (FLEET-LOOP-1 to FLEET-LOOP-5).

What starts a run is a trigger. The ones you’ll meet first:

TriggerWhat raises it
Source changeA new publication on the loop’s lineage, for example CI running vect bundle --lineage change@main --source-position <n>
Fact changeA plan-time data value, re-read every facts_poll_every (default 1m), differs from the value the loop last pinned
Explicit targetA human vect apply or destroy, which is submitted to the loop that owns the scope
Stale nodesNodes marked stale (see below)
Forced fullThe full_run_every timer (default 24h, plus a random offset of up to 10%), or an operator

The forced full run is the answer to the drift caveat from earlier. It plans every node in scope and bypasses the “inputs unchanged” skip, so out-of-band edits get noticed within a day by default (FLEET-TRIG-1 to FLEET-TRIG-4, FLEET-TRIG-10). Triggers are written to the workspace’s store before they’re acknowledged, so a restart or a lost wake-up signal loses none of them.

Each trigger is reduced to a seed set: the directly affected nodes it names. The run’s affected set is those seeds plus everything downstream, cut off at the loop’s scope. That boundary matters. A run never plans a node outside its scope, so a dat change can’t touch ctl, and a ctl change reaches dat through its owner instead of directly.

Seeds are computed when a trigger is admitted, not when it arrives, against the newest facts and source. That has two consequences. Triggers that arrive close together and overlap are merged into one run carrying all of their seeds, and a trigger whose seeds an active run already covers is dropped.

An explicit-target trigger never merges with anything, so a targeted apply never applies more than its targets (FLEET-SEED-1 to FLEET-SEED-10).

vect serve starts without contacting its identity provider, so the example’s serve.vect uses a placeholder issuer. Pushing a change through the loops is different: a served runner authenticates its API, so uploading a bundle needs a real issuer, as in the serving section of How runs work. That’s why we stop at showing the loops running.

Coming from Terraform: Terraform has no concept of a continuous reconciler; the equivalent is a CI job on a cron schedule that runs terraform apply for everything. A loop gives you the same safety net (the daily forced full run) plus the thing a cron job can’t: reacting to the specific change that arrived, and only planning the part of the graph it reaches.

Overlapping runs and “never go backwards”

When changes arrive faster than runs finish, two runs will eventually want the same node. vect’s default model is eventually consistent and never regresses.

A run holds a lease on a node from its Plan through its commit, five minutes at a time, renewed while the node’s activity runs. A run that can’t get a lease doesn’t fail; it waits, and the wait is reported with the holder’s name. If a holder dies, the next run that leases the node recovers it. A wait longer than lease_wait (default 30m) fails the run naming the holder (FLEET-CONC-1 to FLEET-CONC-3, FLEET-CONC-11).

Before applying a node, a run also compares what it’s about to apply with what’s recorded: the bundle’s source position, the loop’s fact generation, and the version of each upstream. If the recorded version is newer on some comparison and older on none, the run skips the node. A slow run on an old commit can’t overwrite what a fast run on a newer commit already applied. If the two are incomparable (neither dominates), the run skips the node and marks it stale (FLEET-CONC-4 to FLEET-CONC-8).

Staleness

A stale node is one applied from inputs that have since changed. Two things mark it: a node commits an output that differs from its previous version, which marks every consumer whose recorded version saw the old one, and a node skipped by the incomparable case above. A stale mark is a hint recorded on the node’s Inventory record, not a run state, so it survives restarts and anyone may set it without the node’s lease.

A loop looks for stale nodes in its scope on every facts_poll_every tick and raises a stale-nodes trigger, which plans them against the newest facts and outputs. If the follow-up fails, it’s retried with exponential backoff, from one minute up to an hour. The mark is cleared only when a run commits from the newest inputs, or re-plans from them and finds nothing to change. So staleness turns the rare unordered overlap into a follow-up run instead of a lost update (FLEET-STALE-1 to FLEET-STALE-4). It’s also how a change reaches a consumer in another loop of the same workspace: that consumer is marked stale and its own loop picks it up.

Reading an upstream that’s mid-apply

By default, a node reads each upstream’s last converged version, even while another run is applying that upstream. That’s the right choice most of the time: you don’t want a deploy to block on an unrelated database migration. When it isn’t, you can say so with a concurrency block on the producer or the consumer:

main.hcl
executor "terraform" "db" {
  source = "./roots/db"

  # Readers of this node wait for an in-flight apply instead of
  # reading its last converged version.
  concurrency {
    outputs = "wait"
  }
}

outputs = "wait" on the producer, or inputs = "wait" on the consumer, makes the consumer wait for the in-flight apply to commit and then read the new version. When the two disagree on one edge, the stricter setting wins. Waiting like this holds no executor slot, so a long wait doesn’t starve other work (FLEET-READ-1 to FLEET-READ-3, LANG-CONC-1 to LANG-CONC-3).

Exclusive scopes

Some changes shouldn’t overlap with anything: a failover, a schema rewrite, a rotation of a shared credential. For those, mark a stack (or an executor) exclusive:

main.hcl
# Only one run at a time may change anything inside this stack.
stack "ops" {
  source = "./ops"
  db     = terraform.db.name

  concurrency {
    exclusive = true
  }
}

A run that may change any node inside an exclusive block takes a scope lease on it first, in canonical address order before any node lease, and releases it when all its nodes there have committed, failed, or been skipped. While it’s held, no other run changes a node inside that scope (FLEET-EXCL-1, FLEET-EXCL-2). Loops use these same needs to decide admission: a queued run starts as soon as the leases it predicts don’t overlap those of the active runs, and never before an older queued run it conflicts with.

A default you can swap: on a stack block, concurrency accepts only exclusive. On an executor block it also accepts inputs and outputs, each "last_converged" (default) or "wait". Values must be known at plan time.

Catch-up waits across loops

The last piece is a change that spans two loops. Say one commit adds an output to ctl and a dat node that consumes it. The dat run can’t evaluate that node until ctl has applied the change, so the node catches up: it waits, holding no slot, until the upstream has a version that matches what the bundle assigns it, then continues. If the upstream loop’s run fails, the waiting node is reported blocked by that failure. If the wait exceeds catch_up_wait (default 1h, configurable per loop as in the dat loop above) the node fails. A plan-only run doesn’t wait; it reports the input as unknown and names the upstream (FLEET-SPAN-1 to FLEET-SPAN-6). A change that reaches the upstream only through facts never causes a catch-up wait. It propagates through staleness instead.

These waits coordinate through the workspace’s Inventory only, so the two loops can run on different runners.

What you have now

The model, in short: evaluate everything, plan only what’s directly affected, and stop propagating the moment a consumer’s inputs come out the same. Tasks are reused from Inventory by trigger fingerprint instead of rerun. A server’s loops turn publications, fact changes, and timers into runs scoped to the part of the graph they reach, with a daily full run as a drift check. Leases, version comparisons, and staleness keep overlapping runs from regressing anything.

Some of what’s in the spec isn’t here yet: change records and per-node attribution, locks that freeze a subtree, and cross-workspace triggers through vect_stack aren’t implemented. We left them out rather than describe them.