Skip to content
Your own steps

How to run your own code as a step

By the end of this guide you’ll have a smoke test, written in Go, running as a step in your graph between two Terraform roots. It takes an upstream root’s output as input, hands a typed result to the next root, runs again only when you say so, and fails the run cleanly when the check fails. The same recipe works for a database migration or any other operational job.

You need to have finished First steps, a Temporal dev server on localhost:7233, Go, and the vect binary. The project is docs/site/examples/howto/custom-steps/smoke/ in the vect repository. Run IDs and deployment pointer numbers in the output will differ from yours.

1. Pick the executor

vect runs your code through Temporal, so a step is a temporal_activity (one function) or a temporal_workflow (a durable workflow that may call many activities). Start with an activity: it’s one function and one task_queue. The workflow flavor has the same inputs, result, and trigger; the change guide runs one.

Both are task executors. They own no infrastructure, so there’s nothing to destroy, and vect remembers their result instead of re-running them on every apply.

2. Write the step into the graph

main.hcl
executor "terraform" "app" {
  source = "./roots/app"
}

# Your own code, as a step: one Temporal activity on a queue you serve.
executor "temporal_activity" "smoke" {
  task_queue             = "smoke"
  activity               = "SmokeTest"
  input                  = { url = terraform.app.url }
  start_to_close_timeout = "30s"
  timeout                = "2m"

  retry_policy = {
    max_attempts = 1
  }

  # Re-run when the URL changes, or when you bump the keeper.
  trigger = {
    keepers = { suite = "v1" }
  }
}

executor "terraform" "report" {
  source = "./roots/report"
  inputs = {
    smoke = temporal_activity.smoke.result
  }
}

Reading it top to bottom:

  • input is sent to your function as its single JSON argument. Referencing terraform.app.url is what orders the step after app; report consumes result, so it runs after the step. No depends_on needed.
  • start_to_close_timeout bounds one attempt inside Temporal. temporal_activity requires it or schedule_to_close_timeout, and the check happens at plan or apply time, not at vect validate. retry_policy is passed to Temporal as is; vect adds no retries of its own, so for a side-effecting step keep max_attempts low.
  • timeout = "2m" is vect’s own bound on the whole call, for any executor (the default is 1h). See timeout.
  • trigger says what makes the step run again; step 5 covers it.

report’s root declares what it expects from the result. That variable type is how you “declare the schema” of a step’s result:

roots/report/main.tf
variable "smoke" {
  type = object({
    healthy = bool
    status  = number
    url     = string
  })
}

Coming from Terraform: the schema block (see schema) refines data blocks only. Executors don’t take one. A task’s result is typed any, and vect converts it to the type of whatever consumes it, here the root’s variable. Compatible types convert (a number result into a string attribute works), and result attributes the type omits are dropped silently. A result that doesn’t fit fails at plan, and the error names the input path that doesn’t match and why. We changed healthy = bool to list(string) and, once the step had a result (step 4 below), got:

plan of workspace "dev": failed (run 01a1e14c-ab2f-75df-acbe-308aa3f9b01c)
  temporal_activity.smoke  no-op   inputs unchanged
  terraform.app            no-op   inputs unchanged
  terraform.report         failed  terraform.report: plan: invalid configuration: inputs.smoke.healthy: list of string required, but received bool

The message names types and attribute names only, never the result’s values.

Check the project offline:

vect validate
vect graph
Success!
  (root)  3 blocks
terraform.app
temporal_activity.smoke
terraform.report

3. Write the worker

The activity is an ordinary Go function in a worker you run yourself. vect doesn’t build or host it, so keep worker/ out of the bundle with .vectignore (the example’s does) and give it its own module:

worker/main.go
// SmokeInput is the activity's single argument: the JSON that vect sends from
// the step's `input`.
type SmokeInput struct {
	URL string `json:"url"`
}

// SmokeResult is what the step exposes downstream as `result`.
type SmokeResult struct {
	Healthy bool   `json:"healthy"`
	Status  int    `json:"status"`
	URL     string `json:"url"`
}

func SmokeTest(ctx context.Context, in SmokeInput) (SmokeResult, error) {
	fmt.Fprintf(os.Stderr, "worker: SmokeTest started, url=%s\n", in.URL)
	status, err := probe(ctx, in.URL)
	if err != nil {
		// Non-retryable: a refused connection will not fix itself in this run.
		return SmokeResult{}, temporal.NewNonRetryableApplicationError(
			"smoke test could not reach "+in.URL, "Unreachable", err)
	}
	return SmokeResult{Healthy: status == http.StatusOK, Status: status, URL: in.URL}, nil
}

(probe is a five-line http.Get with a timeout; the full file is in the example.) Registration is the part that must match the HCL: the queue name and the activity name.

w := worker.New(c, "smoke", worker.Options{})
w.RegisterActivityWithOptions(SmokeTest, activity.RegisterOptions{Name: "SmokeTest"})

Two rules for the function. Its argument and return value must be JSON-encodable, because that’s what crosses the wire. And make it idempotent, since vect never retries a side-effecting call itself and recovery is “plan again, then apply” (EXEC-IDEM-2). Don’t put secrets in input: Temporal stores it in history, so vect rejects an ephemeral value there.

Start the pieces. In one terminal, something for the test to hit, then the worker:

mkdir ~/www && echo ok > ~/www/index.html
cd ~/www && python3 -m http.server 8080
cd worker
go mod init example.test/worker && go mod tidy
go run .

The worker dials TEMPORAL_ADDRESS, default localhost:7233. CLI-direct runs like these need no separate vect worker; the only worker you start is this one, which serves only the smoke queue.

A default you can swap: vect finds the Temporal server through the workspace’s temporal block in server.vect. A step can name another one with endpoint = { address = "…" }, but then your worker must serve that server too. Leave it out and the step runs where the rest of the workspace does.

4. Apply

In another terminal, from the example folder, point vect at the server config, then plan:

export VECT_CONFIG=$PWD/server.vect
vect plan 2>/dev/null
plan of workspace "dev": succeeded (run 01a1e2a1-a10c-7163-aeba-853296ee417e)
  temporal_activity.smoke  run    will run
  terraform.app            run    will run
  terraform.report         defer  unknown inputs: inputs.smoke

plan doesn’t run your code. It reports run for the step with an unknown result, so everything downstream of it is deferred until apply. Now apply. (Both commands end in 2>/dev/null, which leaves out the engine’s own output and Temporal’s client log, which go to standard error; keep them in your job log.)

vect apply 2>/dev/null
apply of workspace "dev": succeeded (run 01a1e2a1-b28d-7543-bef6-200b68da3082)
  temporal_activity.smoke  applied (run)  will run
  terraform.app            applied (run)  will run
  terraform.report         applied (run)  will run
deployment pointer 1 appended

The worker printed SmokeTest started, url=http://localhost:8080. To see what report received, ask inspect to read the deployed step result from Inventory:

vect inspect --observe temporal_activity.smoke --observe terraform.app
…
node "terraform.report" {
  depends_on = ["temporal_activity.smoke"]
  result_sources = [{
    address    = "temporal_activity.smoke"
    provenance = "observed"
    version    = 1
  }]
  …
  input = {
    inputs = {
      smoke = {
        healthy = true
        status  = 200
        url     = "http://localhost:8080"
      }
    }
    source = "./roots/report"
  }
}

(Output trimmed to report’s node. vect output shows root outputs only, and this project declares none.)

5. Decide when it runs again

Apply again with nothing changed:

vect apply 2>/dev/null
apply of workspace "dev": succeeded (run 01a1e2a1-cb1c-7d1f-ab11-0dca7c97c184)
  temporal_activity.smoke  unchanged  inputs unchanged
  terraform.app            unchanged  inputs unchanged
  terraform.report         unchanged  inputs unchanged

The worker printed nothing. vect recorded the step’s result in Inventory and reused it, so report still got its input but your code never started. The step re-runs when its trigger fingerprint changes, and the fingerprint is made of two things you control:

  • Its inputs (the default, use_inputs = true). Change the URL in roots/app/main.tf to http://localhost:8080/index.html and the step runs again, because input changed. app and report update too:

    vect apply 2>/dev/null
    apply of workspace "dev": succeeded (run 01a1e2a1-d64b-756e-acf3-9136d76ac209)
      temporal_activity.smoke  applied (run)     will run
      terraform.app            applied (update)  will update
      terraform.report         applied (update)  will update
    deployment pointer 2 appended
  • Your keepers. Change suite = "v1" to "v2" and the step runs even though no input moved. Use this to force a re-run, like terraform_data’s triggers_replace:

    vect apply 2>/dev/null
    apply of workspace "dev": succeeded (run 01a1e2a1-e1c6-76c2-8d3e-8e473ef3c1f1)
      temporal_activity.smoke  applied (run)  will run
      terraform.app            unchanged      inputs unchanged
      terraform.report         unchanged      inputs unchanged
    deployment pointer 3 appended

For a migration, where the inputs are context rather than cause, set use_inputs = false and let only a keeper (a schema version, say) trigger a run. Change and reuse shows that with a workflow.

6. When the step fails

Stop the web server, bump the keeper to "v3", and apply. The worker logs the error, and the run fails:

vect apply 2>/dev/null
apply of workspace "dev": failed (run 01a1e2a1-eddd-7822-b776-3a5c9de3e683)
  temporal_activity.smoke  failed     temporal_activity.smoke: apply: activity "SmokeTest" (vect-…) failed (error type "Unreachable")
  terraform.app            unchanged  inputs unchanged
  terraform.report         skipped    upstream temporal_activity.smoke failed
node temporal_activity.smoke failed: temporal_activity.smoke: apply: activity "SmokeTest" (vect-…) failed (error type "Unreachable")
deployment pointer 4 appended

Everything after the step is skipped, so a failed smoke test stops the rollout, and the command exits 1, as for any failed run (runs). A failed apply still appends a deployment pointer when it commits a node or the accepted bundle changed (STOR-DEPLOY-1 and STOR-DEPLOY-2). Here the keeper edit changed the bundle, so the pointer is expected, and the recovery below appends the next one.

Notice what vect shows: the activity name, its ID, and the failure’s type, the second argument you gave NewNonRetryableApplicationError. It never shows input or result, so put what you want to see in the error type, and read the detail from the worker’s own log (could not reach http://localhost:8080 … connection refused) or the Temporal UI.

A step that never finishes fails on timeout. Stop the worker too, set timeout = "10s", bump the keeper to "v4", and the apply ends after ten seconds. Its failure reason is deadline exceeded (timeout 10s), shown here with the rest of the line trimmed (…), because on some runs Temporal’s own activity … failed message is reported instead:

vect apply 2>/dev/null
apply of workspace "dev": failed (run 01a1e2a1-f83f-77a8-b2c7-7c94818fbc85)
  temporal_activity.smoke  failed     temporal_activity.smoke: apply: …
  terraform.app            unchanged  inputs unchanged
  terraform.report         skipped    upstream temporal_activity.smoke failed
node temporal_activity.smoke failed: temporal_activity.smoke: apply: …
deployment pointer 5 appended

(That’s also what you see when no worker serves the queue.) The activity itself keeps running in Temporal; see below.

vect logs replays what vect itself recorded for the step, not your worker’s output:

vect logs --run 01a1e2a1-eddd-7822-b776-3a5c9de3e683 --node temporal_activity.smoke
temporal_activity.smoke [apply #1] | activity vect-… started on task queue smoke

To recover, fix the cause: start the web server and the worker again, put timeout back to "2m" and the keeper back to "v3", and run vect apply. A failed step is never memoized. Here the timed-out activity was still waiting in Temporal, and the restarted worker finished it, so the recovery adopts that completed execution (adopted previous execution) instead of starting another (EXEC-IDEM-6). Without such an execution, the step runs again. Either way report follows:

cd ~/www && python3 -m http.server 8080
cd worker && go run .
vect apply 2>/dev/null
apply of workspace "dev": succeeded (run 01a1e2a2-2d78-72f9-8e4b-06a01ca5874b)
  temporal_activity.smoke  applied (run)  adopted previous execution
  terraform.app            unchanged      inputs unchanged
  terraform.report         unchanged      inputs unchanged
deployment pointer 6 appended

(report shows unchanged because the result matches what it last applied with.) One more apply is unchanged throughout.

Things to know

  • No worker, no progress. If nothing serves the smoke queue, the step waits until vect’s timeout runs out. When a step hangs, check that the worker is up, that it’s on the same Temporal server, and that the queue and activity names match the HCL exactly.
  • Cancelling vect apply doesn’t cancel your code. It stops waiting; the activity keeps running. Only a run that is recovering a previous run adopts that run’s completed activity instead of starting another (EXEC-IDEM-6); an ordinary later apply starts a fresh one. Keep steps idempotent.
  • Approvals aren’t available for steps yet. temporal_workflow can’t pause an apply for a human decision; that is a planned feature (#61), so don’t build on it.
  • Standalone activities need a recent Temporal. temporal_activity uses the standalone activity API, so the server and go.temporal.io/sdk must both support it. The dev server from the current Temporal CLI and a freshly tidied module do.

See also