Staging now goes down to its floor with one command and comes back with another. On 2026-09-16 a park took 87 seconds and the bring-up that followed took 677, from an estate holding little more than a project, two buckets, a node image and some secrets, to a Kubernetes cluster with Argo CD reporting its root application synced and healthy. Nobody typed anything in between.

Staging is a whole estate: control plane nodes, a managed database, a load balancer and a gateway, billed whether anyone is testing on them or not. Switching it off while nobody works on it stops that spend, and it is only worth doing if switching it back on is cheap.

The first teardown was a ledger#

The first time I parked staging, on 2026-09-13, it was a hand-built list of targeted destroys plus four repairs made outside OpenTofu. It worked, and it taught me what a list like that gets wrong. It names what should go, so anything added to a root afterwards survives by omission and keeps billing. It has no memory of why each step exists, so the next run repeats the same surprises. And it asks a person to type the same careful sequence every time, which is how a careful sequence stops being careful.

Bringing the environment back was worse. The bring-up runbook had a ledger of its own, including two variables a bring-up has to pass on the command line and a close-up step that takes node access away again. Forgetting that close-up leaves every node's Talos API published.

Declaring what survives rather than what goes#

Each environment now has a small record listing, per OpenTofu root, the resources a park must not destroy. Each destroy plan is built as a destroy with every declared survivor excluded, so the default for anything new is to be destroyed. A resource added next month goes with the next park unless someone decides, in writing, that it should stay.

The survivors are the things a rebuild cannot mint again. The Talos machine secrets bundle is the clearest case: a fresh bundle is a different cluster, and losing it means resetting every node by hand. Beside it sit the secrets planted by hand, the generated database password (so a rebuilt database carries the password its stored copy already has), the registry namespace, and the egress address and DNS challenge registration that the zone's records name. Under all of that is a bootstrap floor no plan can reach at all: the project, its credentials, the state and image buckets, the reserved edge address and the node image.

Before trusting the exclusion flag with a real estate I measured what it does in destroy mode, in a scratch root, against the exact OpenTofu release in use. Excluded resources and everything they depend on are kept, while what depends on them is still destroyed, which is exactly what lets the database password survive while the database that reads it goes. A kept resource does not appear in the plan at all, not even as a no-op, so a survivor can only reach the gate through a missing exclusion. And once only survivors remain, the same plan reports nothing to do, which is why a second park is a no-op rather than an error.

Every plan goes through a kill switch#

Both directions run through one entrypoint, and it refuses production by name before it reads anything else. It loads the operator's credentials file only when that file is readable by its owner alone and sets none of the variables that would let a value from some other shell leak into a root.

Nothing is applied with auto-approve. Every step builds a saved plan, a small gate reads it as JSON and classifies each change, and only then is that saved plan applied, so what the gate read is what runs. The gate knows what the moment forbids. Bringing up, it refuses any delete, replacement or forget, and it refuses to create a new Talos secrets bundle, ever. Parking, it refuses to touch anything declared to survive or belonging to the floor. It prints actions and addresses and never a value, because a plan's JSON carries sensitive values in plain text.

Steps run under errexit, so a failing command stops the run instead of passing on the strength of the step's last line. However a run ends, by error, interrupt or hang-up, the exit path closes node access through the same gate, and when it cannot, its last line says how to close it by hand. The exit code tells the two apart, so stopped safely and stopped with access open never look alike.

Refusing early, and naming everything#

A bring-up starts with three checks before any plan. One verifies the bootstrap floor: both buckets, the node image, the secret store, the edge address and the DNS records that answer with it, and a credential that can reach none of what the bootstrap withheld from it. Two of those checks pass by being refused. A key that could list IAM applications could grant itself anything, so a refusal is the answer I want.

The second compares the identifiers the roots carry with the estate and with the environment record, because those ids were transcribed by hand and a stale image id still names a real image: a plan built on it succeeds against the wrong one. The third proves every secret a root declares is either planted by the root or named by a runbook step.

Each check prints a pass or fail line per item and keeps going after a failure, so whoever is repairing the estate sees every problem in one run rather than one per attempt.

What held a park up#

Two things outside my code decided how park had to be written.

The cloud provider's Terraform provider, at the two versions before the one I now pin, could not destroy a server attached to a private network at all. The private interface delete answered with a 412 from inside the server's own delete, where no targeting could reorder it, and the error named no resource. Stopping the servers did not help. The fix landed upstream in the next release, so the roots moved to it and park carries no workaround.

The other is a gateway network that can sit in a detaching state with no documented cause or remedy. The cloud root is therefore destroyed in two plans: the first keeps the gateway network and what it rests on, the second deletes it out of band after the gate has passed, then waits a bounded five minutes. The provider treats a gateway network that is already gone as deleted, which is what lets the saved plan still apply. On a timeout, the run names the ids and prints what still holds the private network rather than guessing.

A park then removes what outlives every root, database snapshots and backups whose instance is gone and volumes a PersistentVolumeClaim left behind, and finishes with a sweep of the whole project. Anything neither a survivor nor the floor accounts for fails the run. That sweep is what replaces reading the console, so a resource a root stopped tracking shows up in a terminal rather than on an invoice.

Coming back without a person#

The bring-up decides for itself how much it needs to do. When the cluster's resting plan is empty and no node holds a staged configuration, it writes the client configurations, waits for every node to be ready and never opens node access, so a second run against a running estate applies nothing. Otherwise it opens access, probes each node, and asks for an automatic apply only from the nodes still in maintenance mode, because the applies run in parallel and the same request on a running control plane would reboot all three at once. Staged nodes are rebooted one at a time, with a health check after each.

One OpenTofu behaviour cost a live bring-up before the driver handled it. Applying a saved plan re-reads the root's committed variable files and refuses any value that differs from what the plan recorded, but it never compares a default. So a plan built with a command-line override against a committed file stops at apply, while the same override against a default applies cleanly. The driver now restates exactly the overrides it planned with, rather than trusting the plan file to carry them.

Every wait is bounded and says what it last saw when it gives up. The run finishes by proving the edge answers on the environment's host, applying the platform root and waiting for Argo CD's root application to report synced and healthy.

What the numbers say, and what they do not#

The timings come from the run logs. Park took 87 seconds, most of it the two cloud destroys. Bring-up took 677 seconds, mostly the cloud root and the cluster apply, at a little over three minutes each. That bring-up rebuilt an estate parked minutes earlier, so the pair times the machinery and not the data: there was no etcd restore and no tenant load to carry back. A real recovery objective needs that drill, and I will not quote one until it has run.

Parking an environment, and bringing it back unattended