I was lucky to be in assignments where I’ve learned a lot about building and running platforms in the cloud. I also learned that you need many resources to build and run a platform. Since we cannot allocate all of our time to this platform blueprint, we decided early to use agents for implementation and documentation.
Many of the Terraform and Kubernetes manifests behind Parts 2 and 3 came through pull requests opened by our coding agent. It mapped the reference architecture onto an unfamiliar provider, read live state and pushed branches.
The agent could prepare a branch, but it could not merge the pull request or run a terraform apply, and neither could one of us without going through the same review path. A reviewed merge started the only apply workflow, using an identity scoped to one environment.
My work shifted. I typed fewer resource blocks and spent more time comparing proposals with live state, reading plans and checking whether the recorded reason matched the change. But that only works when the reviewer knows enough to stop a plausible answer.
The Same Guardrails You Would Give A Person
Before we started handing work over to an agent, we ensured that an AI harness was in place. Each repository has an operating guide and skills that defines its job, what belongs elsewhere and the rules we learned while building. Stability, blast radius, least privilege and recoverability come before convenience. Providers and backends stay in roots. Reading live state is routine while changing it directly is strictly forbidden.
The guidance gets more specific near the work. Our runbooks folder has its own rules because an incident procedure needs different checks from a Terraform module, while Terraform and Helm conventions load only for sessions that use those tools. The agent starts each task with a small, relevant piece of the platform’s history instead of one huge prompt.
Every check from Part 3 runs on all pull requests, and the per-environment identities from Part 2 limit the eventual apply. We already needed those controls for ourselves. Everyone has a bad afternoon, copies a value from the wrong tab or points a command at the wrong environment.
For larger changes, Claude writes and ChatGPT reviews before I do. The second model has none of the discussion that produced the diff and must judge the evidence on the page, which is useful for finding unstated assumptions or a solution that has grown beyond the problem. I make the final decision. Two models can agree on the same bad idea with impressive fluency.
Everything Goes In The Repo
A human who debugged the managed OpenSearch will probably remember its provider-assigned high port number. A new agent session starts without that memory and uses the default port number.
Before closing a session, we put what it learned into the repository. Decision records capture the reason for an architecture, the alternatives we rejected and the condition that would make us revisit it; the next session reads that before proposing a change, and a reviewer can inspect the reasoning without reconstructing it from a diff. Review became the bottleneck quickly, so that context saved real time.
A findings log records smaller provider constraints, such as a bucket that cannot be deleted while it still contains object versions. Without that entry, a later session has to investigate the same failure again. That short feedback loop produced more decision records, findings and runbooks than we would have written by hand. We also keep track of open issues in GitHub, if there is no time or urgency to fix it right away.
They still go stale. At one point, decision records referred to Terraform variables that no longer existed. We treat stale documentation as a bug because the agent will follow it just as confidently as current instructions.
Where Our Agent Was Confidently Wrong
The load-balancer work from part 3 showed what happens when the agent is uncertain. An early test appeared to show that the allowed source ranges did nothing, so it treated the observation as a provider limitation and moved filtering into the ingress controller. A bad L7 configuration then returned 403 for every request in test. Its next proposal concluded that this cloud could not gate the surface reliably.
After two days, we had three architectures around a mechanism we had never isolated. I stopped the sequence, removed the workarounds and repeated the L4 check with a disposable load balancer. An excluded caller was blocked in ten out of ten attempts, and access returned when we restored the admitted range. spec.loadBalancerSourceRanges was enforced. We never found what had invalidated the early observation.
L7 filtering still has a job. Our UI and Zeebe routes have different audiences, and the ingress controller needs the real client address from the PROXY protocol chain. The final design kept both layers for distinct reasons. The agent had sounded equally confident through all three conclusions.
Version drift was quieter. The agent used chart versions it knew from training data instead of checking the registry, and both the GitOps controller and log shipper deployed cleanly several releases behind current. Nothing failed to draw our attention to their age.
The generated Terraform also pinned the STACKIT provider to 0.80, where service-account key files looked unavoidable. Workload identity federation had arrived in 0.82, and Terraform could manage the trust from 0.95. Both versions predated our build. We found the gap when local applies moved to GitHub Actions, upgraded the provider and gave infra, test and prod federated identities. The old dependency had excluded a safer design from consideration.
It also invented plausible details. We saw variable names that sounded native to the repository and did not exist. Once, it drafted a commit message describing a step that had never run, turning an intention into an apparent fact.
During a prod outage, the agent added and removed a dummy range on a managed service’s access list in the hope that the provider gateway would reconfigure itself. Nothing changed. We eventually traced the failure to an egress policy pinned to the conventional port for a service the provider had placed on a high range. The guess had changed the control plane before we understood the traffic path.
Runbooks An Agent Can Run
Somebody follows a runbook under time pressure and without the author’s context. Any missing precision has to be supplied during the incident.
Our standard grew out of procedures we would use while tired. Every command block is self-contained and names its environment and stack. Commands, paths, metrics and thresholds must come from the repository or a read-only query. Verification includes the expected output. A destructive step identifies its point of no return and the data at risk.
# Illustrative verification step: self-contained, with a named healthy value
export KUBECONFIG=~/.kube/config/k8-test.yml
kubectl -n camunda get externalsecret app-connection-config
# READY is True within ~30s. If it stays False, the credential did not sync,
# and you are in step 4, not step 5.An experienced operator may be able to rescue a step that says “check it worked” by inventing the test from memory. An agent may instead report success against a condition it never evaluated. We write the check down, including the healthy value, which makes the same runbook usable by a tired person and an agent under supervision.
We have not handed a runbook to an agent and walked away. The flat confidence in the last section is why.
What’s Next
The agent made us faster at producing changes. Review, written context and executable checks became a larger share of the work.
Supervision stays in the operating model for one reason I have not found a technical control for: an engineer will often hesitate when they are out of their depth, while the agent can sound exactly as confident when it is guessing. That is how the load-balancer investigation lasted two days.
Part 5 measures what the platform holds under load. We ran six tests and got three wrong answers before the numbers meant anything.