Agent Orchestration: The Control Plane Is the Product
Building one agent, or a group of them, is weekend work now. What takes a feature from a product requirement to a merged, tested change is the substrate underneath: a deterministic control plane that conducts non-deterministic agents across the whole lifecycle, and can prove what shipped and why.
The Agents Are Not the Hard Part
Chaining model calls is commodity. Most "AI SDLC" projects that stall at the demo stall for the same reason: eight agents end up contending for the same state, nobody can say which one is allowed to move a feature forward, and a review that returns "fail" does not actually stop anything. The fix is not a better agent. It is the system that sequences them.
This page describes that system as a set of design decisions you can adopt one at a time. It is drawn from a production platform that RJ Lindelof's engineering team built and runs, which takes a feature through product requirements, architecture, technical review, code generation, code review, a provisioned test environment, testing, and final approval with eight specialized agent personas, one per phase - and through which, by that team's count, around 90 real workflows had moved when the design was written up.
The pitch inverts the usual one. The agents are the hook. The orchestration substrate - the state machine conductor, the split between generating and judging, parallel dual review, fail-closed gates you can prove, a read plane that cannot write, and a versioned agent fleet - is the achievement.
The Conductor Is a State Machine
The authoritative state of every workflow is a server-side finite-state machine, persisted to a durable database. Step order and gate policy are code, not agent-driven control flow. The agents propose the next move; the state machine decides it. The agents are the soloists, and the state machine holds the score.
That one choice turns several independent, non-deterministic processes into a single coherent pipeline. Sequencing long-lived, resumable, multi-phase work across a fleet of agents is a distributed-systems problem, and it has to be engineered as one: durable state, explicit transitions, idempotent steps, and a single owner for every decision about what happens next. This is the piece most teams skip, and it is why most agentic pipelines never leave the sandbox.
Every arrow is a transition the state machine owns. Each one has a gate, and nothing crosses it because a model decided it should.
Which Agent, When, and With What Context
The state machine is not bookkeeping. It is the brain of the system, and it owns three decisions that teams usually leave to the agents. That is the difference between a fleet of agents and a pipeline you can control.
Which agent
When a step clears its gate, the state machine unlocks the next agent in the order and makes that step claimable. Nothing runs because a model decided to run.
At which moment
A step becomes available only after every upstream gate it depends on has cleared. Order is enforced by the transition table, not by agents agreeing among themselves.
With which context
When the next agent starts, its context is assembled deterministically from upstream - prior step outputs plus compressed distillates - and injected at fetch time. Each agent begins with exactly the inputs its phase depends on, and nothing it does not.
Deterministic context assembly is also what keeps a long relay honest. An eight-stage pipeline that lets each agent gather its own context accumulates drift at every hand-off; one that builds each stage's inputs the same way every time does not. It is context engineering applied between agents rather than inside one session.
Separate State From Content
The split between workflow state and work product is just as deliberate as the state machine itself. Each lives where it can be trusted for its own job.
| What | Where it lives | Shape | Why there |
|---|---|---|---|
| Authoritative state | The state machine, in a durable database | Which phase a workflow is in, which gates it has cleared, who claimed it | One writer, explicit transitions, a single answer to "where is this feature" |
| Work product | Object storage, keyed by workflow, step, and kind | A versioned Markdown file per artifact - the PRD, the architecture document, and so on - each immutable, with its own content type and ETag | Every version is kept, any two can be diffed, and a reviewer always knows exactly which bytes they approved |
Keeping artifacts as plain, versioned documents rather than rows in the state table is what makes the later patterns cheap: a deterministic diff between two versions of an artifact, an evidence trail per step, and a context builder that reads exactly the upstream versions a step was approved against.
Generate With Full Power, Judge With a Bounded Service
Generation lives in the IDE
The agents run at full capability where the developer already works - Claude Code or Cursor, reached through an MCP plugin - with a human in the loop. Generation gets maximum model power and the developer's attention.
On the server, generation is deliberately absent. The generation client is a stub, and the server's chat endpoint returns 501 Not Implemented. Nothing on the server can write an artifact on its own.
Judging lives on the server
The server has exactly one model role, and it is a decisive one: judging. It scores a finished artifact against a rubric through a hosted model API and returns a structured verdict.
The block-or-allow choice over that verdict stays deterministic, in the calling code. Verdicts are typed inputs that compose into the state machine. The pipeline's control flow never depends on a model's output being right on a given day.
Neither responsibility leaks into the other. Generation can use the best model available without that model ever holding a gate, and adjudication is a bounded, inspectable service whose output is data, not a decision. It is the LLM-as-judge pattern with the judge's authority taken away and handed to code.
Two Reviews in Parallel, Never Averaged
Every phase runs two reviews at the same time, and they merge into a single review result without ever being blended into a single score.
The Critic: quality
A model-based reviewer focused on quality - completeness, clarity, consistency with upstream artifacts. Its output is a structured verdict with findings, not a free-text opinion.
The security evaluator: policy
A deterministic, policy-driven evaluator that produces its own structured verdict. The policy that decides which checks apply, and when, lives in one source-of-truth YAML bundle backed by a versioned registry.
Why they stay unblended
A quality signal and a security signal must never be averaged into one soft "looks good." That is precisely how a security failure gets laundered through strong quality reasoning: an excellent, well-argued architecture document that skips the threat model would score high on the blend.
The evaluator also fails closed. A check the registry does not recognize returns an explicit not-yet-implemented result rather than silently passing, so a typo in the policy bundle or a check that was declared but never built shows up as a blocker, not as a quiet green.
A Verdict Is Not Enforcement
It is easy to build an evaluator that returns "fail." That is not enforcement, and conflating the two is where most of these systems quietly break. A produced verdict is not an enforced verdict.
A blocking check is only real when a non-compliant artifact is actually stopped at the transition. In practice that means the approval call itself refuses to advance - an HTTP 409 with a blocked_by_security reason and the failing check id in the body - and that refusal is proven, not assumed.
Produced
The evaluator says "fail." A dashboard turns red. Something, somewhere, could still move the step to done.
Enforced
The transition refuses. The approval returns 409 with the failing check id, and no side path exists from "in review" to "done."
Proven
A negative integration test publishes a deliberately non-compliant artifact and asserts the block. It goes red the moment the gate regresses.
Every blocker ships with its negative test, so the gate list is short and true rather than long and aspirational. On the platform RJ Lindelof's team runs, three checks are enforced this way: a threat model at the PRD step, and ADR completeness plus an approved authentication pattern at the architecture step. Three, named and proven, not twenty that exist only in a policy file.
A negative test is stronger change-management evidence than a policy document that asserts a control exists. It is reproducible, machine-checked, and fails loudly if the orchestration ever lets through something it should not. An auditor can rerun it. That is what self-enforcing policy looks like, and it is the same idea as the release evidence pack on the Security and Compliance page, applied to each gate.
The Pipeline Can Go Backwards, Under Control
A strictly feed-forward pipeline has an obvious failure mode: a downstream agent discovers that an upstream artifact is missing something it needs, and there is nowhere to put that finding. Most systems either work around the gap silently or surface it too late to matter. The alternative is a controlled backward edge - a circle-back.
Every agentic step interrogates its upstream inputs before it generates anything - a "grill" pass that walks the decision tree for its task and checks that each branch is actually covered - and then re-checks its own output afterward. When either pass finds a gap, the agent can request a circle-back to the upstream step. The state machine has explicit states for exactly this: grilling, selfEvaluating, gapDetected, and circleBackActive. The engine derives the whole behavior from a step's position in the order, so a newly added agent inherits it with no extra wiring.
| Gap class | What it means | Who approves the circle-back |
|---|---|---|
| Additive | The upstream artifact is right but incomplete - a missing field, case, or constraint | The developer who found it |
| Structural | The artifact's shape needs to change, but its intent holds | The developer who found it |
| Foundational | The upstream artifact solved the wrong problem | Escalates to the original step's approver |
What happens on a circle-back
- It is never automatic. A human approves it, like every other transition.
- The upstream agent regenerates a complete artifact, not a patch.
- A deterministic section-level diff - no model call - shows the reviewer exactly what changed.
- The same dual review, Critic plus the fail-closed security gate, runs on the new version, whoever is approving it.
The loop budget
A strict budget keeps the backward edge from turning into infinite regress: one circle-back per step, two per workflow.
The result is a pipeline that is deterministic and forward-driven by default, but can fold back on itself - once or twice, under human control - the moment an agent proves it has to.
QA Agents Reason Over a Map of the Product
Regression testing is where most automated pipelines go blind: they either re-run everything or quietly guess. A QA agent can do neither if it queries a live knowledge graph of the application under test. The nodes are screens and API families; the edges capture how they actually relate - which screen navigates to which, which screen calls which API, which screens share a locator.
Given a change, the agent walks the graph from the touched code to every screen and flow that can reach it, and tests that blast radius rather than the whole product or a hunch. The graph on the platform RJ Lindelof's team runs holds roughly 600 UI screens and close to 2,000 edges - large enough that "re-run everything" is expensive and "guess" is dangerous.
Navigates to
Which screens a user can reach from which, so a change to one step of a flow selects the whole flow.
Calls
Which screens depend on which API family, so a backend change selects every screen that consumes it.
Shares a locator
Which screens reuse the same UI element, so a component change selects every place it renders.
A Read Plane That Cannot Write
The execution model is a command-query split (CQRS) across two surfaces, and the split is architectural rather than cosmetic.
The dashboard
Status, artifacts, and activity logs for every workflow, with no mutation path into workflow state at all. It can watch everything and change nothing.
The IDE plugin
The MCP plugin in the developer's IDE is where every state change happens. The single bridge between the two planes is a claim token that hands a workflow off to the IDE.
That split is how you observe many concurrent workflows without the observability surface ever being able to corrupt the state it is observing. It is a decision you make when you are engineering for throughput and correctness, not for a screenshot.
A Versioned Agent Fleet
The personas are a locked, versioned registry of prompts compiled into the plugin - not an external tool the pipeline shells out to, and not a prompt each developer keeps on their own machine. You can diff an agent, pin its version, and audit which persona version produced which artifact.
Treat a persona change the way the Engineering Practices page treats a model upgrade: it is a change, it goes through review, and it runs against an eval fixture set before it is promoted. An unversioned persona is an unreproducible artifact.
Where the Real Bugs Live
None of this comes free, and the gates are the opposite of easy. The invariant is simple to state: no state advances past a gate it did not clear. Keeping it true means the check has to live at the transition itself, so that no side path - an admin endpoint, a retry handler, a bulk-update script - can move a step from "in review" to "done."
That seam is where the real bugs live, and where most of the engineering effort goes. The substrate stays strong only by continuously pressure-testing exactly that boundary, which is the other reason every blocker carries a negative test. Expect to keep finding side paths; plan for the work of closing them.
Orchestrated, or Just Chained?
Ten yes-or-no questions. Every "no" is a place where a model, not your system, is making a decision about what ships.
- Is the authoritative state of every workflow held outside every agent, in one server-side state machine?
- Are step order and gate policy written as code, rather than as instructions in a prompt?
- Does the machine, not the model, decide which agent runs next?
- Is each agent's context assembled deterministically from approved upstream outputs at fetch time?
- Is work product stored as versioned, immutable artifacts, separate from workflow state?
- Is the block-or-allow decision on a model's verdict made by deterministic code?
- Are quality and security verdicts kept separate rather than averaged into one score?
- Does an unknown or unimplemented check fail closed?
- Does every blocking gate have a negative test that proves the block still happens?
- Is backward movement explicit, human-gated, and capped by a loop budget?
Where It Fits in AIDLC
An orchestration substrate spans every phase rather than living in one. The PRD and architecture steps are Analyze and Ideate with gates attached; code generation and code review are Develop; the provisioned test environment, graph-driven QA, and final approval are Launch; and the versioned persona registry, the negative tests, and the circle-back history are the material Curate learns from. Where Agents in the Pipeline covers the controls around one agent, this page covers the system that conducts all of them.
Frequently Asked Questions
Because the hard problem in an agentic SDLC is sequencing long-lived, resumable, multi-phase work, which is a distributed-systems problem. When the authoritative state lives in a server-side state machine, step order and gate policy are code: agents propose the next move and the machine decides it. That keeps several non-deterministic agents from contending for the same state and makes every transition auditable.
A produced verdict is an evaluator returning fail. An enforced gate is the transition itself refusing to advance: the approval call returns an error such as HTTP 409 with the failing check id. A negative integration test then proves it by publishing a deliberately non-compliant artifact and asserting the block, so the test goes red the moment the gate regresses.
Yes, under control. When an agent finds that an upstream artifact is missing something, it requests a circle-back instead of silently working around the gap. The request is human-gated and classified by blast radius, the upstream step regenerates a complete artifact with a deterministic diff for the reviewer, the same reviews run again, and a loop budget of one circle-back per step and two per workflow prevents infinite regress.
Conduct the Fleet, Do Not Just Run It
Start with the controls every single agent needs, then put the state machine and the gates around all of them.