Common wisdom says to put a strong model in charge and let cheaper models do the work. I wired four Claude models together in Claude Code to test that advice on Terminal-Bench 2.1, a benchmark that drops an agent into a sandboxed terminal and asks it to solve 89 real command-line tasks, from compiling projects to recovering passwords, with five attempts at each. The score is simply the share of tasks the agent manages to solve, and I ran it under the official conditions.
It backfired in four separate ways:
- Delegation talked Opus into refusing valid security tasks.
- A review step I had left optional got skipped, and the unreviewed work succeeded half as often.
- The priciest model ended up in the highest-volume seat.
- Whenever the orchestrator went past six hand-offs, it spent nearly four times as much to succeed half as often.
The final bill came to $1,178, and the system solved 78% of the benchmark’s tasks, which put the run in seventh place at roughly twice the cost of the top single-model entry. Here is the four-model rig that produced it, wired through Claude Code with no third-party framework:
Model roles and delegation flow
Four Claude models, each pinned to one role, wired through Claude Code. The orchestrator plans and delegates; it never edits a file itself.
Every other entry on the board is a single model; mine was a four-model orchestrator. It solved 78% of the tasks, good for seventh place, and it would have taken third if my own orchestrator hadn’t talked Claude into refusing to work.
Terminal-Bench 2.1 leaderboard
Terminal-Bench 2.1, official entries plus mine. The outlined extension shows where the same run lands if the three refused security tasks are credited from the direct-run control: 78.0% becomes 80.5%, and seventh place becomes third.
Three of those failures deserve a closer look, because the model never actually got them wrong: it refused to attempt them. In a direct-run control, the same model solved all three. That gap is the entire distance between seventh place and the podium.
Build the same orchestrator in two minutes
Before looking at why the models refused to work, here is the exact wiring that caused it: no framework, no orchestration library, no glue code, just Claude Code and four files. The first is your CLAUDE.md. If you already have one, you do not replace it; you append this block to the bottom, and the rest of your project instructions stay exactly as they were.
CLAUDE.md (append to your existing file)
# Orchestration
You are the orchestrator (Fable 5). Plan the task
and delegate the work to the role subagents below
via the Task tool. Do not edit files or run
state-changing commands yourself. Each subagent
runs on its own model and cannot see this
conversation, so give it a self-contained brief.
## Roles
- scout (Haiku 4.5): read-only reconnaissance.
- executor (Opus 5): edits, commands, builds.
- verifier (Sonnet 5): reviews the work.Then three role files under .claude/agents/, one per teammate. Each pins a model and an effort in its frontmatter, so swapping the whole team is a one-line edit:
.claude/agents/executor.md
---
name: executor
description: Edits, commands, builds, debugging.
model: claude-opus-5
effort: xhigh
---
You are the executor. You receive one
self-contained subtask and implement it: edits,
commands, builds, debugging. Do the subtask
yourself; do not spawn subagents..claude/agents/verifier.md
---
name: verifier
model: claude-sonnet-5
effort: xhigh
---
You are the verifier. Given the task and the
current state, report whether the requirements
are met and, if not, what is missing..claude/agents/scout.md
---
name: scout
model: claude-haiku-4-5
effort: xhigh
tools: Bash, Read, Glob, Grep
---
You are the scout: read-only reconnaissance.
Report what was asked. Never change state.That is the whole orchestrator. Change model: claude-opus-5 to claude-sonnet-5 and you have a cheaper executor. Change effort to test high against xhigh. You can stand up a completely different team in the time it takes to edit frontmatter.
These prompts are intentionally bare. Every line describes orchestration, not task hints. That restraint is the whole reason the 78% is comparable to the single-model scores on the board. Those entries run with no custom prompt at all, so the moment I start writing smart instructions I stop measuring orchestration and start measuring my own prompt engineering, and the comparison quietly stops being fair. I kept the scaffold to the bare wiring so that the only thing separating this run from a plain single-agent one is the delegation itself. Everything that follows is a property of that wiring, not of a prompt I tuned to the test.
Backfire 1: Delegation made Claude refuse the work
Three tasks never resolved, not once in five tries: vulnerable-secret, break-filter-js-from-html, and password-recovery. The executor never even got the chance to get them wrong, because it would not attempt them.
Every attempt came back the same way, in one turn, at zero cost:
Claude Code can’t respond to this message with Opus 5. Try rephrasing the request in a new session or change your model.
That is the safety classifier refusing legitimate benchmark tasks: find a leaked secret, block an XSS payload, recover a password, the kind of work a security engineer does on a Tuesday.
The catch: I ran the exact same three tasks through a plain single-agent Claude Code session, no orchestration, and asked Opus 5 directly. It solved all of them, six for six. The same model, on the same tasks, refuses the work when it arrives as a delegated subtask and does it without complaint when I ask it myself.
Direct vs. delegated outcomes
Opus 5 solves the three security tasks when asked directly, and refuses every time they arrive as a delegated subtask.
My first guess was that the newer Opus 5 had simply turned more cautious than the Opus 4.8 that ranks just above me, so I tested it: same task, same delegated setup, both models, and they behaved identically. The refusals do not come down to one model being more careful than another; what changed is the framing. I cannot prove the exact mechanism, but the shape of it is hard to miss: a subtask handed over by another agent, stripped of its original context and reduced to a bare “find the secret,” seems to read as more suspicious to the safety layer than the same request from a human. What I can prove is the effect: my orchestrator manufactured the refusal, and the single-agent runs on the leaderboard never hit it, because they never delegate.
That is a failure mode you only get once you start wiring models together, and it set the theme for everything that followed: the problems were in the framework, not the models.
Backfire 2: Optional verification cost me 48 points
The verifier was optional. The orchestrator decided when to call it, and it did so on 76% of runs. That single choice was the strongest predictor of success in the whole benchmark:
Resolve rate by review status
When the orchestrator called its verifier, tasks resolved at 91%. When it skipped review, 43%.
A 48-point gap, same models, same tasks. While there is some selection bias here (the orchestrator only verifies work it thinks is finished), the gap indicts my prompt. My CLAUDE.md said “the verifier reviews the work.” It never said “you must verify before you finish,” so a quarter of the time the orchestrator skipped review and shipped, and half of those shipped wrong. I had left the single most valuable step in the pipeline optional, which makes the 78% a floor set by a loose prompt rather than a ceiling set by the models. It also hints at how much was left on the table: first place on this board is 83.8%, and the work my verifier actually reviewed resolved at 91%. With verification enforced on every run, even first place was within reach.
Backfire 3: I tiered the team backwards
Cost analysis revealed a massive unforced error. The whole run cost $1,178, which made it the second most expensive entry on the entire leaderboard, behind only Codex with GPT-5.5 and roughly twice the price of the first-place run.
Total run cost by benchmark entry
Total dollar cost of the full 89-task, five-attempt run, mine against the official Terminal-Bench 2.1 entries.
That is the number that should stop you, because all that money bought seventh place at nearly the price of first. Cost tracks one thing above all others: how much the executor generates. Output tokens correlate with a trial’s cost at r=0.93; the number of delegations barely matters. So the bill is set by whoever does the high-volume work, and I had put the single most expensive model in the run on exactly that seat.
Orchestrator spend by role
The executor generates most of the tokens, so putting the most expensive model there set the whole bill.
Opus 5, the executor, was 58% of the spend. Haiku 4.5, the cheapest model, sat in the scout role and moved the success rate by a single point. I put the most expensive model on the highest-volume job and the cheapest on the one that mattered least. In other words, the expensive part was not multi-agent orchestration itself but the way I had wired it. Move Sonnet or Haiku onto the bulk implementation, keep Opus for the genuinely hard reasoning, and the arithmetic says the same architecture runs at a fraction of the cost. That re-run is my next experiment. The cheap-hands, expensive-brain idea from the last post was right; I just built it upside down.
Backfire 4: Six hand-offs sent the system over a cliff
The number of hand-offs decided more than the cost. It also decided success, and not in the direction you would guess: past a certain point, more delegation stopped meaning more thoroughness and started meaning thrashing.
Resolve rate by number of hand-offs
Three to five hand-offs resolved 90% of the time (at $2.41 a trial). Six or more collapsed to 50%, at $9.51 a trial, nearly four times the cost.
Trials that delegated three to five times resolved 90% of the time. Fewer than that, and the orchestrator was under-decomposing, trying to hand off whole problems in one shot. Six or more, and it had lost the plot, re-delegating the same work and burning nearly four times the tokens for half the success. A good orchestration prompt should not just permit delegation; it should cap it.
Reliability, not capability, was the ceiling
The run cost about $1,180 across 445 trials, and the thing holding the score down was not raw ability: any single attempt solved 78% of tasks, and given five attempts that climbs to 93%.
Cumulative task coverage across five attempts
A single attempt solves 78% of tasks; five attempts reach 93%. The gap is unreliability, not inability.
The team can almost always get there; it just cannot do it reliably on the first try. Twenty-eight of the 89 tasks were flaky: solved on some attempts, missed on others, same model and same task, different luck. Four were simply out of reach, never solved in five tries: compiling CompCert, two different MIPS interpreters, and a text-retrieval task. That is the real ceiling, and it is small.
Hard tasks barely dented it
Here is the one result that surprised me in a good way. I expected a four-model contraption to fall apart on the hardest tasks, where a wrong hand-off compounds and the whole thing thrashes. It did not.
Resolve rate by task difficulty
Resolve barely moves from easy to hard: 85%, 79%, 76%. The orchestration does not fall apart on the hard tier, even though those tasks cost 2.4 times as much per trial.
From easy to hard, the resolve rate drops nine points: 85%, 79%, 76%. That is a gentle slope, not a cliff, even though the hard tasks cost 2.4 times as much per trial and made the executor generate more than twice the output. Whatever else the orchestration got wrong, it held its footing on exactly the tasks where you would expect coordination overhead to sink it. If there is a case for wiring models together, it starts here.
What 445 agent trajectories revealed
Terminal-Bench keeps a “Hacks” column that docks your score for gaming the tests, and the benchmark has a history of it: one current top-five entry, Cursor CLI with Grok 4.5, carries a 9-point reward-hacking penalty, so its real score is closer to 70% than the 79% on the board. Worth asking, then, whether my orchestrator was quietly cheating its way to 78%. The trajectories say no: no successful gaming, because the sandbox blocks the device and mount escapes agents reach for when they go looking for the answer file. A clean run is worth as much as the score.
A handful of other numbers stuck with me:
- Failure cost more than success. A losing trial averaged $3.60 and 26 minutes; a winning one, $2.52 and 19 minutes. The agent flails longer before it gives up than it works before it succeeds.
- The bill is brutally top-heavy. The most expensive 25% of trials ate 57% of the total spend. One task, cell segmentation, cost $23 in a single attempt and nearly hit my per-trial budget cap.
- Agents read far more than they write. 1.16 billion tokens went through the run: 598 million read in, 23 million generated out. A 26-to-1 ratio, and only 48% of it served from cache.
- The whole thing ran for 145 agent-hours. Wall-clock, across the 445 trials, 1,289 delegations. A small server farm’s worth of models talking to each other to fill in an 89-row scorecard.
None of that is exotic; it is the ordinary physics of running frontier models in a loop.
Orchestration is a framework problem
Every problem that held the score down lived in the framework rather than the models: the framing of a delegated task, a prompt that made review optional, an expensive model on the wrong seat, an orchestrator that thrashed when it over-delegated. Not one of these problems came from the models being too dumb.
So, is orchestration worth using? Four frontier models behind one orchestrator did not beat a single good model on this benchmark, and they cost more doing it, yet I would not write the approach off, because everything that went wrong traces back to how the orchestrator was configured: the prompt discipline, the verification contract, the model-to-role assignment, the context you hand across a delegation. I made all four mistakes in a single run and still finished seventh. Fixing the refusals alone would have taken third place, and enforcing verification would have put first within reach.
The good news is that the framework is the part you control, and it is four files you can edit in two minutes.
The setup is above; you need Claude Code and a Terminal-Bench harness, nothing else. If you have a theory on why a delegated request trips the safety layer when a direct one sails through, I would genuinely like to hear it.
As for the question this experiment set out to answer: yes, orchestration is worth it, but the framework around the models needs far more careful adjustment than the defaults I ran with here. What is your take? Share your thoughts on Hacker News.
