A CTO agent and an engineer agent on the same model run on the same weights; only their system prompts differ. I don’t see how the title adds expertise. What it does add is a handoff, and every handoff passes on a summary that loses detail. Yet the agent companies are built on titles. Paperclip’s README puts it plainly: “Your agents have a boss, a title, and a job description.” MetaGPT and ChatDev work the same way, and Claude Code has agent teams. This might be an artifact of a company organized around human work, not agent teams.
My setup with roles:
| A role team | My setup |
|---|---|
| CTO | A lead session that watches the builder and checks its claims |
| Engineer | One builder that owns the change from start to finish, tests included |
| QA engineer or code reviewer | Blind critics, each a fresh session that returns one verdict |
| CEO / board | Me |
My setup hands off too, and loses detail like any other: the builder starts from a written task, and what reaches me is the lead’s summary. So the question isn’t whether a split costs detail, but whether it buys something worth that. A role team splits by job title. I split only where a separate agent gets something a single one can’t: an independent review or truly parallel work. In the review, the lost detail is the point: the critics never see the builder’s account of its own work.
The critic judges the work, never the builder’s story about it.
Only for work that doesn’t depend on anything else in flight.
A company has roles because one person can’t know everything or be in two places at once.
Berkeley’s MAST study annotated more than 1,600 runs of seven multi-agent frameworks, ChatDev and MetaGPT among them. The authors call the gains on popular benchmarks “often minimal” and sort the failures into 14 modes in three groups: system design, misalignment between agents, and checking the work.
Anthropic, which built a multi-agent research system, counted the cost: agents use about 4× the tokens of a chat, multi-agent systems about 15×. It also says where they don’t fit:
Some domains that require all agents to share the same context or involve many dependencies between agents are not a good fit for multi-agent systems today. For instance, most coding tasks involve fewer truly parallelizable tasks than research, and LLM agents are not yet great at coordinating and delegating to other agents in real time.
Cognition’s case against multi-agent systems comes down to two principles:
Share context, and share full agent traces, not just individual messages
Actions carry implicit decisions, and conflicting decisions carry bad results
A team organised by job title breaks both by design. Each role hands the next one a summary instead of its trace: the engineer gets the CTO’s task, not the reasoning behind it. And each role makes decisions the others never see, so the next one can act against them without knowing. My critics see even less, on purpose, but they don’t build anything. One builder makes every decision in the change, and the critics only judge it.
Two experiments put a split setup next to a single owner. Matt Shumer’s Claude of Duty tried both on one artifact. Three rounds of six parallel agents took its defects from 60 to 47 and back up to 66. One sequential pass by a single owner cut them from 66 to 26.
The Co-Coder paper ran Claude Code’s agent teams against a single sequential agent: 16.3% against 20.1% on CodeProjectEval, and 54.1% against 56.8% on DevEval. The paper’s own system, Co-Coder, splits the work too, but differently. Before any agent starts, it maps which files depend on which and hands each agent a group of files that mostly stand alone. That split beat the single agent on quality, speed and cost. The difference is dependencies: agent teams divide the work and then coordinate as they go, while Co-Coder splits only where the parts barely depend on each other. That’s the split Anthropic’s warning leaves room for, and the only kind of parallel work I split for.
A code reviewer on the same model, starting from the same framing as the engineer, is likely to miss what the engineer missed. In ChatDev, the code reviewer gets the team’s ideas from earlier phases along with the code, runs on the one model the whole company uses, and sends its comments straight back to the programmer. My critics are set up the other way round:
The closest study I’ve found is a preprint by Tae-Eun Song. One fresh same-model review plus one cross-model review found 56.7% of planted errors, against 42.7% for two fresh same-model reviews (p = .006; 30 artifacts, 150 planted errors, 900 review sessions). What it can’t tell is whether the gain comes from a fresh model or a stronger one. The mixed pair didn’t find significantly more than two reviews by the strongest outside model, and a lightweight outside model did no better than a review by the same model.
Claude Code has a second model built in, the advisor, and its docs suggest it for “tasks you want independently checked before Claude declares them done.” The main reason I don’t use it as a reviewer is its input:
The advisor receives the full conversation, including every tool call and result, and returns guidance that Claude applies before continuing.
That is the session’s whole conversation so far, not a fresh context. By the time the advisor is called, it holds the builder’s plan, its framing of the problem and its account of what it did, so the review starts from the builder’s story, the one thing my critics are kept away from. The other differences matter less. Claude calls it mid-task, when it decides to, and no setting forces a call. It runs without tools, so it can’t run anything. And every model it accepts is a Claude model, ranked at or above the one doing the work. It’s a reasonable second opinion mid-task. As a merge gate, it arrives contaminated.
The one measurement I know of doesn’t back me up yet. In Song’s preprint, a model reviewing its own work in the session that produced it found about as many planted errors as a fresh session (F1 27.1% against 28.6%, not a significant difference). The errors in that study were planted, though, so the session had never claimed its work was proven. A claim like that is exactly what my critics never see. I wrote up the advisor’s gaps, and the panel I built for decisions, in an earlier post.
It starts from the same framing as the programmer, on the same model, and reads the code instead of running it.
In late October a builder changed the fiscal-receipt add-on in my shop’s ERP, so that copies of the production database can’t issue real receipts or keep real credentials. Its report said “Proven locally”. That was true as far as it went, and nobody who read it saw what it left out. Both critics found it, independently and blind: passwords and other credentials still sat in cashier sign-in log rows. Astra also found two receipt-download calls that skipped the new guard, and wrongly called a key ID a credential; the lead session caught that mistake in the vendor’s spec and dropped the finding. After one repair round both critics passed the change, and I merged it.
The change stops copies of the production database from issuing real receipts or keeping real credentials. The report is true as far as it goes, and nobody who reads it sees what it leaves out.
That’s the rule I take from this project. Nobody caught the gap by reading the builder’s report, and the lead agent caught Astra’s mistake by reading a spec document. So reading isn’t the problem: reading the code, the data or the vendor’s spec is part of checking. What fails is reading what an agent says about its own work. One of my standing rules puts it this way: “a worker’s summary says what it intended to do, not what it did.”
The rule has a second half, because someone still ends up reading a summary, and at the top that’s me. A summary is safe to read when it carries evidence that can be re-run and someone has actually re-run some of it. The lead’s summary to me links the critics’ verdicts and the commands behind them, and before writing it, the lead re-ran the proof itself on a clean copy.
I needed the same rule for this post. I had asked a Claude Code session about role teams, and its answer said my own measurements had twice shown a single agent beating a split team. They hadn’t. Shumer’s run and the Co-Coder paper, and in the second the paper’s own split beat the single agent. The answer was confident and easy to read, and reading it didn’t catch the error. Tracing every claim in this post to its source did.
For changes to production software, I think my setup beats a role team. For running many unrelated workstreams with nobody watching, role-team tools have machinery mine lacks.
In Paperclip, agents wake on heartbeats or on a schedule, tasks link up to company goals, and company, agent and project budgets pause work at set limits. My equivalent is one overnight queue that gives each task a fresh Opus 5.5 builder, with Fable and Astra judging blind. Everything else starts when I start it, so I am the bottleneck.
For genuinely different domains, such as support, marketing or operations, a role means different tools, data and context, and splitting there is right. My two Slack agents for the shop are split that way.
And the review costs money. I pay for it on purpose, but not on every merge:
| Tier | For a change where | Review |
|---|---|---|
| Full | a mistake could reach money, customer data, fiscal records or credentials before anyone notices, and can’t be undone in minutes | Both critics, plus a design check before any code is written |
| Light | the change reaches users or runs unattended, but a mistake would show and could be undone | One fresh Claude Opus |
| None | a preview and a one-click rollback cover it | No critic |
Each repo declares its tier by what a change touches, and the lead may raise it but never lower it without me.
Song’s preprint is why I think one fresh critic is a real review: a top-tier cross-model reviewer did not score significantly differently from a fresh same-model session, and the two found partly different errors. The second vendor is the extra I pay for where a mistake is hardest to undo.
Paperclip also lets you choose a model per agent, so a role team could review across model families. The issue is what the team is organised around, not the tool.
I haven’t yet run the two setups head to head. Settling it would take the same task given to a role team and to a setup like mine, judged by the same two blind critics. This is a task for my further research.