AI agents · three studies, one story · ~10 min read

One agent or a team of agents? The answer isn't what you'd guess.

Give an LLM a task and it handles the whole thing itself — plans, writes, checks, fixes — in one long session. Splitting a complete job across specialized agents — a planner, a coder, a tester, a critic — feels like it should work better than one agent, and that intuition powers a lot of AI products right now. But three recent studies put it to the test, and the honest answer is: only sometimes. The interesting part is finding out exactly when.

Quick vocabulary check. A single-agent system (SAS) is one model doing the whole job in one long session. A multi-agent system (MAS) is several agents — planner, coder, tester, critic — splitting that job and passing messages to each other. This post is about figuring out when splitting the job actually pays off.

TL;DR for the impatient

The team advantage is evaporating. As base models get better, agent teams win by less — often by nothing — while still burning 5–200× more tokens.
Teams only pay off when the task splits. Parallelizable work gained up to +80%; sequential planning got worse under every team layout tested.
Match the compute and most wins vanish. Give one agent the same thinking budget as the whole team, and the solo agent usually comes out on top.
The winning move is dynamic. Try solo first, verify, escalate only on failure — the best accuracy in the study at ~12% of the team's cost.

Part 1 · The price of teamwork

AI teams used to win big. The gap is almost gone.

The first study, from a team at the University of Illinois (Gao et al., 2025), runs 9 popular multi-agent frameworks against a plain single agent on the same tasks. Most of these frameworks were designed back when base models needed the extra hands — planning was brittle, tool calls failed, context windows were short. A lot of that isn't true anymore, which raises the obvious question the paper goes after: are these frameworks still earning their keep? To find out, they reran the original benchmarks — same framework, same dataset — on a current model.

Figure 1 · The MAS advantage is evaporating as base models improve

Same framework, same dataset — re-run with a modern base model. The gap that motivated multi-agent designs shrinks from +8–16 points to under 3.

How to read it: each row is two dots — blue for solo, green for team — and the bar between them is the team's lead. Watch that bar collapse in every "Gemini-2.0-Flash" row.

Gao et al., Table 2. The first row of each pair is what the original paper reported back in the GPT-3.5 era; the second re-runs both setups on a modern model (Gemini-2.0-Flash). On the math benchmark the team's lead shrinks to 0.8 points — basically a rounding error.

On roughly 80% of the test questions, the team and the solo agent land on the same result — both right, or both wrong. The team just costs more to get there. Figure 2 shows how much more.

Figure 2 · What all that chatter actually costs

Token multipliers, MAS ÷ SAS, per dataset (log scale). Input tokens: 1.2× to 220×.

How to read it: bar length = how many times more tokens the team burns for the same task. The scale is logarithmic, so each gridline is a big jump — 220× means one team run costs as much as 220 solo runs' worth of input.

Gao et al., Table 3. That 220× on AIME (competition math) comes from agents re-reading the entire conversation on every debate round — and these numbers already assume optimistic caching. Worse: on three of these benchmarks (GSM8K, DS-1000, BigCodeBench), the team you're paying extra for is actually less accurate than the solo agent.

The authors also traced failures back to three recurring patterns. Two of the three, it turns out, are caused by the collaboration itself — not by any single agent being weak.

The weak link

One agent on the team simply isn't good enough for its job. The good news: this one has a surgical fix. The authors' tracing method pinpointed exactly which teammate in a three-agent math debate to upgrade — no guesswork, and it even knew which agent could safely be downgraded to save money.

Too many cooks

In one framework, the "analyst" and "tester" agents piled so many unnecessary corner cases onto the coder agent that it started failing tasks a solo coder handles fine. The team didn't catch an error — it created one.

The telephone game

In a math debate, a correct answer appeared early — then got dropped when one agent summarized the discussion for the next. No one downstream could ever recover it. A small omission at hand-off snowballed into an unrecoverable failure.

Their proposed fix: stop picking one architecture, and decide per request instead. A cheap model looks at each incoming question and routes the easy ones to the solo agent, the hard ones to the team. The cascade variant is simpler still — always try the solo agent first, check the answer, and call in the team only when that check fails. Figure 3 tests that cascade against both of the pure setups it's built from.

Figure 3 · The cascade beats both of its parents

Try SAS, check the answer, escalate to MAS only on failure. Best accuracy on 6 of 7 datasets (ties MAS on AIME) at a fraction of MAS cost.

How to read it: top panel — taller is more accurate, and the amber cascade bar wins or ties in every group. Bottom panel — shorter is cheaper (log scale). The magic is amber sitting near blue (cheap) on cost while beating green (expensive) on accuracy.

Gao et al., Table 4. On grade-school math (GSM8K) the cascade beats both pure setups on accuracy while costing 12% of what the always-on team costs — that's where the paper's "88% cost reduction" headline lives. The catch: cascades need a cheap way to check an answer (does the code pass its tests? does the number match?). Open-ended tasks like "write me a report" don't have one yet.

So the real question isn't "is a team better than one agent" — it's "which requests are worth calling a meeting for," and that turns out to be something you can answer cheaply, per request. Of course, this study used off-the-shelf frameworks, wired up however their authors happened to wire them. Maybe a carefully designed team structure does better. That's what the second study checks.

Part 2 · Does team structure matter?

Five team structures, six tasks, no winner

The second study (Kim et al., 2025) changes exactly one thing at a time: how the team is put together. They ran 260 setups — five team layouts, six benchmarks, three model families — while keeping everything else the same: same prompts, same tools, same thinking budget. Here are the five layouts:

Single-agent

One model does everything, step by step.

Independent

Everyone works alone; answers get stapled together at the end. Nobody checks anybody.

Centralized

A manager hands out subtasks and reviews the results before signing off.

Decentralized

No manager — the agents debate each other until they agree.

Hybrid

A manager, plus some hallway conversation between teammates.

Run the same task through all five team layouts, and here's the surprising part: which one wins has almost nothing to do with the layout itself. It depends entirely on what kind of task you gave it:

Figure 4 · There's no best team — only a right team for the task

Performance change vs going solo. Averaged across everything, teams change performance by −0.3% — but the swings range from +81% to −70%, so the average tells you nothing.

How to read it: blue cell = the team beat the solo agent, red = it lost. Scan row by row: the color is decided by the task (row), not the team layout (column) — no column is blue all the way down.

Kim et al., Figure 2. Financial analysis — a task that splits cleanly into independent lookups — gains up to +80.8% from teamwork. Step-by-step planning (PlanCraft) loses 39–70% under every team structure, because you can't parallelize a recipe. And on SWE-bench, where the solo agent is already strong, every team makes things slightly worse. ("n/a" cells appear only in the paper's figure, not its text.)

That's the accuracy picture. The same 260 runs also let the authors trace something accuracy alone hides: once one agent makes a mistake, how much worse does it get by the time the team hands back a final answer?

Figure 5 · One mistake, how many times does it multiply?

How much one agent's mistake gets multiplied on its way to the final answer (whiskers: 95% confidence interval).

How to read it: longer bar = a single mistake does more damage by the end. 1× would mean "no worse than a solo agent" — every team layout is above it, but a manager (centralized) contains the damage best.

Kim et al. When nobody checks anybody, one agent's mistake gets amplified 17.2×; adding a manager cuts that to 4.4×. Then the twist: in the full statistical analysis, error amplification turns out to explain almost none of the performance differences (p = 0.658). The real killers are wasted effort and coordination overhead — all those meetings. The obvious villain is a red herring.

There's no general answer to "do AI teams help" — it comes down to whether the task splits cleanly, whether anyone can check the work along the way, and how strong the solo agent already is. Get those three right and the outcome is predictable. But both studies so far share the same blind spot: the team always got to think more than the solo agent it was measured against, just by making more calls. The third study finally closes that gap.

Part 3 · The fair fight

Give one agent the same thinking budget, and it usually wins

The last study (Tran & Kiela, 2026) points at a flaw running through every comparison so far: a team makes more model calls, so it quietly gets more total thinking time than the solo agent it's being measured against. It's not obvious the team is smarter — it might just be billing more hours. Tran and Kiela test this directly by giving both sides the same total thinking budget, with the team splitting its share across its members, on hard multi-step questions.

The theory in one line

Y is the answer, C is the full context, and M is the summary one agent hands to the next — written as M = g(C) because it's some function g of the context, never more than that.

I(Y; C) ≥ I(Y; M)

In plain English: a summary can't tell you anything about the right answer that wasn't already in the thing it was summarized from. So if a solo agent and a team get the same amount of thinking time, the solo agent — reading the full context directly — starts a step ahead. The team is stuck working from a compressed version of it.

That's the theory. Figure 6 is the actual experiment: five team variants against one solo agent, thinking budgets held equal from 100 tokens up to 10,000.

Figure 6 · Same budget, same task: the solo agent pulls ahead

Accuracy vs thinking budget. Blue: the single agent. Gray: all five team variants. Hover the lines for exact numbers.

How to read it: higher line = better at that budget. Left chart: the blue solo line sits on top from 500 tokens onward. Right chart: gray above blue in the shaded zone looks like a team win — the caption explains why it's an accounting illusion.

Tran & Kiela, Table 1. Left is the clean result: the solo agent beats every team variant at every budget above 100 tokens. Right looks like a counterexample — teams "win" at tiny budgets — but the paper's appendix explains the trick: when you request a tiny budget, a team's many separate calls sneak in more actual thinking than one call does (shaded region). The win is an accounting artifact. Also notice how fast returns flatten: 1k–2k tokens of thinking already gets you close to the 10k result.

The most convincing experiment here is the one where the authors try to break their own theory. The solo agent's information edge only holds if it gets a clean, trustworthy context to read. So they corrupted it — replacing a growing fraction of the context with random tokens. If team structure genuinely helps filter out noise, this is where teams should start winning. And they do:

Figure 7 · Make the context messy enough, and the team finally wins

Qwen3-30B · multi-hop QA · fixed 1k thinking budget. "Substitution" swaps a fraction of the context for random tokens — deleting real clues and planting fake ones.

How to read it: follow the two lines left to right as the context gets noisier. Blue (solo) slides steadily down; green (team) stays flat. The crossing point is the moment teamwork genuinely starts earning its keep.

Tran & Kiela, Table 13. With a clean context the solo agent wins comfortably. Crank the noise to 70% and the solo agent collapses (0.260 → 0.200) while the team barely flinches (0.229 → 0.225) — and takes the lead, exactly as the theory predicted. Milder corruption doesn't flip the ranking; only the nastiest regime does. The open question: does messy real-world data actually look like that regime?

Once compute is actually matched, most of the reported team advantage disappears. What's left shows up exactly where the theory predicts: contexts too noisy for a single agent to read cleanly. So treat any "our multi-agent system beats a single agent" claim with some suspicion until you know whether the compute was matched too.

So what should you build?

Don't pick a team. Pick per task.

Start with one agent. Across all three studies, that's the safer default — usually just as accurate, and far cheaper. Bring in a team when there's genuinely a lot to deal with: the work splits into many independent pieces (Kim et al.'s decomposable tasks), or the input is so large and noisy that one agent can't make good use of all of it (Tran and Kiela's degraded-context result). Outside of that, splitting the job just adds cost without adding accuracy.

✅ Stick with one agent when…

The task is sequential — each step depends on the last (planning, refactoring, debugging).
The whole context fits in one window and the model is already decent at the task.
There's no cheap way to check the work partway through — so a team's mistakes would go uncaught until the very end.
Latency and token cost matter. (They always do.)

🤝 Bring in a team when…

The work splits into genuinely independent chunks — parallel research, many separate lookups.
The input is huge or noisy and needs filtering before anyone can reason over it.
Each piece can be verified cheaply — tests pass, numbers match, schema validates.
Even then: try solo first, and escalate only when the check fails.

References

Single-agent or Multi-agent Systems? Why Not Both?

Gao et al., May 2025 — the cost audit from Part 1: the failure taxonomy, the router, and the cascade.

arxiv.org/pdf/2505.18286

Towards a Science of Scaling Agent Systems

Kim et al., Dec 2025 — the 260-configuration referee match from Part 2, across five team structures and six benchmarks.

arxiv.org/pdf/2512.08296

Single-Agent LLMs Outperform MAS Under Equal Thinking Token Budgets

Tran & Kiela, Apr 2026 — the fair fight from Part 3: matched thinking budgets, the telephone-game theory, and the noise crossover.

arxiv.org/pdf/2604.02460