Serial vs. Parallel AI Agents: We Benchmarked 200 Tasks Both Ways

We ran the same 200 agent tasks two ways: as one long serial conversation, and as 200 independent parallel jobs. The parallel version was cheaper, faster, and more accurate at the same time. Not a tradeoff on any axis.

This article is the full writeup: the test setup, why serial degrades, what happens at larger scale, and the practical pattern for getting the parallel shape without building infrastructure.

The test

The task is a support-ticket router. For each incoming ticket, the agent has to pick the right team and the right priority. It is a real multi-turn agent task: for one ticket, the agent makes three or four tool calls. It looks up the customer, pulls their open tickets, checks SLA status, and then writes the routing decision. Each ticket also adds roughly 1,400 tokens of history that a serial agent will carry on every subsequent call.

We ran 200 tickets three ways:

1. GPT-5 Mini in one long serial chat. The cheap model, brute-forced.
2. GPT-5.5 in one long serial chat. The frontier model, brute-forced.
3. GPT-5 Mini broken into 200 independent parallel jobs. The cheap model, architected.

Mini parallel is the cheapest row, the fastest row, and the most accurate row. Mini serial, the brute-force version of the cheap option, is the worst row: sixteen times the cost of Mini parallel, 68 minutes instead of one, and almost ten points of accuracy lost. GPT-5.5 serial held its accuracy, and paid two hundred times more than Mini parallel to do it.

Why serial degrades

Serial token cost is O(n²). Every new task resends the entire conversation until you compact it. At task 200 of the same workflow, you are processing 285,000 tokens of mostly irrelevant history on every call.

The cost is only half the problem. The model also loses track of what is current and what is stale as the conversation grows. We measured accuracy on the last ten tasks at each point in the run, and it declines as the history piles up. It declines much faster on the cheaper model. So you cannot fix the bill by dropping to a cheaper model and keeping the serial shape. You just trade money for errors.

Run the same tasks in parallel and both problems disappear at once. Each job carries only the context that task needs, so cost per task stays flat whether you run 100 or 10,000, and every task gets a fresh context window with nothing stale in it. That is why the parallel run is more accurate, not just cheaper: 99.0% against 89.4% for the same model.

Smarter architecture beats a bigger model

We ran a second test on a different workflow, a multi-step refund pipeline, to check whether architecture alone could close the gap with the frontier model. We compared GPT-5.5 as a single agent, Mini as a single agent, and Mini with an orchestrator that spawns sub-agents for each independent step.

Mini with sub-agents matched GPT-5.5 on accuracy and cost the same as Mini alone. Once the problem is decomposed, the bigger model stops buying you anything. On token spend, the gap is wider still.

At scale it gets absurd

Here is serial versus parallel on the same multi-turn workflow as the task count grows:

At ten thousand tasks, brute force is roughly nine thousand times more expensive than the architected version. This is the mechanism behind every "we burned our monthly token budget in four days" story: it is almost never that the team used AI too much. It is that the work ran in the wrong shape. We wrote about the industry-level version of this in The Supercomputer Era of AI Is Over.

The catch: parallel is an infrastructure problem

If parallel wins on every axis, why does anyone run serial? Because serial is the default shape of a chat window, and parallel is an infrastructure project. To fan 10,000 tasks out yourself you need a job queue, per-integration rate limiting, retries with backoff, state so partial failures do not duplicate work, and some way to see what ran and what failed. Most teams get three tickets into building that and go back to the long chat.

This is the actual reason we built the Cotera Native App for Snowflake. A SQL query is already a parallel fan-out: every row is an independent task with exactly the context that task needs.

SELECT
  ticket_id,
  priority,
  cotera.agents.agent('<your agent id>', ticket_body) AS routing
FROM support.open_tickets
WHERE created_at > DATEADD('day', -1, CURRENT_DATE());

Each row becomes one agent run with a fresh context. Cotera handles the queueing, retries, and rate limits, and the results come back as a column. You get the cheapest, fastest, most accurate row of the benchmark table as the default shape, not as an infrastructure project. The full patterns are in Run AI Agents from Snowflake with Cotera.

How to tell which shape your work needs

Not everything should be parallel. The test is dependency:

• Tasks are independent of each other (enrich these leads, triage these tickets, analyze these accounts): parallel, always. This covers most row-shaped operational work.
• Each step depends on the last (a negotiation, a debugging session, an exploratory analysis): serial is correct, and a bigger context window earns its cost.
• A mix (a pipeline with independent steps inside it): decompose. An orchestrator with sub-agents gets frontier-model accuracy at small-model cost, per the refund-pipeline test above.

The one-line version: if you can write the work as rows in a table, it should run in parallel, and it is currently costing you somewhere between 17x and 8,700x to run it as a conversation.

FAQ

Why is the parallel run more accurate, not just cheaper? Every task gets a fresh context containing only what it needs. The serial run makes each decision while attending over hundreds of thousands of tokens of stale history, and measured accuracy declines as that history grows.

Does this mean small models are as good as frontier models? For decomposed, well-scoped tasks, our data says yes: Mini parallel beat GPT-5.5 serial on this benchmark, and Mini with sub-agents matched GPT-5.5 on the refund pipeline. For long open-ended reasoning in one context, bigger models still win.

What about prompt caching? Doesn't that fix serial cost? Caching discounts re-read tokens; it does not eliminate them, and it does nothing for the accuracy degradation. The O(n²) shape remains, just with a smaller constant.

How do I run this pattern from Snowflake? Install the Cotera Native App from the Snowflake Marketplace, then call a predefined agent per row from SQL. Setup takes about ten minutes with the install guide.

Try These Agents

Browse the full catalog on Solutions, or start with these:

• Lead Enricher & Qualifier — Qualify inbound leads from CRM and warehouse data with structured scoring
• Company Growth Analyzer — Research company growth, traffic trends, and competitive landscape from a domain
• Market Intelligence Agent — Monitor competitors, hiring patterns, and market moves across your target accounts