SalesSales Research Analyst
Get financials on a company
Find Hightouch on Crunchbase and return their total funding, last round type, and a one-line description.
✓ 2 tool calls, $0.030. Cheapest pass — straight to the funding number.
Benchmarks
G
zhipu
GLM 5.2
The cheap open-weights contender — 3/5 for $0.52, tripped up only by JSON hygiene.
3/5
benchmarks passed
$0.521
spent in total
16
tool calls
What it is.
Zhipu AI's GLM 5.2, run through OpenRouter. A 1M-context open-weights model from the Beijing lab; the successor to the GLM 5.1 that also appears in this matrix.
What it does well.
- Cheapest way to clear three benchmarks: $0.52 total.
- Solid tool use — clean passes on Crunchbase funding, CX complaint mining, and Apollo pricing extraction, with tight 2-4 tool-call loops.
- Native tool-calling worked out of the box through OpenRouter; no text-format tool-call leakage like the Nemotron family.
How it broke.
- Final-JSON discipline is the weak spot. On the coding task it truncated the answer mid-string — the `code` field was cut off before the closing brace, so the object never parsed.
- On the Reddit task it produced a final message that simply wasn't valid JSON.
- Both failures were about *emitting* clean, complete JSON — not about gathering the right data, which it generally did.
Results by agent
Five real jobs.
Passed
MarketingMarketing Research Analyst
Find what customers are recommending on Reddit
Read a real r/sales thread on enrichment tools and rank the top 5 by how many people recommended them.
✗ 3 tool calls, $0.119. Read the thread but the final message wasn't parseable JSON.
CXCustomer Insights Analyst
Find what customers are complaining about
Pull Google Shopping reviews for AirPods Pro 2 (USB-C) and return the top 3 recurring complaints, with verbatim quotes.
✓ 2 tool calls, $0.112. Clean complaint extraction with quotes.
CodingSenior Engineer Assistant
Read API docs and write working code
Read Stripe's official docs and write a real, working webhook-verification function in TypeScript.
✗ 5 tool calls, $0.202. Truncated mid-string — the code field was cut off before it closed.
Web ScrapingSales Outreach Specialist
Scrape a competitor's pricing page
Scrape Apollo.io's pricing page and return every tier (Free, Basic, Professional, Organization) with name, price, and top 3 features.
✓ 4 tool calls, $0.058. All four Apollo tiers extracted.