# Find what customers are complaining about

> We asked an AI to find what people are complaining about on a real product page. 17 of 24 models could do it.

Source: https://cotera.co/benchmarks/agents/cx-google-shopping-complaints

---

**The task:** Pull Google Shopping reviews for AirPods Pro 2 (USB-C) and return the top 3 recurring complaints, with verbatim quotes.

**Tools:** Google Search, Google Shopping

## Why this matters

Customer experience teams need to know what's actually breaking for real customers before support tickets start piling up. The data is sitting right there on the product page (Google Shopping reviews are the public version of NPS comments). The job is to pull the reviews, group the recurring complaints, and quote them verbatim so a CX lead can prioritize against real signal.

## What we saw

We asked a customer insights analyst agent to pull reviews for AirPods Pro 2 (USB-C) from Google Shopping and return the three most recurring complaints, with verbatim customer quotes. The rubric checks that it found the right product, returned exactly three complaints, and pulled the quotes verbatim from real reviews.

12 models passed. The four that failed were interesting: MiniMax M3 thought it through in a private scratchpad and then never wrote the answer, Llama 4 didn't attempt to read the reviews at all, and Grok and Qwen retrieved the reviews and then closed the run without writing anything.

## What worked
- DeepSeek hit it for $0.053 with four tool calls (one product search, one review pull, three complaints with verbatim quotes). This is the perfect pass.
- Most of the passes returned the same three top complaints: case rattle, battery degradation, and connection drops. The signal is sitting right there in the reviews, and the models that bothered to look at them all found it.

## What broke
- MiniMax M3 listed the complaints in a private thinking block and then closed the run without ever writing the answer. The reasoning was there, but the output was not.
- Llama 4 didn't pull any reviews. It tried to answer from training memory and returned the wrong shape.
- Grok and Qwen both read the reviews and then never wrote the final answer. Grok did nine tool calls before closing the run with no output, and Qwen did ten.

## Agent system prompt

```markdown
You are a customer insights analyst for a CX team. Mine product reviews for recurring pain points. ALWAYS pull real reviews with the tools; never summarize from memory. Group reviewer complaints by recurrence and cite verbatim snippets.
```

## Grading rubric
- Did it identify the actual AirPods product?
- Did it return exactly 3 complaints?
- Are the quotes pulled verbatim from real reviews, not paraphrased from memory?

