AI Context Benchmark: 36% Cheaper, 38% Faster in Claude Code
We ran the same 20 tasks 60 times in Claude Code to measure what structured context is worth in cost, time and quality.
The verdict: in our August 2026 benchmark, giving Claude Code structured context through Unabyss made the same work 36% cheaper and 38% faster than connected tools alone, cut follow-ups by 60%, and raised the quality of accepted output from 7.7 to 8.9 out of 10. We ran the same 20 tasks 60 times and changed one thing: whether the assistant had our context structured and waiting, or had to go assemble it.
The bigger surprise was the middle option. A carefully hand-written context file, the sensible fix most people reach for, barely moved cost, time or quality at all. Here's why, and what we think it means.
What did we actually test?
We tested one variable at three settings. Every run used Claude Code with the same ten MCP servers connected: Apollo, GitHub, Gmail, Google Calendar, Google Drive, Linear, Neon, Slack, Trigify and Vercel. The only thing that changed between arms was what the assistant knew when the session opened.
Arm A is the baseline: tools connected, no context layer. The assistant can reach your data but knows nothing about you at the start. Arm B adds a hand-written CLAUDE.md holding role, company, positioning, team and writing voice, the sensible answer if you've never seen a context layer. Arm C adds Unabyss, a context layer synced from LinkedIn, the website, X, Slack, GitHub, Linear, Gmail, Drive and Calendar. Every task ran once per setup, in a fresh session with no memory of anything before it, with prompts pasted in identically across all three.
Why does AI context need measuring at all?
Because connecting tools solves reach, not understanding. Wire up enough MCP servers and an agent can touch almost everything about your work: mail, calendar, tickets, repos, docs. What it can't do is start from an understanding of any of it. Every session opens cold, and that context has to come from somewhere.
Usually it comes from one of three places. You supply it yourself, which costs your time. The agent goes hunting across ten tools, which costs tokens and minutes. Or it proceeds without it and hands you something generically right and specifically wrong. The test measures what it's worth to have that context structured and ready instead of assembled on the spot.
Does structured context make AI cheaper, faster and better?
Yes: against the baseline, having context ready cut cost by 36% and time by 38%. Follow-ups, meaning every extra reply needed to reach a usable result, dropped 60%, from 1.5 rounds of correction per task to 0.6. Quality of the output we finally accepted rose from 7.7 to 8.9 out of 10.
But the interesting comparison isn't Unabyss against nothing. It's Unabyss against the file. A carefully written CLAUDE.md is a real improvement over nothing: it cut follow-ups by 37%. But it barely touched cost or time, and it added almost nothing to quality: 7.84 against the baseline's 7.74. A static file tells the assistant what's true in general. It can't tell it what happened yesterday. Knowing about someone isn't the same as knowing what they're working on, and that gap is where most of the value turned out to live.
Where does context pay off, and where doesn't it?
Context pays off almost entirely on work that depends on knowing things, and almost not at all on work that doesn't. That split is the real finding, and it's cleaner than we expected.
Take the control. T1 asked for a Python script to deduplicate a CSV, a task that needs no context whatsoever. All three arms landed within two cents and three seconds of each other. That's the control working: the differences everywhere else come from context, not from having more tools switched on. The gap opens on tasks that need grounding: planning a marketing campaign, briefing for a call, a daily update, working out what to do tomorrow. T9, a month-long marketing plan, cost 0.55 with Unabyss.
The biggest quality jumps all share a shape: they're questions about your own situation, where a generically competent answer is worthless. "Have I discussed our current ICP with Filip, and where?" went 7 to 10. "Give me a daily update and my priorities for tomorrow" went 7 to 10; without context it's a productivity lecture, with it a ranked list of what's actually in flight. And five tasks came out level or slightly behind, because where a task is self-contained, context is dead weight. That's the honest read: the value is concentrated exactly where you'd expect it, and nowhere else.
Is one benchmark enough to trust?
No, and we didn't treat it as such. A single run wobbles, so we re-ran three of the tasks a second time across all three setups, nine extra runs. Individual figures moved, in one case by half, but the ordering held: Unabyss produced the best output in all six repeated task-runs and the lowest cost in five of six, with eight of nine quality scores landing within a point of the first round. Treat the percentages as approximate and the direction as settled.
The full report has every task, every cost, every score, the scoring scale and the method, down to how the CLAUDE.md was written and when. If you want to poke holes in it, that's the point of publishing it.
Download the full benchmark report (PDF)
What does this mean for your day-to-day work with AI?
It means the slow part of working with AI is the setup around each answer, and that's the part context removes. The model writes in seconds. The time goes into three recurring costs: re-explaining who you are at the start of a session, correcting answers that came back generic, and the back-and-forth of feeding in, one message at a time, what it needed from the start ("no, this is for the enterprise plan", "remember it's B2B"). The follow-up count in this benchmark is that third cost, measured: 1.5 rounds per task without context, 0.6 with it.
Three practical lessons come out of the data:
- Load less, but load what's current. More context isn't automatically better. In the "Lost in the Middle" study by Nelson Liu and colleagues at Stanford, models used information at the start or end of a long input far more reliably than information buried in the middle (Liu et al., TACL 2024). Every stale memory and unused tool definition also rides along on every request and gets billed; the MCP context tax covers that side.
- Freshness matters more than detail. Arm B's file was carefully written and still barely moved quality, because it described the business in general rather than this week. Context that's out of date produces confident answers aimed at the wrong target. How to Take Care of Your AI Context covers how to keep it current.
- Don't expect gains on self-contained work. A CSV script or a regex doesn't get better because the model knows your ICP. Context pays on planning, briefing, writing in your voice and questions about your own work.
Keeping one memory tidy in one tool is manageable. Keeping ChatGPT, Claude and Cursor tidy by hand, each with its own separate memory drifting on its own schedule, is the part that doesn't scale. That's the job Unabyss does: it builds your personal context from the apps where it already lives, keeps it current, and serves the same version to every tool over MCP. For how it compares with other options, see the best AI memory tools for individuals.
