36% Cheaper, 38% Faster: What Structured AI Context Is Worth
A benchmark on cost, speed, and quality with structured context ready at session start
We ran the same 20 tasks 60 times in Claude Code, changing one thing: whether the assistant had our context structured and waiting, or had to go assemble it. With context ready, the same work came out 36% cheaper, 38% faster, needed 60% fewer follow-ups, and scored higher on quality - 8.9 vs 7.7 out of 10.
The bigger surprise was the middle option. A carefully hand-written context file - the sensible fix most people reach for - barely moved cost, time, or quality at all. Here's why, and what we think it means.
What did we actually test?
One variable, three settings. Every run used Claude Code with the same ten MCP servers connected - Apollo, GitHub, Gmail, Google Calendar, Google Drive, Linear, Neon, Slack, Trigify, and Vercel. The only thing that changed between arms was what the assistant knew when the session opened.
Arm A is the baseline: tools connected, no context layer. The assistant can reach your data but knows nothing about you at the start. Arm B adds a hand-written CLAUDE.md holding role, company, positioning, team, and writing voice - the sensible answer if you've never seen a context layer. Arm C adds Unabyss, a context layer synced from LinkedIn, the website, X, Slack, GitHub, Linear, Gmail, Drive, and Calendar. Every task ran once per setup, in a fresh session with no memory of anything before it, with prompts pasted in identically across all three.
Why does context need measuring at all?
Because connecting tools solves reach, not understanding. Wire up enough MCP servers and an agent can touch almost everything about your work - mail, calendar, tickets, repos, docs. What it can't do is start from an understanding of any of it. Every session opens cold, and that context has to come from somewhere.
Usually it comes from one of three places. You supply it yourself, which costs your time. The agent goes hunting across ten tools, which costs tokens and minutes. Or it proceeds without it and hands you something generically right and specifically wrong. The test measures what it's worth to have that context structured and ready instead of assembled on the spot.
What did the numbers show?
Against the baseline, having context ready cut cost by 36% and time by 38%. Follow-ups - every extra reply needed to reach a usable result - dropped 60%, from 1.5 rounds of correction per task to 0.6. Quality of the output we finally accepted rose from 7.7 to 8.9 out of 10.
But the interesting comparison isn't Unabyss against nothing. It's Unabyss against the file. A carefully written CLAUDE.md is a real improvement over nothing - it cut follow-ups by 37%. But it barely touched cost or time, and it added almost nothing to quality: 7.84 against the baseline's 7.74. A static file tells the assistant what's true in general. It can't tell it what happened yesterday. Knowing about someone isn't the same as knowing what they're working on, and that gap is where most of the value turned out to live.
Where does context actually pay off - and where doesn't it?
Almost entirely on work that depends on knowing things, and almost not at all on work that doesn't. That split is the real finding, and it's cleaner than we expected.
Take the control. T1 asked for a Python script to deduplicate a CSV - a task that needs no context whatsoever. All three arms landed within two cents and three seconds of each other. That's the control working: the differences everywhere else come from context, not from having more tools switched on. The gap opens on tasks that need grounding - planning a marketing campaign, briefing for a call, a daily update, working out what to do tomorrow. T9, a month-long marketing plan, cost 0.55 with Unabyss.
The biggest quality jumps all share a shape: they're questions about your own situation, where a generically competent answer is worthless. "Have I discussed our current ICP with Filip, and where?" went 7 to 10. "Give me a daily update and my priorities for tomorrow" went 7 to 10 - without context it's a productivity lecture, with it a ranked list of what's actually in flight. And five tasks came out level or slightly behind, because where a task is self-contained, context is dead weight. That's the honest read: the value is concentrated exactly where you'd expect it, and nowhere else.
Is one benchmark enough to trust?
No, and we didn't treat it as such. A single run wobbles, so we re-ran three of the tasks a second time across all three setups - nine extra runs. Individual figures moved, in one case by half, but the ordering held: Unabyss produced the best output in all six repeated task-runs and the lowest cost in five of six, with eight of nine quality scores landing within a point of the first round. Treat the percentages as approximate and the direction as settled.
The full report has every task, every cost, every score, the scoring scale, and the method - down to how the CLAUDE.md was written and when. If you want to poke holes in it, that's the point of publishing it.
