Vercel improved its data science agent by replacing a chain of scoped agents with one agent that manages its own state in a sandboxed file system.
2
A recurring job turns common queries into roughly 100 skills, giving each new run company-specific context instead of starting from only a system prompt and semantic layer.
3
Andrew Qu argues that narrow agents with deep company knowledge can be more useful than general-purpose agents, and says Vercel has roughly 20 such agents in use.
Summary
Andrew Qu describes Vercel's attempt to put an agent on every desk, starting with a data science agent that could answer questions about customers, products, sales, and metrics. The first version used a large prompt containing the Snowflake schema. The second split the work across planning, execution, and reporting agents. The third gave one agent control of its own state, but early users found it poor despite passing 30% of internal evaluations. The breakthrough came from copying Claude Code's simple file-system model. Vercel put the semantic layer in a sandbox and gave the agent tools to list, read, write, and run bash commands. The evaluation score doubled. Vercel then distilled recurring queries into roughly 100 skills and built Eve, an agent framework based on file-system conventions. Qu's practical conclusion is that agents need specific knowledge about a company's data and work. Vercel now uses roughly 20 agents across functions such as marketing, legal, and data science.
Vercel started by looking for work that people hated repeating
Andrew Qu and Vercel's CTO wondered whether an agent could sit on every desk, extending agent use beyond coding and technical work. Qu asked people in marketing, sales, finance, and legal what they disliked most about their jobs. The strongest use case came from the data team. Vercel was growing faster than the lean team could keep up with its customer, analytics, metrics, and sales data. When someone had a question, a data scientist had to stop their work, write and run a query, analyze the result, and return with a recommendation. Qu describes this as a serious productivity problem because the data team spent too much time answering one-off questions.
The first agent was a large prompt with the Snowflake schema pasted into it
The initial version followed the simplest possible approach. Qu took a dump of the Snowflake schema and pasted it into a system prompt with the user's question. When the model produced SQL, he copied it out and ran it himself. The purpose was to test whether current models could write valid SQL from a useful structure. The result gave the team some confidence, although Qu said the models were not very good yet. Better context and guardrails seemed to improve their behavior. This version did not complete the full workflow, since a person still executed the SQL and handled the result.
A chain of narrow agents completed more of the data workflow
Qu and Vercel's VP of data broke the data scientist's work into separate stages. D0 used a query agent, a planning agent, an SQL agent, and a reporting agent. Each agent had a focused system prompt and tools limited to its job. The planning agent, for example, could read entity YAML and search schemas before passing its work onward. This design completed the loop from question to answer instead of asking Qu to copy and run SQL manually. It still created problems because each stage received only a summary and a small snippet from the previous stage. The next agent could not fully inspect the earlier reasoning or recover easily from an error.
One agent with its own state could revisit mistakes
The third design put the work inside one large agent with a maximum of 100 steps. The agent could plan, build, execute, and report while managing its own state. Qu's team wanted it to look back at what it had done and work out which steps led to the current situation. When a query or join failed, the agent could explore more information, read additional files, and correct its approach. This gave the team confidence, so they released it to a small group of trusted Vercel users. The response was immediate and negative. Although the system passed 30% of their evaluations, real users asked questions the team had not anticipated, and manually adding scenarios did not seem scalable.
The file system was the main improvement over Vercel's hand-built tools
Claude Code and Opus 4.5 changed Qu's view of the problem. Claude Code answered most of Vercel's questions without missing a beat, so the team examined what made it different. The important part was a simple file-system interface with tools such as listing files, reading files, and running bash. The model could explore and write work where it needed to instead of following a tightly prescribed tool sequence. Vercel rebuilt the data agent in a sandbox, dumped the full semantic layer into it, and let the agent read, write, and execute files. A few Vercel-specific tools were added on top. Qu says the evaluation score then doubled.
Repeated queries became skills that supplied accumulated context
Once Vercel opened the agent to more users, it received thousands of queries each day about customer metrics, sales metrics, npm downloads, billing information, and other data. Many requests had the same shape. A recurring job now takes recent queries and distills them into skills. Qu says Vercel has roughly 100 skills covering tasks from aggregation to looking up information about particular people. A new run otherwise starts with almost no context beyond the semantic layer and system prompt. A skills folder gives it knowledge from previous work before it starts. Qu presents this as a way to turn repeated agent behavior into reusable company-specific knowledge.
Eve packages file-system conventions into an agent framework
Vercel built Eve after seeing people repeatedly fork D0 and rebuild the same improvements. Qu compares the approach with Next.js, where files in expected folders define how a web application is assembled. In Eve, developers create folders for skills, tools, and channels, then add the relevant files. The framework supplies a runtime with durability, isolated execution, model connections, and channels. It can use open-source adapters for Postgres, OpenAI's Responses API, Docker, and other connectors, or Vercel products such as Vercel Workflows, Vercel Sandbox, and Vercel Connect. Vercel also rewrote D0 in Eve, leaving a structure made from system instructions, skills, and tools.
Company knowledge makes narrow agents useful in real work
Qu says Vercel tested well-funded vertical agents that connected to Snowflake, but found that a useful agent needs more than database access. Vercel's knowledge of websites and web properties affects when a query should run and how different facts relate. General products can be good to try, but Qu recommends building an agent that contains as much company-specific knowledge as possible. Vercel has roughly 20 agents in use, including agents for marketing retrospectives, deciding whom to contact, reviewing the first version of a legal contract negotiation, and answering data questions. He says the data team now has more time to improve Snowflake, add missing sources, and fill gaps in the data.
"We thought we were cooking. We thought this was, you know, nailing 30% of our evals, but we couldn't have anticipated some of the questions that were being asked."07:18
Who should watch
You are building an internal agent and want to compare a mega-prompt design, a chain of agents, and a single stateful agent.
Your agent performs repeated company-specific tasks and needs a way to turn past runs into reusable skills.
You are deciding whether to use a general vertical product or build an agent around your own data, workflows, and terminology.
