Enterprise agent teams should choose frameworks by observability, setup cost, and support for models and tools.
2
A single model with a small set of clearly specified tools is often a better starting point than a multi-agent system.
3
Evaluation should track tool calls, parameters, tool results, intermediate reasoning, and final answers, with different failure fixes for different failure rates.
Summary
Shaan Desai explains how Cohere approaches enterprise LLM agents, from framework selection through deployment. He recommends judging frameworks by observability, setup cost, and support, then starting with a single agent and a small set of well-defined tools. Tool descriptions should use clear examples and simple input types, while instructions and conversation history should stay manageable. Multi-agent systems can work when a router has precise instructions and each sub-agent handles a narrow task. Desai treats human approval as part of the design for actions such as sending email or processing sensitive outputs. Cohere evaluates agents with golden sets that include expected tool calls, parameters, outputs, and final responses. The response to failures depends on their rate and type, ranging from prompt changes to annotation data and fine-tuning. He closes with North, Cohere's packaged environment for building agents connected to enterprise applications.
Framework choice depends on observability, setup cost, and support
Desai says Cohere assesses frameworks using three criteria: whether they are easy to debug and fix, how quickly a team can build and iterate, and whether the framework is documented and supports the needed models, tools, and functions. For large-scale enterprise agents, he recommends going native or using LangGraph because these systems need high observability. For quick tests and proofs of concept, he points to CrewAI and AutoGen because they have low setup effort and make it easy to combine prebuilt agents and tools. He presents these recommendations as temporary because framework support is still changing.
A single agent with carefully designed tools is the right starting point
Desai recommends starting with one LLM and a handful of tools. In one client project, APIs accepted 10 to 15 parameters, which made reliable tool calls difficult. The team improved performance by simplifying the APIs, writing sharp descriptions with examples, and converting nested dictionaries into simpler types such as lists, strings, or floats. He also says short, direct instruction lists work better than long instructions that confuse the model and can induce hallucinations.
Long conversation histories can cause hallucinations
Desai says conversations that run beyond about 20 turns can induce hallucinations across many models and frameworks. His recommendation is to cache the history and retrieve only the parts that matter for a new query. This keeps the agent from carrying every previous exchange into each decision. The goal is to preserve relevant context while avoiding a long stream of back-and-forth messages that can reduce performance.
Multi-agent systems need strict routing and narrow sub-agents
In a multi-agent setup, a routing model chooses which sub-agent should handle a request. Desai says the router needs clear tool descriptions and precise routing instructions that cover edge cases, especially when work moves between several agents. Without that guidance, the model may keep attempting actions that are not the best route to an answer. Each sub-agent should handle an independent, specific task with a small set of tools and return its result.
Human approval should guard actions and sensitive outputs
Desai treats human-in-the-loop controls as necessary for enterprise applications. A Gmail agent, for example, can ask for permission immediately before sending an email instead of sending it automatically. Teams can define rules that trigger approval under specific conditions. Approval may happen before a tool call, or after execution when the returned information contains material that should not be processed without review. The same pattern applies to HR support and financial analysis agents.
Agent evaluation must include the intermediate steps
A useful evaluation checks more than the final answer. The model must choose the right tool at the right time, pass accurate parameters, reason over returned results, and recover when something goes wrong. Cohere builds a golden set containing user queries, expected function calls, expected parameter inputs, expected tool outputs, and expected final responses. Running this corpus through the agent framework helps identify where a failure occurs and makes debugging more direct.
The fix should match the failure rate and failure type
Desai describes three response levels. For low-severity problems or a low failure rate, improving prompts, tool API specifications, and tool inputs may close the remaining performance gap. When a model hallucinates on a particular task or a tool failure occurs around the 10 to 20 percent range, a targeted annotation dataset can help. For high failure rates, especially when APIs are hard to call or names are difficult to distinguish, Cohere uses a larger synthetic corpus and fine-tuning.
North packages agent connections and debugging into one deployment
Cohere's North product brings these lessons into a single-container deployment. It can connect to retrieval-augmented generation, vector databases, search, and applications such as Gmail, Outlook, Drive, and Slack. In the demo, North connects to Gmail, Salesforce, and Google Drive, retrieves Salesforce information, shows the tools called and their outputs, and exposes the reasoning process for debugging. The product can also use recent conversations and revise a tool call when a different application is more appropriate.
"The nice thing about doing this and building this evaluation set is that we can run this large corpus of evaluations through our agentic framework and assess any critical points of failure."13:08
Who should watch
You are choosing between agent frameworks and need a practical way to compare their debugging, iteration, and integration costs.
Your agent can call APIs but fails because tools have complex parameters, unclear descriptions, or long conversation context.
You need an evaluation and approval design for agents that can take actions in enterprise systems.
