Browser agents are being held back by missing harnesses, tools, and dependable infrastructure rather than by the models alone.
2
Reliable browser agents combine multimodal models, code generation, reusable skills and memory, compressed page context, and consistent browser environments.
3
The web needs agent-friendly accessibility, authentication, and trust systems before browser agents can work safely at enterprise scale.
Summary
Paul Klein IV argues that browser agents are ready for more work than current products deliver. Recent models can handle longer tasks and use interfaces, but teams still need to build the surrounding harness: tools, memory, skills, code execution, and context compression. Production systems often combine visual interaction with generated code that intercepts or replays network requests. They also need browser infrastructure that produces the same layout and inputs on every run. Klein then turns to changes required from websites, including accessibility trees, Web MCP, agent-friendly sign-up and login flows, and a way to certify trusted agents. He sees the largest opportunity outside AI companies, in businesses such as logistics firms, banks, and factories that still run daily work through PHP forms. Browserbase's proposed platform bundles the runtime, sandbox, tools, models, and feedback loop for these tasks.
Browser agents have moved past the model bottleneck
Klein says the web was built for people, so pages change, consume context inefficiently, and often fail to load or present blockers. He used to maintain web automation scripts and found them painful to keep working. In his view, models were the main limitation until recently because they struggled with long-horizon tasks. That has changed through better computer-use models and reinforcement-learning environments based on real-world interfaces. He says teams should keep updating their assumptions because model capabilities change quickly. The remaining problem is the engineering around the model, especially the harness that lets an agent use tools and interact with the world.
A domain-specific harness can improve a model's results
An agent harness is the scaffolding and systems around a model. Klein includes tools such as a code interpreter, screenshots and other audio or video inputs, a browser, and subagents. He points to coding products as evidence that a custom harness can outperform the same underlying model on a domain. Factory, Claude Code, and Cursor are examples he names. The lesson is practical: a company does not need to be an AI lab to build a useful harness for its own work. Klein recommends adding a harness and measuring it against the baseline model, even though he leaves open whether custom harnesses will eventually beat models trained through reinforcement learning.
Klein describes a large gap between what computer-use models can potentially do and what reliable products currently achieve. He cites a Brockman post about asking why he is not using Codex for a task, because the task can feel outside the model's abilities. Klein says coding agents complete far more tasks than computer-use systems partly because coding has received more tooling and engineering attention. He sees non-coding work as a larger opportunity, with many real-world tasks that could benefit from computer use. His advice is to work on the missing tools and systems inside companies rather than wait for another model improvement.
Reliable browser agents combine models, skills, and code
Klein's production recipe has three parts: multimodal models, harness engineering, and reliable infrastructure. An agent can use a stronger model for a complex page and a simpler model for an easy one. It can also inspect a screen, intercept network requests, and write code to replay those requests instead of clicking through every step. He says the most reliable browser agents in production often write code alongside browser interaction. Repeated tasks should use memory and skills rather than rediscovering a site on every run. Browserbase's browser.sh publishes website skills, while WebMCP can provide existing knowledge. The harness should compress page context and expose only the tokens needed for the task.
Consistent browser infrastructure is part of agent reliability
Klein says browser environments are difficult to scale because production agents need the same inputs and outputs every time. He uses people running OpenClaw on Mac minis at home as an example of an approach that may work for an individual but does not fit thousands of customer agents. A home Mac mini can be reached over SSH and may avoid some CAPTCHA problems through a residential IP, yet Klein has not seen a SOC 2 compliant Mac mini setup at scale. He says the browser must render the same page size and layout across runs. A mobile layout on one run and a desktop layout on the next gives the agent inconsistent inputs.
Websites need agent-friendly interfaces and authentication
Klein says the web itself must change if agents are expected to use it. Browser agents increasingly read accessibility trees and labeled areas rather than consuming an entire raw DOM. Chrome's Web MCP lets a website publish MCP servers inside a page, so an agent can issue approved calls such as submitting a registration form without installing a separate MCP server. He also calls authentication a larger unresolved problem. Giving an agent a personal password is risky, while service accounts require controlled permissions. He mentions WorkOS's OffMD as a way for an agent to discover how to sign up for a site and obtain its own account. Websites should design agent-first signup and login flows.
Klein says websites were designed to block bad bots, but they now need to distinguish those bots from legitimate agents. CAPTCHAs are not effective enough against agents. He mentions authenticated approaches such as Web Bot Auth, which can identify an agent and its source, but says the problem remains unsolved. In his view, the web needs something like a Verisign moment for agents: a trusted certificate issuer that can vouch for an agent and its vendor. Authentication, human approval for sensitive actions, and agent identity all affect whether enterprises can allow agents to access their systems.
A browser-agent platform needs scale, model choice, identity, and observability
Klein says a practical platform must support one agent and thousands of agents, while remaining model agnostic so developers can switch providers as models change. It also needs to handle agent identity and work with anti-bot providers to establish trusted access. Observability is required when an agent can visit any website. Teams need screen recordings, logs, and network activity to see what happened and feed that information back into the agent. Klein mentions Browserbase's Auto Browse as an example of a feedback loop for improving an agent over repeated runs. Browserbase Agents bundles the harness, runtime, sandbox, code execution, fetch, search tools, and models behind a prompt.
Browser agents can automate the overlooked real economy
Klein says the value of computer use is not limited to large AI-native companies. He points to a logistics company in Singapore, a bank in South Africa, and a lumber factory in Mexico. These businesses may still use PHP websites, forms, and people clicking buttons each day. Browser agents can automate those workflows without requiring every company to rebuild its software. He describes Browserbase Agents as a way to provide a ready-made browsing agent and use it as a subagent inside a larger system. The agent records what it did, remembers useful steps, and can optimize later runs. Klein's focus is on solving customer problems instead of rebuilding browser infrastructure from scratch.
"The most reliable browser agents that we see in production right now are often writing code alongside using the browser to actually automate a task."07:32
Who should watch
You are building a browser or computer-use agent and need to decide whether to spend time on models, tools, or runtime infrastructure.
Your agent repeats website tasks and wastes context rediscovering pages, skills, and workflows on every run.
You need agents to access customer systems safely and are working through identity, authentication, anti-bot controls, or enterprise approval.
