Autonomous agents should be trained around a person's goals and workflows instead of being prompted for every individual task.
2
Reactive work, such as customer support and meeting coordination, is likely to be replaced before work that requires open-ended planning.
3
Agent builders should improve systems in stages, starting with prompts and then adding task decomposition, fine-tuning, data filtering, and reinforcement learning when needed.
Summary
Fryderyk Wiatrowski describes a future in which agents handle the repetitive reactions around a person's work while the person keeps the activities that require judgment and leverage. He uses hiring as an example: a founder may want to keep the decision to hire, while an agent handles LinkedIn searches and meeting setup. He expects reactive jobs to go first because their rules are easier to describe. Peter Albert then gives a practical development process for browser agents. Teams can begin with prompting, reduce noise in context, split complex tasks into state tracking and verification steps, and design tool interfaces that models can use reliably. Fine-tuning can use simulated interactions, filtered examples, and model-based judges. Reinforcement learning comes later because it has higher setup costs and makes changes harder. The talk is ambitious about job replacement, while also admitting that current agents cannot yet be trusted on long, complex tasks.
Agents should work from a person's vision instead of waiting for prompts
Fryderyk Wiatrowski argues that a reliable agent should behave more like a good employee than a chatbot. After onboarding and training, the employee receives a vision and finds tasks independently. A prompt-driven interface is useful for demos, but it does not match this model of work. The design problem is to embed agents into existing workflows so they can notice what needs doing. Wiatrowski asks builders to separate high-leverage activities from surrounding noise. A founder may want to preserve hiring decisions, while delegating meeting setup and searches for candidates. The agent should handle the operational work without requiring daily instructions.
Agents can remove low-leverage work around decisions that remain human
Wiatrowski uses the founder's role to show how agents could be assigned narrow problems. Hiring a strong team is highly valuable, but searching for people on LinkedIn and arranging meetings consumes time without being the decision itself. An executive assistant can handle this work, although hiring one requires a large salary and brings additional skills that may not be needed. A specialized agent could handle only the meeting problem. Zeta Labs' Jace is presented as an example. It can read an email, check the founder's availability, and arrange a meeting without browser or tool interaction beyond access to email and the calendar.
Reactive work is easier to replace because its rules can be written down
The talk separates human work into reactive and proactive modes. Customer support is a reactive example: when a customer asks for a refund, the system can check whether the person used the app and then approve or deny the refund. A founder's work is more proactive and harder to describe with a fixed set of rules, but it still contains a reactive layer. Wiatrowski expects this reactive layer to go first. Once the rules and initial training are in place, the agent can respond to triggers without another prompt. The current limit is trust. Agents may complete hundreds or even up to a thousand steps, yet they are not reliable enough for blind delegation.
A trigger pool can move agents from assistants to workflow participants
Wiatrowski proposes starting with a pool of events that cause agent reactions. The pool could contain Slack messages, emails, phone calls, market movements, or issues in Linear. Agents would select tasks according to their rules, do preliminary work in a browser, suggest a solution, and ask the person for approval before taking the final action. This changes the interaction from asking an agent to manage a calendar to having the agent notice the need and prepare the work itself. As rule sets become more descriptive, the same approach could cover more of the proactive parts of jobs. Better foundation models would allow those rules to include more complicated reasoning.
Browser access gives agents a general way to act across software
The proposed trigger aggregator would collect events from many parts of a person's work, including email, Slack, phone calls, market data, and Linear. The next challenge is giving agents a way to act on those events. Wiatrowski favors browser agents because APIs are often limited or poorly implemented, while browser interaction can work across a wider range of software. He suggests beginning with an existing environment such as Linear or Slack, then expanding toward a general aggregator. Full replacement of human jobs depends on both better reasoning and the ability to perform actions reliably in the tools where work actually happens.
Agent systems should be improved in stages because later methods slow iteration
Peter Albert describes a progression for building language-model systems. Prompting is the fastest place to start, while cognitive architectures, fine-tuning, and reinforcement learning require progressively more time and setup. He says prompting changes can happen within hours, whereas fine-tuning and reinforcement learning may become projects lasting weeks or months. His advice includes rewriting prompts with language models, separating instructions from content with XML-like structure, and matching the formats common in a model's training and fine-tuning data. Developers should reduce irrelevant context, use text values when possible, and ask the model to reason about one decision at a time. These choices reduce unnecessary computation and make the task easier for the model.
Task decomposition, state, and verification can reduce cognitive load
When prompting stops producing enough performance, Albert recommends splitting the task into smaller pieces. Longer tasks can track explicit state, generate and revise plans, replan when needed, and keep notes or a scratchpad for intermediate work. Parallel execution can reduce the latency introduced by multiple stages. Tool interfaces also matter. Key-value updates let the model select a field and then decide how to change it. Full rewrites can perform better because models often struggle with small diffs, although they add latency. Albert advises avoiding recursive structures and keeping cognitive components to a minimum, since each extra component can add brittleness even while reducing the load on an individual model call.
Fine-tuning and reinforcement learning need better data and clear feedback
Albert recommends collecting fine-tuning data by simulating real interactions in the application. One model can play different human roles while the rest of the system behaves normally. The examples should vary in difficulty and include enough conditions for the model to learn more than a simple response pattern. Developers can also distill a multi-step pipeline into a model that maps initial inputs directly to final outputs, trading some performance for lower latency. Filtering is another major source of improvement. Execution feedback or language-model judges can reject weak examples. Reinforcement learning can optimize several agent steps together, but Albert places it last because it has high setup cost and makes switching models more difficult.
"We went from prompting agents to do things to manage our calendars to agents doing this on their own."07:07
Who should watch
You are designing an agent that needs to notice work and act inside existing business workflows, rather than wait for a chat prompt.
Your team is deciding whether a task is ready for automation and needs a distinction between rule-based reactive work and open-ended planning.
You are building browser agents and want a practical order for prompt design, decomposition, fine-tuning, data filtering, and reinforcement learning.
