Reinforcement learning gives production models a systematic way to incorporate defects, business metrics, and environmental feedback.
2
Agent training can create its own synthetic data through rewarded environment trajectories, while mock tools and users let agents fail safely.
3
Human feedback can define rubrics for LLM judges, then production feedback can train reward models at larger scale.
Summary
Alessandro Cappelli argues that the difficult part of GenAI is not building an MVP. The long phase is improving the system from its first demo through production and ongoing use. Proprietary models and instruction fine-tuning can steer behavior, but they do not provide a systematic way to incorporate defects and business signals. Cappelli presents reinforcement learning as the method for connecting those signals to repeated model improvement. Smaller RL-trained models can reduce serving costs and latency, which matters for enterprise workloads such as call summarization and speech-to-speech support. Agents make the problem harder because they consume more tokens and can change data in connected systems. Cappelli explains how existing or mocked environments, business rewards, LLM judges, and production feedback fit together. He is also direct about the cost: RL is harder to run than prompting or supervised fine-tuning, with algorithms such as PPO requiring several language models to be orchestrated.
Cappelli calls the belief that deployment is a short final step the "myth of the last mile." A demo built on a proprietary model or instruction fine-tuning may look good, but those approaches do not systematically absorb defects discovered in production. With a proprietary model, teams can change the system prompt, yet fixing one behavior can create another defect. With instruction fine-tuning, they can keep revising the dataset, which raises the question of whether they will create another dataset every week. He describes the path from MVP to production and continued improvement as the real marathon.
Reinforcement learning connects feedback to model improvement
The way forward, according to Cappelli, is to accelerate the model life cycle by integrating feedback from clients, business metrics, and the environment. Reinforcement learning provides a mathematical mechanism for turning those signals into improvement. He contrasts this with prompting and instruction fine-tuning, which also steer behavior but do not provide the same feedback loop. The point is operational as well as technical: a team can find defects before and after production, use the relevant signal, and retrain or refine the model instead of repeatedly editing prompts or rebuilding datasets.
Smaller models can make enterprise use cases affordable and fast enough
Cappelli says RL can reach the same performance as supervised fine-tuning with a much smaller model. That changes the economics of large enterprise workloads. AT&T, for example, summarizes every transcript between a customer and an agent, and this costs millions of dollars at scale. A smaller model can reduce serving costs and can meet latency limits that larger models cannot. For speech-to-speech customer support, he says half a second already feels strange and that a third of a second is preferable. He also points to ownership of business data and the trained solution as another benefit of using an in-house model.
Agents need environments where their actions can be evaluated
Agents raise the requirements because they use more tokens, have more complex workflows, and may change records in connected databases. This leaves less room for mistakes and makes the token economics harder to justify. Cappelli describes two ways to train them. If a company already has a workflow, such as Manulife's existing agents, a model can be connected directly to that environment. If no environment exists, the company can mock its tools and create a mock user, including an LLM that interacts with a tool-using chatbot. Rewards can then measure business outcomes, KPIs, or qualities such as helpfulness and adherence to tone and vocabulary guidelines.
The environment can produce the initial agent-training data
Agent training data is difficult to find because web data rarely contains an agent using tools in a company workflow. Cappelli says an environment and a reward function solve part of this problem by producing trajectories as a byproduct of training. The reward indicates which trajectories are good, allowing rejection sampling to create a dataset for bootstrapping the first model training run. Existing company data can make the environment more realistic. Customer support transcripts can train a mock user to repeat questions, behave anxiously, or describe urgent situations. In a medical-supply setting, the correct response might be escalation to a human or a 911 call.
Human feedback can define evaluators without a large annotation campaign
Cappelli questions the assumption that keeping a human in the loop requires weeks of annotation. Some rewards are systematic, such as whether code runs or syntax is correct. Others come directly from business outcomes, such as a customer-support system's containment rate, meaning the percentage of calls completed end to end by the model. For open-ended properties such as tone and adherence to business requirements, humans can define rubrics, prompts for LLM judges, and scenarios. He says this work can take minutes to hours rather than weeks and does not need to be repeated dozens of times.
Production feedback can evolve from judging to reward models
In the question-and-answer section, Cappelli distinguishes early feedback from the larger stream available after launch. A small amount of human feedback can first improve the LLM judge by clarifying what counts as good or bad. Once production generates thousands of feedback signals, the team can use them to train reward models and scale the feedback process. For implicit signals, such as whether a suggested completion is accepted, he says the right approach depends on the use case. Teams can compare training approaches against an evaluation and select the one that gives the best measured performance.
RL needs specialized infrastructure because the algorithms are difficult to run
Cappelli presents Adaptive Engine as a platform for observing, training, evaluating, and serving models in one system. He says speeding up the life cycle includes evaluating behavior and finding defects before and after production, not only making training faster. The difficulty is that RL is harder to run than changing a system prompt or preparing an instruction-tuning dataset. PPO, one of the well-known RL algorithms, requires four language models to be orchestrated at the same time. Adaptive Engine provides pre-built recipes so users can define rubrics without implementing algorithms such as GSPO or constructing the full training setup themselves.
"The nice thing is that when you train a model with reinforcement learning and you have an environment and you have a reward in place, basically, you just build what is a byproduct of your environment, you created a synthetic data set pipeline."09:35
Who should watch
You are moving a GenAI proof of concept toward production and need a way to turn defects and business outcomes into repeatable model updates.
Your agent has to use tools, interact with customers, or change live records, and you need mock environments and measurable rewards before deployment.
You have some production feedback but want to replace repeated annotation campaigns with LLM judges and reward models.
