# Post-training and RL A pack of 10 talks from the AI Engineer YouTube channel, in the order to watch them. 5h 05m of video. Page: https://aietalks.com/packs/post-training-and-rl In one financial tool-use task, a trained 4 billion parameter model outperforms a 235 billion parameter baseline, but only because the team knows which behavior to reward. Human preference, imitation and verifiable correctness produce different systems, and the base model must already contain skills that reinforcement learning can combine. Product work makes those distinctions concrete. An inbox or coding task needs a realistic environment, and engineers still have to read the rollouts because reward hacks can look like progress. When a task has no exact answer, judges, grounding and known end states have to provide a usable signal. Asynchronous rollouts keep GPUs busy but introduce stale-policy variance, so environments, inference and training must run separately without losing the tests that defined success. ## This pack is for you if - Your prompted agent has plateaued, but you are not sure which training method fits the failure. - You can describe good behavior to a person, but cannot yet turn it into a clean reward. - Your RL run works at small scale and stalls, drifts or wastes GPUs when rollouts get long. ## The talks, in order ### 1. What's Next After RLHF? Diogo Almeida, TypeSafe AI | 18:05 | AI Engineer World's Fair 2026 Video: https://www.youtube.com/watch?v=cJ0EOzey--o Summary: https://aietalks.com/talks/whats-next-after-rlhf.md Why start here: Almeida asks what the model is actually being trained to want. Human preference produces a good assistant, while correctness or calibrated decisions imply different systems. Bigio comes next because he turns Almeida's three objectives into concrete choices among SFT, DPO and RFT. ### 2. RFT, DPO, SFT: Fine-tuning with OpenAI Ilan Bigio, OpenAI | 1:46:15 | AI Engineer World's Fair 2025 Video: https://www.youtube.com/watch?v=JfaLQqfXqPA Summary: https://aietalks.com/talks/rft-dpo-sft-fine-tuning-with-openai.md Why here: Almeida separates the goals; Bigio shows what each learning signal can and cannot teach. His warning that RFT needs a clean grader also explains why Singh looks earlier in the training process for the atomic skills that RL will later combine. ### 3. The Base Model Is Dead Varun Singh, Arcee AI | 17:45 | AI Engineer World's Fair 2026 Video: https://www.youtube.com/watch?v=xbPriQWXtWM Summary: https://aietalks.com/talks/the-base-model-is-dead.md Why here: Bigio starts with a capable model and chooses a post-training method. Singh asks how the model became ready for that method at all: mid-training supplies agentic data, and supervised learning supplies skills for RL to compose. Crawford then shows one of those compositions in action through disciplined tool use. ### 4. Stop Making Models Bigger, Make Them Behave Kobie Crawford, Snorkel | 20:56 | AI Engineer Europe 2026 Video: https://www.youtube.com/watch?v=TNwJ1LMiENk Summary: https://aietalks.com/talks/stop-making-models-bigger-make-them-behave.md Why here: Singh explains why RL needs the right prior; Crawford shows what RL can change once that prior exists. A 4 billion parameter model succeeds by inspecting tables, reading schemas and recovering from errors. Brown follows by asking how a product team can define and measure that behavior before training it. ### 5. RL Environments at Scale Will Brown, Prime Intellect | 18:30 | AI Engineer CODE 2025 Video: https://www.youtube.com/watch?v=_IzZWeuTx7I Summary: https://aietalks.com/talks/rl-environments-at-scale.md Why here: Crawford diagnoses one product behavior and trains for it. Brown gives that work a reusable form: an environment joins the product harness to tasks and rewards, then supports evaluation, data generation or RL. Corbitt comes next with a complete environment built from a real inbox. ### 6. How to Train Your Agent: Building Reliable Agents with RL Kyle Corbitt, OpenPipe | 19:48 | AI Engineer World's Fair 2025 Video: https://www.youtube.com/watch?v=gEDl9C8s_-4 Summary: https://aietalks.com/talks/how-to-train-your-agent-building-reliable-agents-with-rl.md Why here: Brown defines the environment; Corbitt shows how easily a plausible one can still teach the wrong lesson. His email agent needs realistic data and a reward without loopholes. Hang and Zhou then widen that case study into a method for adapting tool behavior through exploration. ### 7. Agent Reinforcement Fine-Tuning Will Hang & Cathy Zhou, OpenAI | 16:55 | AI Engineer Video: https://www.youtube.com/watch?v=p1CmPZ2j6Lk Summary: https://aietalks.com/talks/agent-reinforcement-fine-tuning.md Why here: Corbitt catches reward hacks by reading the rollouts. Hang and Zhou show what those rollouts are for: exploration lets an agent adapt its tool calls to a new domain and budget, provided the reward resists shortcuts. Brown follows with the harder case, where there is no exact answer to grade. ### 8. Reinforcement Learning without Verifiable Rewards Will Brown, Prime Intellect | 19:27 | AI Engineer World's Fair 2026 Video: https://www.youtube.com/watch?v=AQv3qRCG6Gw Summary: https://aietalks.com/talks/reinforcement-learning-without-verifiable-rewards.md Why here: Hang and Zhou assume success can be graded cleanly. Brown deals with reports, browsing and other work where it cannot. Grounding, judges and known end states manufacture a usable signal. Garg and Li come next because generating enough exploratory rollouts turns reward design into a systems problem. ### 9. Efficient Reinforcement Learning Rhythm Garg & Linden Li, Applied Compute | 20:19 | AI Engineer CODE 2025 Video: https://www.youtube.com/watch?v=o15AaYl7Wu0 Summary: https://aietalks.com/talks/efficient-reinforcement-learning.md Why here: Brown shows how to create harder tasks; Garg and Li show the cost of sampling them. Synchronous RL waits for the slowest rollout, while asynchronous RL trades idle GPUs for stale-policy variance. Brown's final talk keeps that trade-off but places it inside a complete post-training architecture. ### 10. Modern Post-Training: A Deep Dive Will Brown, Prime Intellect | 46:52 | AI Engineer World's Fair 2026 Video: https://www.youtube.com/watch?v=V-EDrhIhHzQ Summary: https://aietalks.com/talks/modern-post-training-a-deep-dive.md Why end here: Garg and Li isolate the throughput problem. Brown reconnects it to everything before it: one environment can produce evals, SFT data, distillation data or RL rollouts, while separate inference and training services stop long agent runs from blocking the batch. This is the assembled system the first nine talks prepare. ## Editor's note Corbitt and both Brown talks return to the same warning: a reward curve cannot tell you how the agent earned it. You need the rollouts, especially when the model finds a loophole. Kitaru records agent runs so you can inspect and replay the behavior behind a score, then turn the failure into a better environment or evaluator. Written by the AIE Talks editors (the Kitaru team), not by any of the speakers.