The May 28 DeepSeek update improved reasoning through additional post-training, doubling the model's average reasoning length and bringing reported benchmark performance up to OpenAI o3 and Gemini 2.5 levels.
2
DeepSeek R1 developed reasoning from a base model with pure GRPO reinforcement learning on verifiable math and code tasks, while R1 used cold-start supervised fine-tuning, rejection sampling, and another RL stage to become a useful chat assistant.
3
Distilling long reasoning traces into smaller models produced strong results, including a DeepSeek-distilled Qwen3 8B model that Sapra said matched the performance of Qwen3's 235B thinking model.
Summary
VIbhu Sapra first announces Test of Time Paper Club, a curriculum-based second track that will cover foundational AI engineering papers from July through December, alongside the existing weekly paper club. The main discussion then reviews DeepSeek V3 and R1, with attention to the May 28 update. Sapra argues that additional reinforcement learning increased the model's average reasoning length from about 12,000 to 25,000 tokens and materially improved its reported math, coding, and reasoning scores. He explains how R1 emerged from a four-stage pipeline, while the simpler R1-Zero path applied pure GRPO RL directly to a base model. The talk also covers reflection and "aha" moments, where longer reasoning lets the model revisit failed approaches. The final section examines distillation, which transfers these reasoning traces into Qwen and Llama models. Sapra is candid about remaining weaknesses, including function calling in the earlier R1, language mixing, prompt sensitivity, and limited engineering gains over V3.
Test of Time Paper Club will organize foundational papers into a six-month curriculum
Sapra introduces a second Paper Club track for papers and ideas that remain useful after current trends change. It will run from July through December, with roughly 24 weeks of sessions covering two to four papers at a time. Planned areas include attention, optimizers, gradient descent, RNNs, LSTMs, GPT-2, BERT, scaling laws, Chinchilla, distillation, diffusion, fine-tuning, voice, inference optimization, and evaluation. The sessions will have presentations and discussion, with both a San Francisco in-person section and a remote section. The original weekly club will continue to cover newly released work.
The May 28 DeepSeek update gained much of its improvement from more post-training
Sapra describes the May 28 DeepSeek release as a substantial update despite retaining the existing name. He says the team applied better post-training to DeepSeek V3, with a particular focus on reasoning, JSON output, and function calling. The reported AIME 2024 score rose from 70% to 87.5%. He says the model moved from roughly 12,000 reasoning tokens on average to about 25,000, allowing it to spend twice as long working through difficult problems. In his account, this brought the model to the reported level of OpenAI o3 and Gemini 2.5 on math, coding, and reasoning benchmarks.
Longer reasoning creates another dimension for scaling after pretraining becomes expensive
Sapra frames test-time scaling as a response to the cost of continuing to expand pretraining. Earlier scaling approaches increased model size, training data, or both, moving from Chinchilla-style compute balance toward models trained on far more tokens for faster inference. He says this approach becomes increasingly expensive and eventually runs into limits on available data and compute. Reasoning models add another axis: a capable base model can spend more inference-time computation on hard queries. DeepSeek's approach trains the model to produce longer reasoning traces, especially on tasks with verifiable answers, rather than relying on search or manually assembled chains of thought.
DeepSeek-R1-Zero showed that pure reinforcement learning could produce reasoning from a base model
R1-Zero starts with the DeepSeek V3 base model, which has been trained to predict the next token but has not been tuned as a chat assistant. DeepSeek applies GRPO reinforcement learning without supervised fine-tuning. The training focuses on math and code tasks whose answers can be checked, such as whether a mathematical result is correct or whether code compiles. Rewards also depend on output format, with the thinking process placed between think tags. Sapra says the resulting model became good at exposing its reasoning and extending test-time computation, but it was not yet a polished general assistant. It could have poor readability and switch languages while reasoning.
Long reasoning traces led to reflection and self-correction behaviors
Sapra says that, as models learn to reason for thousands of steps, they begin revisiting earlier work instead of immediately producing an answer. They can re-evaluate a failed path, try an alternative, and continue thinking when the first approach does not work. He calls these reflection moments and describes the appearance of "aha moments" as a central observation in the DeepSeek paper. His example is a math problem involving nested square roots. During the trace, the model notices that squaring both sides may simplify the problem, then explicitly revisits its earlier approach. Sapra presents this as an effect that emerges from incentives for correct answers rather than from supervised instruction about how to reason.
R1 adds supervised cold start and rejection sampling around the RL stage
DeepSeek R1 uses four stages. First, a base model receives a small cold-start dataset of readable, long-form reasoning examples, including human-annotated data. Next comes RL on difficult math and coding tasks with verifiable outcomes. The third stage generates completions, scores them with a reward model, and keeps useful samples. Sapra gives the dataset split as 600,000 reasoning examples and 200,000 general chat examples from 800,000 generated samples. The final RL stage combines reasoning rewards with human preferences for general chat, such as when to be detailed or concise. This process turns the raw reasoning model into a more useful assistant.
Distillation transfers DeepSeek's reasoning gains into small Qwen and Llama models
DeepSeek distilled R1's reasoning traces into Qwen and Llama models using supervised fine-tuning rather than native RL. The released family included Qwen models from 1.5B through 32B and Llama 8B and 70B. Sapra says the distilled models substantially improved on their corresponding base models, and that the newer Qwen3 8B distillation matched the reported performance of Qwen3's 235B thinking model. He also describes an experiment applying RL directly to Qwen3 32B for 10,000 steps, which performed worse than distillation. His conclusion is that smaller models benefit from a useful supervised starting point before attempting more demanding RL.
The earlier R1 still had practical weaknesses despite strong benchmark results
Sapra separates the reported reasoning scores from the model's product behavior. The earlier R1 was worse than V3 at function calling, multi-turn interaction, and JSON mode. It also struggled with language mixing and was sensitive to prompting. He says the May 28 update addressed native function calling and structured JSON output, while reducing hallucinations. He also rejects the idea that R1 was clearly better than V3 on engineering tasks, describing that claim as outdated after the new model update. The talk ends with open reproduction work from groups including Hugging Face and Bespoke Labs, and with an invitation to participate in the Paper Club.
"Rather than explicitly teaching the model how to solve a problem, we simply provide it with the right incentives and it autonomously develops advanced problem-solving strategies."36:43
Who should watch
You are trying to understand why DeepSeek R1 became a reasoning model and want the training recipe explained without reading the full paper.
You are deciding whether to distill reasoning traces into a smaller model or apply RL directly to its base model.
You want a structured reading plan for foundational papers in deep learning, language models, post-training, inference, diffusion, voice, and evaluation.
