Supervised fine-tuning on long reasoning traces can teach a model to produce extended step-by-step reasoning.
2
The OpenThoughts 3 recipe improves results through careful choices about question sources, filtering, answer generation, and teacher models.
3
A specialized model can surpass its teacher in some domains when the data, verification, and evaluation process fit that domain.
Summary
Ryan Marten presents OpenThoughts, an effort to find the missing data recipe behind strong open reasoning models. He explains that DeepSeek R1's released weights are based on supervised fine-tuning, even though reinforcement learning helped generate much of its data. OpenThoughts 3 tests question sourcing, source mixing, filtering, teacher models, and answer quality across math, code, and science. The experiments found that repeated reasoning traces per question, synthetic question sources, model-based difficulty filters, and a carefully chosen teacher can improve supervised fine-tuning. Some common assumptions did not hold: broader source diversity was worse than using fewer high-quality sources, and answer verification did not help for the general SFT recipe. Marten recommends starting with the recipe, then adapting each stage to the target domain. He also describes Curator and Evalchemy, open-source tools for data generation and evaluation.
The missing part of the open reasoning recipe is the data
Marten says DeepSeek R1 was especially interesting because its final weights came from DeepSeek V3 base fine-tuned on 800,000 examples, including 600,000 reasoning examples. Reinforcement learning helped create the data and was used for alignment, but the released model was ultimately an SFT model. The small DeepSeek reasoning models were also unusually strong. Open projects therefore had a broad training recipe but little detail about the datasets behind it. OpenThoughts focuses on that missing link: how much data to use, which creation steps matter, how to choose among alternatives, and how to test those choices.
OpenThoughts 3 improves the scaling curve for supervised fine-tuning
OpenThoughts 3 compares accuracy on competitive math, competitive code, and science questions as the SFT dataset grows. Marten says supervised fine-tuning scales more easily than reinforcement learning in this setting. Using the same base model, the OpenThoughts recipe outperformed NVIDIA's Nemotron Nano data recipe. The resulting 7B model also surpassed the DeepSeek R1 Qwen 7B model that the team originally tried to reproduce. Marten describes dataset size as one way to raise accuracy, while improving the recipe shifts the whole scaling curve upward.
The team tested each stage of the data pipeline instead of guessing
The OpenThoughts pipeline covers question sourcing, mixing sources, filtering questions, generating answers with a teacher model, filtering answers, and selecting the teacher model. Marten says the project produced more than 5,000 datasets and almost 3,000 models on Hugging Face, with about 1,000 experiments directly supporting this work. The team used small-scale experiments to compare individual choices before scaling them. He presents this as a practical way to find useful signals without immediately committing to very large training runs, while admitting that choices that work at small scale can sometimes change at larger scale.
Sampling many reasoning traces for each question can scale data efficiently
The team found that multiple answers per question worked well. At a fixed total number of examples, 30,000 questions sampled once performed similarly to a smaller set of questions with 16 reasoning traces sampled for each question. This gives the team a way to increase the number of training examples without sourcing an equal number of new questions. Marten says that multiplying traces by 16 can correspond to a substantial accuracy increase on the scaling curve. The result is useful when high-quality questions are scarce but a suitable teacher can generate more attempts.
Teacher quality depends on teaching behavior, not only benchmark scores
A model that scores better on evaluation benchmarks is not automatically the best teacher. Marten compares the situation to a brilliant researcher who is a poor lecturer. In their experiments, Qwen 32B was a stronger teacher than DeepSeek R1, so the team changed its recipe even though R1 had been the model most people were using. Claude was also tested and performed poorly as a teacher. Marten suggests that reasoning-trace length, context-window use, token count, step count, and output formatting may explain the difference, but he calls the underlying cause an open research question.
Synthetic questions and model-based filtering worked better than broad source diversity
Some of the strongest question sources were entirely synthetic and outperformed sources scraped from forums or written manually. Marten considers this useful because synthetic question generation can scale. The team filtered questions by asking a language model to estimate difficulty or by examining the length of a model-generated answer. These methods worked better than embedding-based approaches and fastText classifiers that are common in pretraining pipelines. At the same time, using a smaller number of high-quality sources worked better than collecting many sources to maximize diversity.
Marten advises teams building specialized reasoning models to start with the OpenThoughts recipe and then test alternatives when they have enough compute. The team saw different useful filters for code, math, and science. Difficulty labels worked well for code questions, while response length worked better for math and science. A coding answer can contain many lines even when the question is simple, whereas an AIME math answer may end in a number from zero to a thousand. For domains with little data, he recommends transforming existing material into questions and using in-context examples to generate more synthetic data.
Evaluation design matters when the test set is small
Marten calls evaluation essential for making informed data decisions. OpenThoughts uses Evalchemy to support evaluation, sharding, and parallelism. He warns that a tiny test set can produce a noisy score. For AIME, which has only 30 questions per year, the team ran the model on those questions 10 times and averaged the results. Repeating evaluation in this way gave them a more stable signal for comparing data strategies. He presents this as part of the engineering needed to make dataset experiments useful, rather than as a separate final check.
Distillation can surpass the teacher in a specialized domain
Marten describes a legal reasoning experiment involving classification of Supreme Court decisions. The team used 2,000 unique questions, sampled five answers for each question, and verified the answers before discarding incorrect ones. After fine-tuning a 7B model, the result surpassed DeepSeek R1 on that domain. This challenges the assumption that distillation only lets a smaller model catch up to its teacher. Marten says the effect needs more research and application, but the experiment suggests that carefully selected domain data can allow a specialized student to outperform a much larger general reasoning model.
"A better model in terms of its own performance on evaluation benchmarks does not necessarily mean it's a better teacher model."08:54
Who should watch
You are building a reasoning model with supervised fine-tuning and need practical choices for sourcing, filtering, and generating training data.
Your domain has limited labeled examples and you want to expand it with synthetic questions and teacher-generated reasoning traces.
You run small or noisy evaluations and need a better way to compare dataset experiments.
