Post-training data is the main bottleneck for both supervised fine-tuning and reinforcement learning, with RL environments treated as data in a different form.
2
OpenThoughts found that multiple reasoning traces per question and a mix of teacher models can work better than simply adding more questions or using the strongest available teacher.
3
Carefully tagged training data improved credit-card recommendation compliance while also reducing latency and improving throughput in production.
Summary
Mahesh Sathiamoorthy argues that post-training depends more on the quality and design of its data than on the training algorithm alone. As agents work on longer tasks, reliability becomes the limiting factor because tools get called incorrectly or the agent makes a mistake. Reinforcement learning can help, but its environments are data, and teams often lack good environments and datasets. He explains how Bespoke Labs built OpenThoughts by testing different curation choices, including question mixtures, answer generation, filtering, and the number of answers per question. The results were sometimes counterintuitive: multiple answers per question helped, and stronger models were not always better teachers. In an enterprise credit-card compliance example, tagged examples helped a model follow detailed rules without relying on a long prompt. He closes with a stack for building and evaluating environments, running sandboxed rollouts, managing long-horizon checkpoints, and applying SFT, RL, and prompt optimization.
Researchers should curate data while measuring its effect on models
Mahesh Sathiamoorthy describes a gap between the people who create datasets and RL environments and the researchers who consume them. Bespoke Labs tries to do both. While curating data, the team asks what will actually move model metrics rather than treating data quality as an abstract property. He connects this to the change from testing what models know to testing what agents can do. For autonomous agents, the practical target is longer reliability. An agent may call the wrong tool or make a mistake after running for a while, and post-training is one way to reduce those failures.
RL environments are data with a different structure
Sathiamoorthy groups supervised fine-tuning data and reinforcement learning environments under the same broader problem: data is the bottleneck. Models, post-training infrastructure, and compute providers are increasingly available, but enterprises and frontier labs still need high-quality examples and environments. An RL environment packages the setting in which an agent acts, receives feedback, and produces trajectories, so it requires curation just as a supervised dataset does. Bespoke Labs therefore works on both reasoning datasets and agent environments, alongside post-training custom models for enterprises.
OpenThoughts was built by testing a curation recipe
OpenThoughts began after Bespoke Labs saw limited access to high-quality reasoning data outside major labs. The project combined work from Bespoke Labs with researchers at Stanford, UC Berkeley, and the University of Washington. Its curation process starts with source questions, chooses and mixes prompts from different datasets, filters questions, generates answers with teacher models, and filters the resulting answers. The team used ablations to test these choices rather than assuming the obvious recipe would work. The paper also reports that increasing the dataset size with the recipe improved benchmark metrics.
Multiple answers can teach more useful reasoning patterns
One counterintuitive OpenThoughts result was that sampling multiple answers for each question worked well. The team could have used many more questions with one answer each, or answered one question 16 times. The latter choice gave the model more variety in how reasoning could be carried out. Sathiamoorthy connects this to fine-tuning on reasoning traces, where variation in the traces may matter. He also says the strongest model was not always the best teacher. Synthetic question generation and question answering worked in their tests, while answer filtering and some other steps did not work as well.
SFT can provide much of the gain before expensive RL
In OpenThoughts Agents, the team applied a similar process to agent data and RL environments. It tested sources, mixing, filtering, rollout generation, and teacher selection while checking whether the recipe scaled with dataset size. Multiple answers again helped, and stronger models were not always the best teachers. Sathiamoorthy says synthetic rewriting and task augmentation did not work as well as expected. He also found that supervised fine-tuning contributed substantially to the gains. RL helped with the final few percentage points, but it was computationally expensive, so SFT can be a strong choice for many enterprise settings.
Tagged examples helped a model follow credit-card compliance rules
Sathiamoorthy gives a production example involving Credit Karma, an Intuit app that explains why a credit card was recommended. A raw model could produce an explanation, but it did not consistently follow the many rules required for compliance. Putting all those rules in the prompt increased latency. The training data also had imbalances, and fine-tuning could cause the model to hallucinate values such as 0% APR. Bespoke Labs changed the curation recipe by adding tags to the prompt-response examples. The tags directed the model toward the required form instead of making it focus on specific numbers. Compliance metrics, latency, and throughput improved.
A curation tool connects collected data to fine-tuning
Curator was built to make reasoning-data curation easier. It can start from a Hugging Face dataset containing prompts, or from logs collected by a team that wants to obtain responses and fine-tune a model. The tool was used in the original OpenThoughts work and integrates with Tinker and Fireworks. Its role is to turn raw prompts or interactions into data that can be inspected, generated, filtered, and passed into post-training workflows. This reflects Sathiamoorthy's broader view that curation needs practical tooling, rather than relying only on a training run after the data has been assembled.
A post-training stack needs environment management below the training algorithms
The stack Sathiamoorthy describes starts with tools for building RL environments, measuring their quality, and tracking versions. Under that is infrastructure for sandboxes and rollout orchestration. Long-horizon rollouts may require checkpointing, snapshots, and rollback. Above those layers are SFT and RL systems for updating models, along with prompt optimization methods that use language models and reflection to update system prompts or agent harnesses. The same stack supports both environment curation and agent post-training. He presents it as the architecture Bespoke Labs and other teams are building toward.
"As you're curating data you want to put yourself in the shoes of the researcher to see what does it take to actually move the metrics on the models."02:04
Who should watch
You are building a post-training pipeline and need to decide where data curation fits alongside SFT, RL, and infrastructure.
Your agent works on long tasks but fails through tool mistakes or unreliable trajectories, and you need ideas for curating environments and rollouts.
You are adapting a model for an enterprise workflow with detailed compliance rules, uneven examples, or latency constraints.
