Training data is not a pile of tokens waiting to become a model. It is a set of choices about what the system should learn, which mistakes count, and how closely a task resembles the world outside the lab. Useful examples have to be selected, visual variety preserved, teachers chosen and failures found before they disappear into scale. Agent environments make the same problem easier to see. Broken tasks reward the wrong behavior, synthetic users invent details unless they are grounded, and even a realistic sandbox leaves out parts of production. Opt-in product traces can repair the next training set, but only after filtering turns raw behavior into examples worth learning from.
Why first: She and Shpak treat dataset development as an engineering system rather than a one-off export. Cleaning, versioning, random access, streaming and analysis all have to work before anyone can make a defensible selection. Morcos follows by asking which examples should make it through that system.
Why second: She and Shpak build the data machinery. Morcos gives it a target: more information per token for the task that matters. His examples show curation changing training cost, serving cost and domain performance. Lee then makes the same point in a medium where a bad choice is visible at a glance.
Why here: Morcos argues that selection determines what a model can learn. Lee shows how selection also determines its taste. A caption that omits the white wall teaches the wrong visual rule, while aggressive quality filters erase useful styles. Marten takes that experimental mindset into reasoning data.
Why here: Lee curates billions of images with staged filters. Marten applies the same discipline to a smaller reasoning pipeline and tests each choice: question sources, repeated traces, teachers and answer filters. The next talk shows what happens when a good-looking synthetic recipe meets a full training run.
Why here: Marten finds useful synthetic recipes with controlled experiments. This talk follows them into scale, where repetition, replica hashes, precision changes and silent corruption can turn a data result into a training failure. Crawford narrows the problem again by asking whether an agent task deserves to be trained on at all.
Why here: the previous talk shows that a recipe can fail in the machinery around it. Crawford shows that the task itself can be broken. An impossible specification or unreliable environment produces failure without a useful learning signal. Anand then applies the same suspicion to synthetic people.
Why here: Crawford separates genuine difficulty from noise in a task definition. Anand does the same for synthetic personas. Models invent missing context, react to answer order and predict statements better than actions. Gupta and Rajpal follow with a production system that grounds those simulated users in tools and account state.
Why here: Anand says synthetic people are forecasts that need checking against reality. Gupta and Rajpal show that check in practice. Nubank compares simulated and real conversations before using the synthetic set to screen agent changes. Wang then asks how much of the surrounding workplace a simulation must contain.
Why here: Nubank simulates a customer, account state and mocked tools around an existing agent. Wang widens the environment to projects, incidents, deployments, distributed failures and real cloud resources. Zhao and colleagues close the pack by showing another source of difficult data: the product as people actually use it.
Why last: Wang builds a richer imitation of production. Zhao and colleagues begin with opt-in production snapshots, repair teacher predictions, remove noisy moments and train a small model for the next keystroke. It is a fitting end because the dataset is no longer upstream of the product. Use generates evidence, and that evidence changes the next model.