AlphaLab turns a plain-language research problem into code, evaluations, cluster jobs, experiments, and trained models.
2
A strategist proposes experiments while worker agents implement, run, and analyze them through a board that researchers can edit and approve.
3
Rappazzo argues that enterprise value will come from building verifiable environments and research rubrics, while the system in the middle improves itself.
Summary
Brendan Rappazzo presents AlphaLab, Morgan Stanley's open-source system for automating quantitative research. A user supplies data access and a prediction goal in natural language. AlphaLab researches the domain, builds an evaluation or backtest, then runs many experiments across the team's cluster. A strategist agent proposes work, while worker agents write code, configure Slurm jobs, inspect training curves and evaluation results, and return postmortems. People can edit the experiment board, cancel work, add ideas, or chat with the strategist. Early results included a better language-model training configuration, a top-12% result in an NVIDIA Kaggle competition, and internal model improvements moving through risk review toward production. Rappazzo is direct about failures caused by weak evaluations. AlphaLab 2.0 therefore treats carefully designed environments, held-out scores, and qualitative research rubrics as the main source of learning and enterprise expertise. The long-term goal is a harness that improves its own research process.
Long-horizon coding agents made automated research practical
Morgan Stanley had wanted to automate quantitative research because its problems often resemble Kaggle tasks: predict future values from time-series data, sometimes with calibration constraints. Existing production algorithms also offered room for more hyperparameter searches, ensembles, and transfers between desks such as credit bonds and municipal bonds. Rappazzo says the effort became plausible in December 2025, when models such as Opus 4.5 and harnesses such as Claude Code and Codex could handle long tasks. The group then made building an auto-research agent a major effort.
AlphaLab accepts a research goal and handles the full experimental cycle
A user can provide a file path or API access and describe the prediction goal in ordinary language. AlphaLab then moves through research, evaluation or backtest construction, and mass experimentation. The output is a suite of trained machine-learning models. The system is designed to cover both a new dataset with no existing evaluation and an established model where the user wants more experiments. It is also model agnostic, so Morgan Stanley can connect frontier providers or open-source models without rebuilding the harness.
The team wrote its own harness to control tools and standards
AlphaLab uses its own code rather than an off-the-shelf agent framework. Rappazzo says this gives the team freedom to change anything it cares about and keeps the system easy to adapt across OpenAI, Anthropic, and open-source providers. The main tools include full shell access for writing and running code, web search for reading papers and technical material, and a cluster interface built around Slurm. An agent can request resources such as four H100 GPUs and CPUs, submit the job, and avoid handling the lower-level hardware orchestration itself.
The research board lets agents work while people retain control
The initial research phase builds context through a generated to-do list covering tasks such as setting up Python, loading data, and running statistical tests. Each item produces notes in a Markdown file, which later agents can query as they manage their context. In the experimentation phase, a strategist reads the accumulated context and proposes cards on a Jira or Kanban-style board. Worker agents implement each card, write the Slurm configuration, submit the job, inspect training curves and evaluation results, and write a postmortem. The strategist uses those results to choose follow-up work. Researchers can cancel cards, add cards, or steer the strategist through chat.
A weak evaluation can make the whole research loop optimize the wrong thing
Rappazzo says AlphaLab has encountered serious failures, especially when the evaluation itself is poor. The first version uses multiple agents to reduce this risk. One builder writes the evaluation code. Two critics then inspect it, with one looking for conceptual errors and information leakage and the other writing unit and integration tests. Their findings return to the builder, and the cycle continues until the critics accept the evaluation. AlphaLab 2.0 puts even more emphasis on evaluations because the system needs a clear measurement before the team can judge whether changes to the research process help.
AlphaLab has already found improvements in model training and internal systems
In the early academic tests, AlphaLab worked with CUDA kernels, an academic traffic time-series dataset, and language-model training. Compared with a single agent in a loop, it found a better training configuration for an LLM. In an NVIDIA-hosted Kaggle competition, it fine-tuned a NeMoTron model into a reasoning model and reached the top 12% of submissions after joining late and getting about 10 iterations. Internally, the team has given AlphaLab existing models and asked it to keep experimenting. Rappazzo says several of those efforts found meaningful improvements that are moving through risk review and toward production.
Verifiable environments are how researchers encode expertise into the system
Rappazzo's answer to AlphaLab's failures and arbitrary design choices is to build better environments and evaluations. The newer setup resembles Kaggle: a task supplies data and a description, AlphaLab submits containerized models, and the system returns public and private leaderboard scores. Morgan Stanley has built roughly 10 to 20 careful environments. These provide a reinforcement-learning signal for tuning the harness. The team also grades traces with qualitative rubrics that describe what a good researcher does and how well an agent follows the research process. Proprietary data matters, but designing the environment and its verifiable standards is the harder work.
The long-term system should improve its own harness
With a measurable environment, people can manually test ideas such as using two debating strategists. AlphaLab's next step is meta-harness optimization, where a model reads execution traces and results, then changes the harness itself. The team is also collecting useful traces from open-source models and training model weights with methods such as GRPL and on-policy distillation. Rappazzo expects the best system may combine open-source and closed-source models. He argues that general auto-research will become a commodity, while the lasting work for an enterprise or expert is building environments that make research verifiable.
"I really think all of your value as like an enterprise or human expert comes from building environments."19:04
Who should watch
You are building agents that need to write code, run long jobs, and learn from experiments rather than answer isolated prompts.
Your system can generate experiments, but you are unsure whether its evaluation measures the thing you actually care about.
You work with proprietary research processes and want a concrete way to encode them into agent training and assessment.
