A 4 billion parameter model beat a 235 billion parameter model on a financial tool-use task after reinforcement learning taught it to inspect tables, use schemas, and recover from query errors.
2
The main failure was tool discipline. The larger model could reason well, but it queried a table that did not exist, tried again, and then invented an answer.
3
Training only on single-table questions improved performance on harder multi-table questions, while evaluation rubrics helped identify which behaviors needed training.
Summary
Kobie Crawford presents a study from Snorkel and UC Berkeley's RLLM team on improving financial analysis with reinforcement learning. A 235 billion parameter Qwen 3 model failed to answer a YouTube advertising revenue question because it skipped table discovery, queried a nonexistent table, and hallucinated after two failed attempts. A 4 billion parameter model fine-tuned with RL first called get_table_name, inspected the schema, ran a query, noticed a column error, and corrected itself. The run took 21 hours and cost under $500. Training on single-table tasks produced the strongest result and transferred to the harder FinQA reasoning benchmark, where performance rose from 13.9% to 26.6%. Crawford argues that teams should diagnose the exact behavior causing failures before creating training data. Rubric-based evaluations can break a final answer into smaller checks, while RL can still use a single reward value for training.
A larger model can fail when it does not follow the tool-use procedure
Crawford shows a 235 billion parameter Qwen 3 model answering a question about YouTube ad revenue growth from 2023 to 2024. The model did not inspect the available tools or tables. It issued a query against a table that did not exist, tried another query, received no useful result, and then hallucinated an answer. Its reasoning ability did not help because it skipped the basic steps required by the environment. Crawford uses this example to question the assumption that a larger model will automatically solve an application problem when its tool behavior is unreliable.
Reinforcement learning can change application behavior in a small model
The target was a 4 billion parameter model that could outperform a 235 billion parameter model on financial analysis with tools. Crawford describes RL as a way to change behavior, such as how a model interacts with an environment, rather than mainly adding facts to its internal knowledge. The motivation includes lower inference cost, faster operation, easier self-hosting, and tighter control over sensitive financial or healthcare data. A large model may contain far more general capability than a constrained production task requires.
The training run used verified expert-generated data and a self-contained environment
Snorkel generated the dataset with experts who had financial analysis experience, including people with advanced academic or industry backgrounds. The tasks were checked to ensure that queries were valid and that each task had a verifiable answer. The RL setup used GRPO, a 4 billion parameter starting model, the RLLM framework, and a FinQA environment. The environment contains the needed tools and has no external runtime dependency. Crawford says the run lasted 21 hours and cost under $500 per run.
Tool discipline mattered more than deeper reasoning on these questions
The fine-tuned 4 billion parameter model began by calling get_table_name to discover the available tables. It then used get_table_info to inspect the schema before writing SQL. Its first query requested a revenue column that was not present, but the model read the error, found the actual column, and corrected the query. Crawford presents these behaviors as the reason it succeeded. The model learned to discover the environment, use the schema, and respond to an error instead of guessing after a failed query.
Single-table training transferred to harder multi-table tasks
The study compared training on single-table questions, training on a mixture of single-table and multi-table questions, and a curriculum that added multi-table tasks progressively. Single-table-only training produced the strongest uplift. That improvement also transferred to the harder FinQA reasoning benchmark, which contains multi-table questions. Performance there rose from 13.9% to 26.6%. Crawford says the result suggests that fixing a basic interaction behavior can improve performance on tasks that were not directly represented in the chosen training regime.
Rubric-based evaluation helps locate the behavior that needs training
A final pass-or-fail result is useful as the reward for RL, but it does not explain why an answer failed. Crawford recommends breaking evaluation into a rubric with separate checks for the different parts of a response. Those checks can reveal whether the model failed to find a table, inspect a schema, form a valid query, recover from an error, or calculate the answer. Teams can then choose training data around the observed failure instead of generating data for every possible behavior. GRPO can still receive a single value during the RL cycle.
"The 4 billion parameter model having been fine-tuned on this data set tries a table and actually first discovers the tables by using the tool get table names."14:40
Who should watch
You are deciding whether to deploy a large model or fine-tune a smaller self-hosted model for tool-driven financial analysis.
Your model gets the final answer wrong, but you do not know whether the failure comes from retrieval, tool selection, schema use, query construction, or error recovery.
You are building RL training data and need a way to choose examples based on observed behavior rather than on a broad pass-or-fail score.
