Off-the-shelf language models can give fluent financial advice without knowing what happened when similar businesses took the same actions.
2
Financial AI needs data that connects business states to actions and verified outcomes, rather than company context alone.
3
Intuit uses frontier models to suggest possible actions, then uses outcome-grounded models to select advice that is more likely to improve the business.
Summary
Udi Menkes argues that general-purpose language models often bluff when answering financial questions. They know what people have written about money, but they have not observed comparable businesses taking actions and seeing results. He illustrates this with rental properties, an egg supplier, and an apparel company, where generic advice could damage revenue or cash flow. Menkes describes Intuit's approach: turn financial records into business states, derive actions, track outcomes over different time frames, and train models on millions of state-action-outcome examples. Frontier models still help generate possible recommendations, while a model trained with reinforcement learning evaluates those candidates. Intuit tested this approach against leading models and found that a smaller, cheaper model could outperform them when it had better grounding. Menkes extends the argument beyond finance: useful domain AI depends on verified outcomes from a system's own data, along with an advisory experience that understands each customer's preferences.
Financial advice can sound reasonable while changing completely under small assumptions
Menkes describes asking several leading models whether he should invest in a particular real-estate niche or stock-market niche. He supplied extensive context, including books, knowledge, finances, and current models. The model recommended one option with convincing reasoning. After he changed one assumption, it switched to the opposite recommendation. A further small change sent it back to the first option. The advice was fluent and appeared sound, but Menkes could not trust it because the recommendation was so unstable. He uses this experience to frame the problem with off-the-shelf models and money.
Business context does not tell a model what similar businesses experienced
In Menkes's rental-property example, a new landlord has negative cash flow, an open loan, and declining profit. A frontier model with access to the business data recommends buying a second property to add income. An outcome-grounded model instead recommends raising the existing tenant's rent by 5 to 10 percent before renewal. Menkes gives another example involving an egg supplier whose largest customer produces 70 percent of its revenue and whose largest vendor accounts for nearly all costs. Raising prices by 15 to 20 percent could lose most of the revenue, while negotiating a 5 to 10 percent vendor-cost reduction addresses the cost side.
The fluent bluff comes from written advice rather than observed results
Menkes calls a generic, confident answer about money a 'fluent bluff.' Frontier models learned from blogs, books, advice columns, and other writing about money. They did not learn from watching comparable businesses make decisions and seeing what happened next. In Intuit's study of around 100,000 businesses and time frames, 40 percent of frontier-model advice centered on acquiring a new customer. Another 14 percent focused on increasing product revenue. More than half of the advice therefore pointed toward gaining customers or revenue, even when the business's constraints made those actions risky.
Long-horizon business simulations expose the limits of model fluency
Menkes cites research from Princeton in which leading models received tools and data and had to make business decisions across a 500-day simulation. Each model started with one million dollars. Most drove the company into bankruptcy before the simulation ended. A simple rules-based system outperformed almost all of the models. The few models that generated some profit did so only in particular runs, and repeated runs also led to bankruptcy. Menkes's explanation is that a frontier model has read about money, while an outcome-grounded model has observed what happens after decisions.
Causal comparisons are needed to measure whether an action caused an outcome
Menkes uses pricing to show why correlation is not enough. Suppose a restaurant raises prices and becomes more profitable six months later. The increase may reflect the price change, or it may reflect that the restaurant was already more successful. To estimate the action's effect, Intuit compares similar businesses with a similar likelihood of raising prices. In his illustration, businesses that raised prices gained $4,200 in daily profit, while those that did not gained $2,800. The naive difference is $1,400, but adjusting for the businesses' existing differences produces an estimated impact closer to $1,150. He names the measure conditional average treatment effect.
Intuit builds business trajectories from states, actions, and outcomes
Intuit combines data from QuickBooks, TurboTax, Credit Karma, and Mailchimp with financial records such as general ledgers, profit and loss statements, cash flows, and invoices. It turns this information into detailed business states at particular points in time. The system derives actions, such as running a marketing campaign, paying someone, or hiring. It then records outcomes over different time frames, including changes in profit, revenue, and cash flow. These state-action-outcome examples become millions of vectors. A model learns which actions led to better outcomes for businesses in similar situations, and an LLM then turns that evidence into advice.
Frontier models generate possibilities, while grounded models choose among them
Menkes says Intuit still uses frontier models, but for generating candidate hypotheses about actions a business might take. A model trained with reinforcement learning then evaluates those candidates and separates a useful move from one that could harm the business. Intuit tested its system head-to-head with leading models and says a midsize, cheaper model outperformed the frontier models because it was grounded in observed outcomes. The advantage came from the data and the training process, rather than access to a larger model.
A financial adviser needs evidence and an understanding of the customer
Intuit's AI business adviser, in research preview with customers, proactively raises opportunities and explains why it recommends them. The explanations are grounded in similar businesses, what they did, and what happened. Customers can drill into the reasoning and create action plans. Menkes says good advisory software also needs to understand a person's preferences, including what they like and do not like, so the customer can feel part of the decision. He presents this as part of a wider move toward outcome-driven AI, where teams use their systems of record to create domain data and train models toward desired outcomes.
"You don't close the gap with bigger models. You close the gap with experience, embedding experience into the model by looking at verified outcomes in your data."18:31
Who should watch
You are building a financial assistant and need it to recommend actions that account for business constraints, rather than repeat generic advice.
Your team has detailed operational records but has not linked decisions to later outcomes in a form a model can use.
You are choosing between investing in a larger general-purpose model and building a smaller system around domain-specific evidence.
