A model is only faster for an analyst when a usable, source-backed report arrives sooner.
2
Reducing attention work, cache precision, expert precision, or reasoning can lower compute while introducing retrieval, accuracy, or waiting-time costs.
3
A model switch should be judged with the same reports, full production costs, user wait targets, saved settings, and repeatable checks.
Summary
Morgan Suo uses a financial research task to test whether a faster model improves the finished job. The task asks whether circuit-board suppliers are turning AI server demand into cash, using 14 company reports. A useful answer must cite the right sources, align reporting periods, keep companies separate, explain the numbers, and let an analyst recalculate important claims. Suo describes changes in MiniMax M3.1 Flash, including sparse attention, a 4-bit KV cache, low-precision experts, speculative decoding, serving adjustments, and reasoning controls. Each change can reduce computation, but each needs a check against evidence, accuracy, or wait time. She argues that throughput and average scores are insufficient. Evaluation should include retrieval, tool calls, retries, report correction, concurrency, usable-report latency, and the selected reasoning budget. The comparison should use fixed sources and acceptance criteria, record model and runtime settings, preserve failures, and support a small rollout with a way to switch back.
A finished research report must be checkable by its reader
The research task asks whether companies supplying circuit boards to AI servers are turning demand into cash. It uses 14 company reports and requires the model to find the right documents, align reporting periods, keep businesses separate, and explain the numbers. An analyst must be able to open the source behind an important claim and recalculate a result. Suo uses profit versus cash flow to show why this matters: profit can rise while customers have not paid or materials remain in storage. The report therefore needs the relevant cash flow statement and disclosures, not just a positive profit figure.
Sparse attention reduces long-context work while risking missed evidence
Sparse attention first scores blocks of a long input, then applies the main attention calculation to selected blocks while keeping nearby text in view. Different questions can select different passages. This lowers attention work across many documents, but a short footnote can matter more than pages of industry outlook. Suo says the model uses this attention approach in its first three layers and describes a retrieval test with 39 requests, including inputs of about 830,000 tokens. The research evaluation should move important passages to different positions and check whether the model finds cash flow footnotes, uses the correct reporting period, and notices conflicts.
Averages can hide errors caused by a lower-precision KV cache
M3.1 Flash changes the keys and values in its KV cache from 8 bits to 4 bits, while the query remains at 8 bits. This reduces the data that must be stored and moved. Suo says the average score in the comparison is almost unchanged, but one score drops by 6.6 percentage points while other scores rise. That average can conceal a mistake that matters to a user. The comparison also does not isolate which model change caused each result. For the research task, the output still needs checks for source accuracy, calculations, and whether the explanation matches the evidence.
Low-precision experts save movement only when the calculation order stays correct
M3.1 Flash uses a mixture-of-experts architecture and selects some experts for each token. The selected experts use 4-bit weights and activations, while the shared expert is outside this change. Lower precision can reduce data movement and make computation more efficient, but the system must scale the numbers, run the calculation, and restore the scale. The placement of those steps affects the answer. Suo's test therefore measures the complete implementation rather than only the speed of the arithmetic. On the same research questions, the team inspects why an answer still misses source, calculation, or explanation checks.
Speculative decoding needs to be measured through the usable report
Speculative decoding uses a faster draft process to propose a short sequence of tokens. The target model checks the proposal, keeps accepted tokens, corrects rejected ones, and continues. Several tokens can come from one target check, but drafting, checking, and rejecting all take time. Suo says the timing must include source retrieval, tool calls, and retries for the research workflow. The useful measure is the time until an analyst receives a report they can use, rather than only the model's token generation speed.
Higher throughput can make the user's wait much worse
In a serving test with low reasoning effort, concurrency rises from 32 to 64. Total output per second grows by about 1.9 times, while P95 time to the first token rises from about 1.5 seconds to 11.3 seconds. Suo describes that as roughly eight times longer for the wait at that percentile. The median does not capture the full user experience, and the first token may contain reasoning rather than a usable report. The evaluation should set a deadline for the completed report and measure when that report arrives.
Reasoning budgets trade output length against task accuracy
The reasoning control determines how many tokens the model uses for thinking. If the effort setting is left blank, the default is maximum in the document setup Suo shows. In one coding test, the high setting uses fewer output tokens than the maximum setting, while accuracy falls from 80% to 68% on the task items. A research workflow should compare settings on the same reports and check evidence retrieval, calculations, and conclusions. A quick first pass and a report prepared for review may need different budgets, so the chosen control must be recorded for later runs.
A model switch should count the whole cost of an acceptable report
Suo's evaluation starts with fixed source reports, the same information cutoff, and an agreed definition of an acceptable report. The team checks whether analysts can open sources, verify reporting periods, recalculate financial figures, and find evidence that could change the conclusion. Cost and time include retrieval tools, retries, failed attempts, and any human work needed to repair the report. The test should use the expected number of users and check whether reports arrive within the promised time. Repeatability requires saving model and runtime versions, prompts, settings, sources, and failure examples, then starting with a small rollout and a way to switch back.
