Frontier models have improved from losing instructions around 200 to 300 rules to handling roughly 2,000, with the best model reaching about 5,000 in this test.
2
Models now fail in different ways: they forget instructions, refuse the request, spend their reasoning budget on compliance, or produce a polished partial answer before quitting.
3
Long skills files are no longer mainly a compression problem, but teams still need output checks because instruction-following capacity does not guarantee reliable results.
Summary
Laurie Voss reran the IFScale benchmark, which asks a model to include an exact list of words in a business report. The original result found that frontier models began losing instructions around 200 to 300 rules. When Voss applied the same test to newer models, they scored perfectly at the old 500-word ceiling, so he expanded the test to 10,000 words. The new boundary appeared around 2,000 instructions, and one model reached about 5,000. The models also failed differently. DeepSeek forgot rules, Claude refused requests that triggered its safety classifier, Gemini used its thinking budget until little room remained for an answer, and GPT-5.5 began the report before stopping and criticising the task. Voss says this changes prompt design: engineers can stop compressing every skill file into a tiny instruction budget, but they must verify outputs. He also stresses that IFScale is only a proxy and that other research finds reliability can fall when instructions are reordered or when inputs become long and complex.
The old 200-instruction limit came from a measurable benchmark
Voss traces the old ceiling to IFScale, a benchmark from a 2025 paper. The test gives a model a business-report task plus a list of exact words it must include, such as "customer" and "revenue." Each required word counts as one instruction. The score is the percentage of requested words that appear in the output. Voss argues that this is a useful proxy for more complicated constraints, such as including a pricing section or avoiding a specific phrase. If a model cannot reliably track 200 named words, he expects it to struggle with 200 richer instructions. The benchmark therefore gives a ceiling, rather than a promise that real skills files will work at the same size.
The original IFScale result held up when Voss reran it
Voss tested the three models from the original study that were still available through an API: GPT-4.1, Claude Sonnet 4, and Gemini 2.5 Pro. Their accuracy curves matched the earlier results within the noise boundary. At around 500 rules, the models were losing roughly 30 to 50 percent of the requested words. The older frontier models began falling apart somewhere around 200 to 300 rules. Voss also notes that one of the three models was retired after he published the research, leaving an unusually short window in which the original comparison could be repeated.
Newer models passed the old ceiling and forced a larger test
Voss applied the same prompt and word lists to GPT-5.5, Claude Opus 4.7, Gemini 3.1 Pro, and DeepSeek V4 Pro. All four scored 100 percent immediately, so the 500-word benchmark could no longer find their limits. He doubled the vocabulary from 500 to 1,000, then to 2,000, continuing until he reached 10,000 words. The curves finally began to bend around 2,000 instructions, with the best result extending to about 5,000. Voss describes this as close to a tenfold improvement in simultaneous instruction following over roughly twelve months. He says prompt and skills-file assumptions made more than about six months earlier may already be outdated.
The models reached their limits through four distinct failure modes
DeepSeek V4 follows the old pattern: it starts forgetting around 750 rules and drops nearly half by 2,000. Claude Opus 4.7 often refuses at the API level because random word combinations trigger its safety classifier. Voss filtered the word list through OpenAI's safety filter to get Claude to run the test. Gemini 3.1 Pro performs well to about 5,000 instructions, then spends its thinking budget checking the rules and leaves too little room for an answer. GPT-5.5 reaches 99 percent accuracy to 5,000 rules, then begins a report before deciding that the request is stupid and stopping. The last failure is difficult to detect because the output can look complete until its ending.
Long skills files can replace some prompt compression and sharding
Voss says the old advice was to keep each skills file under about 200 instructions and distribute the rest across subskills, subfiles, or specialised agents. That was a way to compress instructions into a small available budget. With models handling about 2,000 named constraints, a team can put a large style guide, brand rules, or legal disclaimers into one prompt instead. The trade-off has changed. Ten thousand instructions may fit, but the prompt becomes expensive and slow. What used to be a hard capacity wall is now a cost and latency decision, and the model used still matters because the tested limits ranged widely.
IFScale shows capacity, not reliable reasoning over a long prompt
Voss is explicit that including random words in a fake business report is evidence for a long skills file, not proof that a model will reason well over one. The test does not measure whether a model resolves conflicts between instructions or reasons clearly across a large body of text. He cites Chroma's context-rot work across 18 models, which found that accuracy on long inputs can fall 30 to 50 percent before the context-window limit. That research also found coherent, well-structured text more likely to hit the failure mode than shuffled instructions. The benchmark's range is wide too, with models failing anywhere from about 750 to more than 9,000 instructions.
Verification is now the harder engineering problem
Voss says the capacity increase does not make outputs trustworthy by itself. Claude's refusal is loud because the API reports an error, but GPT-5.5's partial report can look like a real answer until the reader reaches the end. A model may accept 2,000 rules and return polished text while quietly abandoning the task. Voss says production systems therefore need output monitoring, with another LLM acting as an evaluator when appropriate. He also cites a paper testing 46 models that found small changes in wording or instruction order could radically change compliance. The model may handle the same rules in one arrangement and fail them in another.
Instruction compression gave way to output checking
Voss's final distinction is between compression and verification. A year earlier, the main problem was fitting skills into a small instruction budget without the model losing track of them. That compression problem has largely receded for the models he tested. The remaining problem is checking whether the requested work actually happened. He compares this with testing ordinary code: a better prompt cannot prove that the output is correct. An eval has to inspect the result each time. He also mentions newer benchmarks, including Firebench, CCRBench, and GuideBench, which test instruction following with larger sets of messy constraints.
"A verification problem doesn't get solved by writing a better prompt. It gets solved by checking the output every time."21:21
Who should watch
You maintain skills files or agent prompts that have grown beyond a few hundred separate rules and need to know whether old size limits still apply.
Your system accepts long model outputs and has no reliable way to detect a polished partial answer or a silent failure.
You are choosing a model for a constraint-heavy workflow and need to distinguish instruction capacity from reliable reasoning over long, complex inputs.
