# The Unreasonable Effectiveness of Separating the Task from the Model Maxime Rivest, DSPy & Isaac Miller, DSPy; cmpnd | AI Engineer World's Fair 2026 | 17:11 Source: https://www.youtube.com/watch?v=GgLQ02aO-hs Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/the-unreasonable-effectiveness-of-separating-the-task-from-the-model Published: 2026-07-23 Tags: code-generation, cost, evals, open-source ## TL;DR - A fixed interface for an AI task lets teams change models, prompts, agents, tools, and reasoning without changing the surrounding workflow. - DSPy describes tasks with instructions, code-based constraints, and evaluations, then uses those definitions to optimize the implementation. - DSPy 4.0 previews code-generating modules and qualitative learning, which uses feedback from production to improve evaluations over time. ## Summary Maxime Rivest and Isaac Miller argue that AI programs should be built like reusable software functions. A task gets a defined input and output interface, while the model, prompt, reasoning method, tools, and other implementation details can change behind that boundary. DSPy expands the task definition with instructions, code that enforces requirements, and evaluations that describe what good performance means. This gives teams a basis for testing and optimizing different implementations. Miller describes enterprise benefits, including a Shopify case where the same evaluations supported a move to a cheaper model and a reported 550-fold cost reduction. The talk also covers recursive language models and previews DSPy 4.0 features. DSPy Flex can learn a custom harness for a function, while qualitative learning aims to turn user feedback, traces, and product data into evaluations. The speakers are clear that new techniques may or may not help a given task. Teams still need to measure them against their own business problem. ## Key ideas ### AI tasks gain software properties when their inputs and outputs are fixed [00:34](https://www.youtube.com/watch?v=GgLQ02aO-hs&t=34s) Maxime Rivest compares an AI task with an ordinary programming function. A function has a name, inputs, outputs, and implementation logic, so it can be reused, composed, tested, optimized, packaged, and distributed as a black box. DSPy brings those properties to AI workflows through an open-source Python library. A fixed interface lets a team replace the implementation without changing the places that call it. Rivest gives examples including extracting tax values from farm invoices, correcting grammar from clipboard text, and rewriting text for clarity. ### A stable task boundary lets engineers change the implementation behind it [01:58](https://www.youtube.com/watch?v=GgLQ02aO-hs&t=118s) Rivest says a repeated AI task can have an input interface and an output interface while its internals remain flexible. An invoice extractor or rewriting command can use a new model when one becomes available. Larger tasks fit the same pattern. An inbox plus a new email can produce a drafted reply, and a software specification plus a repository can produce a pull request. The implementation can begin as a simple prompt, then become an agent, gain tools, or use loops. The integration outside the boundary stays the same. ### A complete DSPy task description combines instructions, code, and evaluations [04:47](https://www.youtube.com/watch?v=GgLQ02aO-hs&t=287s) Rivest explains that a simple signature does not fully specify a task for automatic optimization. Instructions describe what should happen, including natural-language directions and the input and output types. Code expresses requirements that must be enforced. In his tax example, a failed basic extraction triggers a more deliberate second pass, and a negative value is sent to a human. Evaluations describe what good performance looks like, including long-tail behavior that may be difficult to write as instructions. Together, specs, code, and evals define the goal. ### Optimization has moved from finding examples to improving instructions and implementations [08:15](https://www.youtube.com/watch?v=GgLQ02aO-hs&t=495s) DSPy initially used code to find few-shot examples because early models could not optimize well. As models improved, DSPy could optimize instructions. Rivest describes a longer-term direction in which more implementation details are delegated away while the task definition remains stable. Miller connects this approach to enterprise cost control. He cites Shopify using the same evaluations while moving from an expensive model to a cheaper one, with a reported 550-fold reduction in cost. The business logic could continue to change without rebuilding the evaluation setup. ### New techniques are useful when they can be tested without changing the signature [10:27](https://www.youtube.com/watch?v=GgLQ02aO-hs&t=627s) Miller says DSPy adds techniques that may solve subproblems, but none is guaranteed to solve a particular application. He uses recursive language models from Alex Zeng's work as an example for long-context programs. The technique can be brought into DSPy with one line while the signature stays unchanged. He also names RLLMs, Jeppa, multi-module GRPO, and other research contributions as approaches users can try inside their implementations. The practical test remains whether a technique improves the user's own task. ### DSPy Flex can learn a custom harness for a function [11:51](https://www.youtube.com/watch?v=GgLQ02aO-hs&t=711s) Miller previews DSPy Flex, a DSPy 4.0 module that moves beyond optimizing few-shot examples and prompts. For a function that needs an implementation, the system can learn a harness over time. The harness is custom to the function, and the user does not need to care about its internal form as long as it solves the business problem. This depends on having the three task ingredients in place: a specification, code that expresses requirements, and evaluations that provide a way to measure the result. ### Qualitative learning aims to turn production feedback into evaluations [12:36](https://www.youtube.com/watch?v=GgLQ02aO-hs&t=756s) Miller describes evaluation building as a difficult part of AI engineering. A binary judgment such as whether an email is good contains less information than feedback about what should change. A dataset evaluation is also a proxy for reality. Qualitative learning asks whether reality can inform evaluations automatically. Production traces, user actions, product analytics, and questions about data representation could provide textual feedback. Miller says models may be able to interpret that feedback, turn it into an evaluation signal, and refine the target over time. ### A highly intelligent model still needs to learn a user's task and context [14:26](https://www.youtube.com/watch?v=GgLQ02aO-hs&t=866s) Miller separates intelligence from knowing everything about a user's situation. Even an extremely capable model would not automatically know how to solve a specific business problem or interact with the relevant people. He uses Albert Einstein helping with email as an example: Einstein might not know what email is, while a future general model would know the concept but still need to learn the user's relationships and working context. DSPy's last-mile learning work focuses on acquiring that context efficiently through the task interface and feedback over time. ## Notable quotes - "Functions are reusable, composable, testable, and optimizable." (00:34) - "If for your repeated AI task, you define an input interface and an output interface, you get to play in the internals." (02:14) - "What we can do is we can solve sub problems for you that make your implementation easier." (10:25) - "Intelligence is very different from being all-knowing." (14:26) - "You should define your problem, and you should hold your prompts, models, and code accountable to the problem that you need them to solve." (15:35) ## Tools & references mentioned - DSPy - DSPy 4.0 - DSPy Flex - recursive language models - RLLMs - Jeppa - multi-module GRPO - Shopify - Alex Zeng - MIT - Berkeley - AlphaGo - AlphaZero - Discord ## Who should watch - You are building an AI workflow that keeps changing as new models, prompts, tools, or agent patterns appear, and you want those changes isolated from the rest of the application. - Your team needs to compare model costs or implementation strategies against the same business requirements and evaluations. - You are struggling to define evaluations for subjective tasks and want to understand how production feedback could improve them over time.