# Pick Problems That Survive When AI Solves Coding

Ara Khan, Cline | AI Engineer | 26:49

Source: https://www.youtube.com/watch?v=iK9gmxXyBR0
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/pick-problems-that-survive-when-ai-solves-coding
Published: 2026-11-11
Tags: agents, coding-agents, evals, harness-engineering, inference

## TL;DR
- Recursive self-improvement can improve an agent harness by repeatedly evaluating failures, changing the harness, and rerunning the evaluation.
- Any part of a system with a verifiable feedback loop, from evaluation and training data to inference and hardware, can become part of an improvement loop.
- Engineers should choose problems that benefit from better models without being completely solved by the next model release.

## Summary
Ara Khan explains recursive self-improvement through the example of a coding agent harness that improves its own performance on software evaluations. A model examines failed tasks, groups the failures, changes the harness, and runs the evaluation again. Khan says this process can extend beyond the harness to evaluation design, training data, post-training, inference kernels, physical hardware, and networking. He argues that many software products are vulnerable to being made obsolete by one frontier-model release, so engineers should reconsider what they build. His advice is to move closer to the improvement loop and choose work that benefits from better models while also contributing to the systems that make those models better. The talk is partly technical and partly career advice. Khan is open about the uncertainty of the timelines, but he treats the feedback loops as already present in several parts of the AI stack.

## Key ideas
### Recursive self-improvement means a system helps build a more capable successor
[05:01](https://www.youtube.com/watch?v=iK9gmxXyBR0&t=301s)
Khan defines recursive self-improvement as an AI system helping build a more capable successor, which repeats the process. A self-improving coding agent is the practical example: the agent improves the system that runs the agent. He connects this idea to the development of software assistants. A 2021 roadmap described a general AI assistant that could create, edit, and explain software, identify bugs, design APIs, and discuss style and best practices. ChatGPT arrived in 2022. Models gained function calling in 2023 and 2024, then began using tools such as file access, editing, web search, and shell commands. Khan presents multi-agent systems and self-improvement as later steps in that progression.

### Agent evaluation has to measure completed work, not just a single answer
[08:05](https://www.youtube.com/watch?v=iK9gmxXyBR0&t=485s)
Khan contrasts simple question-and-answer grading with the work required from a coding agent. An agent asked to set up an MCP may need to search for documentation, edit configuration, download files, update a database, read and change source files, and run shell commands. The evaluation must check whether the requested task was actually completed. He describes Terminal-Bench as a benchmark for agents in terminal environments. Tasks run in separate sandboxes, with graders examining the result, number of turns, token use, latency, tool calls, and the trajectory. The graders also check for reward hacking, such as looking up a solution instead of solving the task.

### Failure buckets turn evaluation into a repeatable improvement loop
[16:59](https://www.youtube.com/watch?v=iK9gmxXyBR0&t=1019s)
Khan describes an evaluation cycle that runs many tasks in parallel and produces an aggregate score plus a set of failures. In his example, Terminal-Bench 2.1 contains 89 tasks, so the evaluation starts 89 independent sandboxed containers. Failures can come from a broken file-reading tool, slow inference, an agent running in circles, or a task timing out. The team groups failures into categories and changes the harness to address them. This is hill climbing: run the benchmark, inspect the failures, make a change, and run it again. The process turns vague model quality into concrete engineering work on tools, time limits, loops, and task execution.

### A model can operate as a meta-agent that improves the harness
[20:00](https://www.youtube.com/watch?v=iK9gmxXyBR0&t=1200s)
Khan says his team used a long-running agent to automate the hill-climbing process. One model ran evaluations, examined failures, changed Cline's harness, and started the evaluations again. The process ran for 17 hours without his direct interference. It found issues including loop detection, background processes that ran too long, and file-index problems. Khan describes this as two learning loops. The main harness acts in an environment and receives a reward, while a meta-agent observes the harness and proposes changes. He warns that real-time improvement can overfit to a narrow problem, so changes should be justified by how well they apply across a wider range of tasks.

### The improvement surface includes inference and the rest of the stack
[23:57](https://www.youtube.com/watch?v=iK9gmxXyBR0&t=1437s)
Khan argues that a model improving a coding harness is one example of a larger pattern. He cites a post from OpenAI claiming that a model autonomously rewrote and optimized production kernels, reducing end-to-end serving costs by 20 percent and raising token-generation efficiency by 15 percent. He also mentions kernel-writing competitions where agents produce optimized GPU operators. From there, he expands the improvement surface to evaluation, harnesses, training data, post-training, world interaction, and inference. His claim is that every stage from electricity entering a machine to a model response can potentially be optimized when the system has a measurable feedback signal.

### Human corrections reveal the edge of current model capability
[17:30](https://www.youtube.com/watch?v=iK9gmxXyBR0&t=1050s)
Khan treats each correction a person gives to a model as evidence of a capability gap. He uses a database example: an agent adds a column and fills it, but configures a cache incorrectly. A human correction fixes the cache settings. Khan says this correction is a delta that should be covered by future training or evaluation. The same idea applies to long-horizon coding tasks, where a model may complete most of the work but fail on one difficult part. Post-training can use these hard, verifiable problems to push the model's capability frontier gradually. The feedback loop depends on being able to tell whether the attempted change worked.

### Physical hardware can also enter a recursive improvement loop
[19:09](https://www.youtube.com/watch?v=iK9gmxXyBR0&t=1149s)
Khan gives a hardware example involving a small device with a camera, speaker, and microphone. He connected it to a computer and let coding agents send tasks to the device, measure its end-to-end response time, and identify cases with high latency or poor algorithms. He says the loop could update the firmware, tool calls, inference model, and speech-to-text model. Overnight experiments then improved the device across those components. The example extends recursive improvement beyond software repositories. A physical system can participate when its behavior can be measured, its changes can be made automatically, and experiments can be repeated.

### Engineers should choose work that benefits from scale without being erased by it
[21:56](https://www.youtube.com/watch?v=iK9gmxXyBR0&t=1316s)
Khan's career advice is to move closer to the recursive improvement loop. He is skeptical of software products that are one model release away from being obsolete, especially products that amount to repeatedly asking a frontier model to perform routine tasks. He recommends work connected to inference, post-training, physical interaction, hardware, networking, data pipelines, and other parts of the end-to-end system. The right problem benefits from better models, but is not solved by scale alone. He also advises shorter timelines, ambitious targeted collaborations, and reflection about whether a difficult project should be pursued now or left for the next model release. The choice is about durable contribution rather than protecting a familiar job description.

## Notable quotes
- "Recursive self improvement is the idea that an AI system that helps build a more capable successor which repeats the process." (05:01)
- "Anything that has a verifiable loop can be optimized, but this is not just limited to software." (18:51)
- "Every single time you're finding the edge of model capabilities, you want to find the delta and we want to push that frontier a little bit further." (17:58)
- "You want to work on problems that benefit from scale, and you also at the same time want to avoid projects that can be solved by scale alone." (25:01)

## Tools & references mentioned
- Cline
- OpenAI
- Google DeepMind
- ChatGPT
- AutoGen
- Terminal-Bench
- Terminal-Bench 2.1
- MCP
- Kimi K3

## Who should watch
- You are building coding agents or evaluation harnesses and need a concrete pattern for turning agent failures into engineering changes.
- Your software product may be easy for a stronger frontier model to reproduce, and you want a framework for choosing a more durable problem.
- You work on inference, post-training, hardware, data, or other parts of the AI stack and want to see how they can connect through measurable feedback loops.

## Editor's note

Ara Khan describes an evaluation cycle where "Failures can come from a broken file-reading tool, slow inference, an agent running in circles, or a task timing out." Kitaru records every model call and tool result from a real agent run as a session. Replay then runs the changed harness against those same inputs and tool responses, without touching real systems.

Written by the AIE Talks editors (the Kitaru team), not by the speaker.

## Related talks

- [Hard Won Lessons from Building Effective AI Coding Agents](https://aietalks.com/talks/hard-won-lessons-from-building-effective-ai-coding-agents) (Nik Pash, Cline, 14:18)
- [Why (Senior) Engineers Struggle to Build AI Agents](https://aietalks.com/talks/why-senior-engineers-struggle-to-build-ai-agents) (Philipp Schmid, Google DeepMind, 10:40)
- [We Kept Rebuilding Our AI Agent. Here's What Survived](https://aietalks.com/talks/we-kept-rebuilding-our-ai-agent-heres-what-survived) (Fuad Ali, Arize AI, 20:06)
- [Continual Learning for AI Agents: From Failures to Durable Improvements](https://aietalks.com/talks/continual-learning-for-ai-agents-from-failures-to-durable-improvements) (Soheil Feizi, RELAI, 22:35)
- [Break It 'Til You Make It: Building the Self-Improving Stack for AI Agents](https://aietalks.com/talks/break-it-til-you-make-it-building-the-self-improving-stack-for-ai-agents) (Aparna Dhinakaran, Arize AI, 14:25)
