Code generation has moved past syntax, so software engineers will spend more time on architecture, planning, security, and validation.
2
As human-written code becomes a smaller source of training data, models can use self-play to create, solve, and assess coding challenges.
3
The abundance of generated code may require new guardrails, evaluations, and programming languages designed for correctness rather than human readability.
Summary
Benoit Schillings argues that software development has entered an AI frontier where generating ordinary code is no longer the main difficulty. Earlier eras were limited first by machine performance and later by the human ability to manage software modularly. Models can now generate syntax well, while people still need to define the intended system, decompose complex problems, and judge architectural consequences. Schillings expects self-play to extend coding ability because models can create challenges, run code, verify answers, and assess some architectural choices. He also predicts that generated code will become nearly free and plentiful, making security, guardrails, reliability, and new evaluation methods more important. He suggests open-ended tests such as lossless code compression, multimodal reasoning with diagrams and dynamic representations, and languages influenced by strong typing or Lean. He sees code-based experimentation spreading into chemistry and biology, where models may find relationships and solutions that humans cannot perceive.
Software development moved from machine limits to human limits
Schillings describes three eras of software development. His early work with assembly language focused on extracting the last amount of power from constrained machines. As computing became cheaper, software teams could brute-force some problems, and the main limit became designing systems in modular, manageable ways. Libraries, functions, and decomposition helped people cope with that limit. Schillings says the human brain's ability to hold context shaped this process, while machine-learning systems are approaching much larger context windows. He argues that software has now entered an AI frontier where writing the code itself is no longer the main challenge.
Architecture and intent are harder than syntax generation
Schillings says frontier models have reached superhuman syntax generation for ordinary functions. The remaining difficulty appears when a model must work across a large existing codebase and make changes without losing the wider context. He uses a codebase with 35 million lines of PHP as an example of what software engineering involves in practice. Humans still need to decide what a system should do, understand the implications of a design, and manage concerns such as hardware optimization and security. He also gives humans an advantage in inductive thinking, where patterns across a broad system lead to a decision.
Self-play can replace some missing human coding data
Code has been unusually useful for model training because large amounts of it are available on GitHub and generated programs can often be checked by compilation, execution, or unit tests. Schillings says this source of human-written training data is reaching its limit, since he estimates that 80% of new code added to GitHub is machine generated. He points to AlphaZero as an example of self-play producing superhuman performance without human game knowledge. Coding models can create challenges, judge whether answers are valid, and sometimes assess architecture. Repeating this process gives models a way to train on increasingly difficult coding tasks.
Schillings says software engineering culture and infrastructure were built on the assumption that writing code was expensive. If code becomes free or nearly free, the amount produced will grow sharply, including code written dynamically. The harder problem becomes keeping systems adequate and reliable when people do not read every generated line. He predicts that within a year, teams may let Gemini or another model generate code without anyone inspecting it directly, much as programmers rarely inspect compiler-generated assembly. That shift requires new processes around active guardrails, security, and system-level validation.
Schillings describes vulnerability detection as an ongoing race. A model can find vulnerabilities in code, developers can patch them, and a stronger model can later discover more subtle flaws. He says teams need to think at least as much about the security implications of code as about producing it. His team is working toward models that write correct code from the start instead of merely detecting a vulnerability and suggesting a fix. He is direct about the difficulty because correctness depends heavily on context.
Schillings says current models are weak at transferring knowledge from one domain to another and at finding useful intersections between concepts. Complex software requires more than local code completion. Models need to plan before they act, inspect a difficult problem, and choose a decomposition that improves clarity or correctness. He calls this area inductive architecture. The goal is to give models a broader view of the system so they can make architectural decisions rather than only emit code one piece at a time.
Coding evaluations should test open-ended engineering
Schillings criticizes evaluations that only check whether code runs and produces the expected output. He says this measures only a small part of software engineering. He proposes more open-ended problems, using lossless text compression as an example. A model could receive a 10-megabyte code sample and be asked to write the best compressor, with the score based on the compressed file and the source code needed to create it. Such tasks could push models toward novel algorithms instead of rewarding solutions that satisfy a narrow fixed test.
Reasoning may need code, diagrams, and new languages
Schillings argues that reasoning should not be treated only as a chain of tokens or even a chain of code. He describes code writing as a visual activity involving block diagrams and the flow of data. Gemini was designed as a multimodal model, and he expects spatial and dynamic representations to become necessary for difficult problems. He also questions whether human languages such as Python are suitable for models that must produce safe and reliable software. Strong typing or ideas from Lean could place more of the correctness burden on the model, even if the resulting language is not easy for humans to read.
Schillings sees code generation spreading beyond software because experiments become much cheaper when models can write and run code quickly. He is especially interested in chemistry and biology. Humans understand only a small part of chemistry, and molecules with more than 20 atoms can already be difficult to predict. Biology has enormous amounts of engineering shaped by nature but poorly documented. Models may find relationships that are difficult for people to see. Schillings also argues that humans have strong biases about what a good solution looks like, which may leave entire areas of progress outside normal human intuition.
"I think that what we're starting to see is this ability to experiment very quickly in code is impacting other domain very quickly because doing experiment becomes basically free."17:40
Who should watch
You work on a large codebase and need to decide where human architecture and review still matter as generation improves.
Your team is building coding agents and needs ideas for self-play, security guardrails, or evaluations beyond passing unit tests.
You are designing programming languages or model interfaces for systems that must plan, reason across modalities, and produce reliable code.
