# The AI Evolution

Mario Rodriguez, GitHub | AI Engineer Summit 2023 | 19:32

Source: https://www.youtube.com/watch?v=yLa3i3rAdEE
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/the-ai-evolution
Published: 2023-11-07
Tags: copilots, developer-experience, evals, security

## TL;DR
- GitHub Copilot became useful when ghost-text suggestions, low latency, and the Codex model were combined in a simple coding experience.
- AI products need semantic understanding, offline and online evaluations, global deployment, and careful handling of security, legal requirements, and responsible AI.
- Mario Rodriguez expects development tools to move from editing procedures in text files toward working with goals, constraints, code reasoning, and collaboration between people and AI.

## Summary
Mario Rodriguez recounts how GitHub Copilot moved from an internal 2020 paper and an unsuccessful early PR experiment to a product used by more than 20,000 organizations and over one million developers. He attributes its early success to ghost-text suggestions, fast responses, the Codex model, and extensive product engineering. He argues that building an AI product at scale requires semantic evaluation, global infrastructure, careful security and data policies, legal preparation, and responsible AI work. Rodriguez then describes a future in which developers state goals and constraints instead of writing every procedure in sequence. Copilot would reason across unfamiliar code, summarize systems, and support collaboration between developers, AI, and other humans inside a redesigned programming environment. He places this work within GitHub's broader move from a code-hosting service to an end-to-end platform with repositories, Actions, Packages, Codespaces, and Copilot.

## Key ideas
### Copilot started with an internal question about whether an AI programmer was possible
[01:35](https://www.youtube.com/watch?v=yLa3i3rAdEE&t=95s)
Rodriguez traces Copilot's origin to an August 2020 paper circulated inside GitHub, titled "An Automated AI Programmer, Fact or Fiction." The paper created a debate because people saw the idea in opposite ways. GitHub first tried the product in the pull request flow, where it did not work well. The team later found traction and shipped Copilot in 2021. Rodriguez describes it as the first AI programmer deployed at scale. At the time of the talk, he says more than 20,000 organizations used Copilot and more than one million developers used it.

### Copilot's first successful experience combined ghost text, speed, and a strong model
[06:38](https://www.youtube.com/watch?v=yLa3i3rAdEE&t=398s)
Rodriguez identifies four components behind Copilot V1, beginning with ghost text. Suggestions could come from a natural-language comment or from ordinary typing, giving developers something to accept, reject, or revise in place. The system also had to be fast enough to keep people in the flow. He says Codex often produced suggestions in less than 100 milliseconds. GitHub later switched the system behind Copilot to GPT-3.5 Turbo. Rodriguez adds that prompt engineering matters, but turning a quick demo into a product used by millions requires much more engineering and continued iteration.

### Developer flow is a better measure of Copilot's success than lines of code
[05:43](https://www.youtube.com/watch?v=yLa3i3rAdEE&t=343s)
Rodriguez says GitHub builds Copilot for developer happiness. He does not define success mainly by the number of lines it generates. The product should remove boilerplate and other work that interrupts concentration, so developers can remain in a state of flow. He includes inline chat and future modalities in this idea, while warning that a chat box is not the whole product. His claim is that developers who are happier and able to stay in flow write better software.

### AI coding products need semantic understanding and production evaluation
[09:00](https://www.youtube.com/watch?v=yLa3i3rAdEE&t=540s)
Rodriguez warns that understanding programming syntax does not make an AI a developer. The system needs semantic work to generate the right code. He recommends offline and online scorecards because an evaluation can look good before deployment and then fail in production. GitHub also places deployments in Japan, Europe, and North America, with multiple data centers, to keep response times low for users around the world. His advice is to prepare evaluation systems early because teams need to iterate quickly once an AI product gains traction.

### Enterprise AI products have to earn trust through security, legal work, and responsible use
[10:49](https://www.youtube.com/watch?v=yLa3i3rAdEE&t=649s)
For products sold to businesses, Rodriguez names security, legal requirements, and responsible AI as areas that affect trust. He advises teams not to store prompts or customer data at rest and not to train on customer data without approval. GitHub provides indemnification to customers for certain issues, and Rodriguez points to the need to account for regulations in different countries. He also says teams must work to prevent the AI from harming users. His warning is direct: trust can grow slowly and disappear quickly after a serious failure.

### The next programming model will describe goals and constraints
[12:15](https://www.youtube.com/watch?v=yLa3i3rAdEE&t=735s)
Rodriguez argues that current coding is based on procedures, text files, and sequential programming. He wants Copilot to operate on goals and constraints instead. His example is a request to create an application using Remix, Redis, Postgres, a particular state machine and UI library, and a specified cloud. In that model, the programming environment has to change as well. Developers could spend more time evaluating architecture choices, vector databases, embedding models, and other parts of an AI system rather than spelling out every implementation step.

### Future coding tools need to reason across code and summarize it quickly
[14:27](https://www.youtube.com/watch?v=yLa3i3rAdEE&t=867s)
Rodriguez asks what it would mean for AI to reason about code in an unfamiliar codebase. He compares this with the brain's ability to summarize and form a search query when looking for how to add a feature flag. He says current AI is far from matching that ability. He also describes a visual example in which people perceive a gray car as blue because the brain supplies context. GitHub Next is exploring ways to break code into parts, reason over those parts, and use architecture advances, retrieval, and visualizations to help developers understand systems.

### AI collaboration requires a new interface for people, agents, and code
[16:23](https://www.youtube.com/watch?v=yLa3i3rAdEE&t=983s)
Rodriguez says most developer interfaces were not designed for an AI collaborator. The current pattern is an issue, a chat or discussion, coding with Copilot, a pull request, review, merge, and deployment. He wants a future interface that lets developers steer the AI and other humans working on the same code. It should let users ignore irrelevant material and verify the parts that need checking. This idea goes beyond code completion or a sidebar chat. It changes how people coordinate work with an AI inside the development process.

### GitHub plans to infuse its broader software platform with AI
[18:48](https://www.youtube.com/watch?v=yLa3i3rAdEE&t=1128s)
Rodriguez closes by describing GitHub as more than a home for open source or a version control system. He lists Actions for CI/CD, Packages, Copilot, and Codespaces as parts of an end-to-end platform. GitHub's direction is to add AI throughout that platform. He does not provide a detailed product announcement, but asks the audience to imagine an immersive experience where people can move in and out of work with AI and other humans. Security remains one of the future areas he says GitHub is considering.

## Notable quotes
- "The main point that I really want to make at the end over here is UX matters in what we're doing." (08:27)
- "Syntax is not software." (09:00)
- "The future of copilot really goes on operating on goals and constraints." (13:29)
- "Your brain is freaking phenomenal at that. The AI that we have today, not even close." (15:08)
- "It's really about advancing how we code and how the world codes together." (17:57)

## Tools & references mentioned
- GitHub Copilot
- GitHub
- OpenAI
- Codex
- GPT-3.5 Turbo
- An Automated AI Programmer, Fact or Fiction
- Bret Victor
- GitHub Next
- Remix
- Redis
- Postgres
- GitHub Actions
- GitHub Packages
- GitHub Codespaces
- Vercel

## Who should watch
- You are building an AI coding product and need practical lessons about latency, product UX, evaluation, and deployment.
- Your team is taking an AI tool to business customers and needs to think through data handling, indemnification, regulation, and responsible AI.
- You are interested in how programming environments might change when AI can reason over code and collaborate with developers.

## Related talks

- [GitHub Copilot: The World's Most Widely Adopted AI Developer Tool](https://aietalks.com/talks/github-copilot-the-worlds-most-widely-adopted-ai-developer-tool) (Dave Bernes, GitHub, 29:49)
- [GitHub Next Explorations](https://aietalks.com/talks/github-next-explorations) (Rahul Pandita, GitHub Next, 18:18)
- [Copilots Everywhere](https://aietalks.com/talks/copilots-everywhere) (Thomas Dohmke, GitHub & Eugene Yan, 18:22)
- [The Agent Awakens: Collaborative Development with Copilot](https://aietalks.com/talks/the-agent-awakens-collaborative-development-with-copilot) (Christopher Harrison & John Peck, GitHub, 1:04:06)
- [GitHub Copilot: The World's Most Widely Adopted AI Developer Tool](https://aietalks.com/talks/github-copilot-the-worlds-most-widely-adopted-ai-developer-tool-skituuev) (Christina Warren, GitHub & Dave, GitHub and Azure DevOps Teams & Alex, GitHub for Startups & Harold, Microsoft, 1:19:45)