Watch on YouTubeAI engineers can build useful systems by combining language models with software, since current models still need code and orchestration.
The AI revolution has a useful starting point in AlexNet in 2012, when increasing compute began to improve model performance at a large scale.
A 10x engineer can multiply their impact by teaching what they know to other engineers and building networks that help more people learn.
Swyx argues that people working on AI are arriving at the right point in history, rather than missing the important moment. He places the start of the current AI revolution around AlexNet in 2012 and points to the rapid growth of compute used to train models. He expects much more investment and much larger models over time. He also defines the emerging AI engineer role in relation to software engineering and machine learning research. His three categories are engineers enhanced by AI tools, engineers who build AI products, and engineering agents that can replace human engineers. Swyx says current language models still need software around them, because they are not artificial general intelligences. The talk ends with a broader definition of impact: a 10x engineer teaches ten other people what they know, then helps those people build wider learning networks.
Swyx compares the present with periods when particular fields were especially well placed for rapid progress. He names Brahmagupta's invention of zero around 600 AD, 1905 and 1927 for physics, 1900 to 1930 for cars, and 1980 to 2010 for personal computing. Against the familiar idea that people are born too late to explore Earth and too early to explore the stars, he argues that AI workers are
Swyx draws on Carlota Perez's work on technology revolutions and their installation and deployment periods. He says earlier cycles include the Industrial Revolution, the age of railways, heavy engineering and steel, oil, and the technology revolution. These periods roughly last 50 to 70 years. He acknowledges that people at the conference may be wondering whether AI is another speculative episode like Web3, then places today's work inside a pattern that has appeared across earlier industrial changes.
Swyx says it is normally difficult to identify the beginning of a change in human civilisation, but AI has a plausible marker in 2012, when AlexNet appeared. A chart in the talk connects that moment with a large increase in compute used for training models. The lesson he draws is that scale began to work, although people took too long to recognise it. He asks the audience to take scaling seriously as a continuing force behind language models.
Swyx gives several reasons for paying attention to six orders of magnitude. One unnamed investor imagines roughly six more magnitudes of compute by the end of the decade. John Carmack is cited on six important insights toward AGI. Swyx also repeats George Hotz's comparison that GPT-3 took about one person-year of compute while GPT-4 took about 100 person-years, then extends the progression toward GPT-10. His point is that people entering AI now will work on top of a very large technical trend.
Swyx frames the AI engineer as a role opening between machine learning and software engineering. He compares roughly 100,000 data science and machine learning engineers with a much larger population of software developers, citing GitHub's figure of 100 million registered developers while acknowledging that the true number is debatable. He expects the AI engineer population to be much larger than the machine learning engineer population. Language models still need code around them, including systems that coordinate and orchestrate them.
Swyx separates the role into three areas. The first is a software engineer enhanced by AI tools such as Copilot. The second is a software engineer building AI products, with Midjourney as his example. The third is an AI product that replaces a human engineer, with Auto-GPT and possibly Ghostwriter as examples. He compares the difference between enhancement and replacement with levels of self-driving, where the question includes whether a human remains in the loop or acts as a fallback.
Swyx names the categories the AI-enhanced engineer, the products engineer, and the engineering agent. He presents them as a possible career progression: first using AI to improve an engineer's work, then building AI products, then working toward agents that can carry out engineering work. He connects this idea to Amjad's recent talk and to Sam Altman's claim that OpenAI sees thousand-x engineers every day. The larger idea is a stack of improvements that can compound over time.
Swyx's preferred definition of a 10x engineer is someone who teaches ten other people what they know. He maps this onto network growth. Attending talks and consuming content grows participation linearly, while helping others learn creates wider connections. He also asks people to build their own networks of AI engineers after returning home. For him, the conference is about what attendees do with one another's knowledge, not only the tools and speakers on stage.
"LLMs themselves are not AGIs yet. We actually have to coordinate them in systems of software."05:46