Large language models became widely useful when ChatGPT gave people an interface they could understand, even though chat remains a poor guide to the models' abilities.
2
Small, locally runnable models could support focused tasks such as summarization, extraction, and retrieval without needing a complete picture of the world or training on scraped internet data.
3
Prompt injection remains an unsolved security problem for applications that combine model instructions with untrusted input, so engineers need safer limits and sandboxes before giving models access to private data or local machines.
Summary
Simon Willison reviews the year in AI engineering through questions he still wants answered. He argues that ChatGPT's chat interface made language models accessible, while also hiding their capabilities and encouraging poor tests such as factual lookup and arithmetic. Llama, Alpaca, and the growing homebrew model ecosystem opened up local experimentation, raising questions about how small a useful model can be and whether models can be trained on public-domain or openly licensed material. Willison demonstrates command-line workflows that connect Hacker News, jq, Claude, SQLite, embeddings, and local models. He also explains why retrieval-augmented generation is easy to prototype and difficult to make work well. Prompt injection is his most serious concern: malicious text can manipulate an assistant that has access to email or other private data. He is enthusiastic about code interpreter as a coding assistant, but wants safer local sandboxes, possibly using WebAssembly. He closes by arguing that these tools can help both experienced and new programmers automate tedious work.
ChatGPT made language models understandable by changing the interface
Willison says large language models became much easier to grasp when OpenAI put a chat UI on an existing model on November 30. Earlier interfaces required people to understand completion prompts or use an API, and many users did not stay with them. ChatGPT made the capabilities immediately visible, but he also thinks chat is a poor interface because it gives users no hints about what models can do. People often judge the technology by asking for arithmetic or factual lookup, tasks language models handle badly. He wants interface ideas that guide users toward the models' useful abilities instead of dropping them into what he calls the "Shark Tank."
Local models turned language-model experimentation into a homebrew activity
The release of Llama changed Willison's expectations about local models. He had assumed that running a language model would require a rack of GPUs and that the important systems would remain inside closed research labs. Llama ran on his laptop, and llama.cpp made it practical to experiment. Stanford's Alpaca then showed that a model could be fine-tuned to answer questions more like ChatGPT with about $500, including compute and GPT-3 tokens used to create its training set. Willison describes this as a Stable Diffusion moment for language models, followed by a rapid increase in academic research and home experimentation.
Useful small models may only need focused language tasks
Willison's main question about the homebrew movement is how small a model can become while remaining useful. He does not need a system that knows every monarch of France or every state capital. He wants something that acts as a "calculator for words," with the ability to summarize text, extract facts, and answer questions through retrieval-augmented generation. He points to 3B models as the smallest size where people are doing interesting work and mentions a newly released Replit model that was reportedly performing well. The question is how far model size can fall while these focused abilities remain available.
Training on openly licensed material could avoid copying the internet
Willison questions whether language models can be trained entirely on public-domain or openly licensed data. He says most models use at least a large scrape of internet content that its authors did not necessarily expect to become training data. Adobe's Firefly showed that an image model could be trained on licensed stock photography, although stock photographers felt they had been surprised by that use. Willison wonders whether Project Gutenberg, United Nations documents, and other out-of-copyright material contain enough tokens to train a model that can perform the tasks he cares about without taking content from much of the internet.
Command-line pipelines make language models useful as connected tools
Willison's llm project began as a command-line tool for running prompts and logging every prompt and response in SQLite. Plugins later added support for API models and local models. He demonstrates a Bash workflow that fetches Hacker News comments, flattens the JSON with jq, and sends the result to Claude's 100,000-token context for a thematic summary with attributed quotes. He can check those quotes against the source material. He also uses embeddings to retrieve relevant paragraphs from his blog and pipes them into Llama 2 7B Chat on his laptop. He thinks these Unix-style pipelines deserve more attention than isolated notebook experiments.
Retrieval-augmented generation is easy to start and hard to make reliable
Willison calls retrieval-augmented generation the perfect Hello World application for language models because a basic version is easy to build. Making one work well is much harder. His blog example retrieves paragraphs similar to a user's query and passes them to a local Llama 2 model with a prompt requiring a single-paragraph answer. The open question is which patterns work across different domains and data shapes. He expects the engineers at the conference to help discover those patterns, since many of them are working on the same problem.
Prompt injection attacks the application around the model
Willison explains prompt injection as an attack on an application that combines instructions with untrusted input. An application might tell a model to perform an action on text supplied by a user, allowing that text to introduce new instructions. He compares the idea with SQL injection, but says parameterizing and escaping queries does not solve prompt injection. His example is an assistant named Marvin that can read and reply to email. A malicious email could tell Marvin to search for password-reset messages, forward them to an attacker, delete the forwards, and hide the evidence. Willison has not seen a convincing solution after 13 months of discussion.
Code-generating assistants need sandboxes before they can run freely
Willison is enthusiastic about ChatGPT Code Interpreter, which OpenAI renamed Advanced Data Analysis and which he jokingly calls "chatty coding intern." It can write Python, run it, and fix bugs, and he found that its environment could also compile C. He used it from his phone and while walking his dog, asking it to research SQLite triggers and produce tested code. He wants this ability on his own machine, but prompt injection makes it unsafe to run generated code directly. His open question is how to build a local sandbox for code that may do harmful things. His current hunch is that WebAssembly may provide the isolation.
AI assistance lowers the barrier for new programmers
Willison rejects the idea that AI tools only help experienced programmers. He says the first three to six months of learning programming are often dominated by small errors and confusing messages, and many people quit during that period. A coding assistant can shorten that struggle and make useful work possible sooner. He also knows former programmers who returned to coding because they could get work done in about half an hour a day instead of spending four hours rebuilding their development setup. For him, the most positive possibility is giving people a way to automate tedious tasks without requiring a computer science degree.
"We need to be 100% sure that this isn't going to work before we unleash these AI assistants on our private data."16:57
Who should watch
You are building applications that combine language models with user-supplied or retrieved text and need to understand prompt injection before adding more capabilities.
You want to run models locally, compare focused small models, or connect model calls through command-line tools and Unix pipes.
You are learning to program or returning to it and want a grounded account of where coding assistants help, along with the risks of running generated code.
