GPT-4-class models now have serious competition, including strong open models and inexpensive alternatives.
2
AI tools are power-user tools because their behavior depends on details that providers often do not document.
3
AI engineering has to address trust, prompt injection, and unreviewed generated content before these systems can be used responsibly.
Summary
Simon Willison argues that AI engineering has moved past the period when GPT-4 had no serious competition. Models from Anthropic, Google, Meta, Nvidia, Cohere, Alibaba, and DeepSeek now occupy similar or lower-cost parts of the market, while some GPT-4-class access is free to consumers. The harder problems are practical and social. ChatGPT can handle long searchable PDFs, but its treatment of tables, diagrams, and files depends on hidden implementation details. AI products also face a trust crisis because users often cannot tell how their data is handled. Willison shows how a markdown image can silently exfiltrate private data through a chatbot, and argues that prompt injection must be understood by engineers. He defines slop as unrequested, unreviewed AI-generated content. His answer is for builders to establish responsible usage patterns and help other people develop the judgment to review and take responsibility for generated material.
Willison starts with the change in model competition since GPT-4 launched. For about a year, GPT-4-class systems were clearly ahead of other language models. His updated comparison groups Gemini 1.5 Pro, GPT-4o, and Claude 3.5 Sonnet among the strongest models, while Claude 3 Haiku and Gemini 1.5 Flash offer much lower-cost options that can still do substantial work. He says GPT-3.5 Turbo is now both relatively expensive and poor by comparison. The practical result is that engineers have more viable models to choose from instead of defaulting to OpenAI.
Benchmark scores do not measure how models feel to use
The MMLU benchmark is popular because many model providers report results on it, but Willison says it mainly tests broad trivia knowledge. He gives a question about Type II-A supernovae and points out that most AI applications do not require that kind of knowledge. For a more practical comparison, he uses the Chatbot Arena, where people anonymously compare model responses to the same prompts and rankings are calculated with an Elo-style system. That ranking places Claude alongside GPT-4-class models and shows openly licensed models such as Llama 3 70B competing near the top.
Willison says the GPT-4 barrier has been broken. Several organizations now compete in the same broad quality range, and he expects these systems to become faster and cheaper as competition continues. He points out that Llama 3 70B can fit on a hard drive and run on his Mac. GPT-4o and Claude 3.5 Sonnet are effectively free for consumers at the time of the talk, so more people can experience both the impressive and unreliable sides of advanced models. Access is becoming less of a constraint, while knowing how to use the systems remains difficult.
AI applications require hidden operational knowledge
Willison uses PDF uploads to show why ChatGPT is harder to use than it appears. A PDF needs searchable text to work reliably, and long files may be handled through an internal search process rather than being placed directly into the prompt. Tables and diagrams may be processed incorrectly, while screenshots of those elements work better through GPT vision. ChatGPT may also use Code Interpreter and packages such as fpdf, pdf2image, or pypdf. Willison learned this by scraping the package list with GitHub Actions because OpenAI does not publish enough detail about the feature. He compares this experience to mastering Excel, where basic use is easy but expert use takes years.
Willison describes public reactions to AI features added by Dropbox and Slack. Users feared that their private data was being used to train models because of confusing terms and default settings. He says neither company was training models on customer data, and both had agreements that prevented OpenAI from training on the data sent to it. That explanation is hard to communicate after users have already lost confidence. Anthropic made a direct statement that it had not used customer or user-submitted data to train its generative models. Willison also points out that model providers still face criticism for training on unlicensed web scrapes.
Rendering untrusted markdown can leak private data
Willison presents a prompt-injection example he calls the markdown image exfiltration bug. If a chatbot can access private information and render markdown images, an attacker can ask it to create an image URL whose query string contains a secret. Loading the image sends that secret to the attacker's server, and the image can be invisible. He says this pattern appeared in ChatGPT, Google Bard, Writer.com, Amazon Q, Google NotebookLM, and GitHub Copilot Chat. His direct engineering advice is not to render markdown images in this way. The repeated mistake shows why chatbot features need security review around untrusted instructions and output.
Language models are gullible because they follow information from any source
Prompt injection can produce security failures, but it can also create strange answers. Willison describes a retrieval application that answered a question about the meaning of life in the voice of a witty gerbil. The source documentation contained an example prompt telling a model to pretend to be a witty gerbil and discuss snacks. Semantic search found that passage as the closest match, and the model followed it. Willison says models are designed to believe information supplied by their users, but they can also accept instructions embedded in retrieved material. Systems that make decisions from unverified context can therefore be manipulated.
Slop is unrequested and unreviewed generated content
Willison defines slop as AI-generated content that is both unrequested and unreviewed. Asking Claude for information is not slop, and publishing AI-assisted writing is acceptable when a person checks it. Publishing whatever comes out of a prompt without review is different because it adds unreliable material to the internet. He compares the term to spam, which helped make unwanted marketing messages recognizable as bad behavior. His concern is accountability. A person can attach their reputation to published work and verify that it deserves to exist. A language model cannot take responsibility for the quality or truth of its output.
"We as humans can do that and so if you're using a language model to help you publish, great text, fantastic, provided you're reviewing that text."16:50
Who should watch
You are choosing models for an application and want quality, cost, and open licensing to be part of the decision.
You are building a chatbot that retrieves documents, renders markdown, or handles private data and need concrete prompt-injection failure modes to test.
You publish AI-assisted writing or product content and need a clear standard for when review and human responsibility are required.
