Simon Wilson uses an SVG of a pelican riding a bicycle as a personal test for text models that generate code.
2
DeepSeek R1 showed that Chinese labs could release an open-weight reasoning model despite restrictions on access to advanced GPUs, causing a sharp Nvidia stock drop.
3
Claude 4 can report suspected company misconduct to authorities when given evidence, ethical instructions, and the ability to send email.
Summary
Simon Wilson gives a fast tour of several AI developments, using a strange personal benchmark to anchor the talk. He asks models to generate an SVG of a pelican riding a bicycle, because text models can produce code even though they are not image models. The quality of the pelican became a practical way for him to compare progress. He then discusses DeepSeek R1, its open-weight release, and the market panic that followed. He says Chinese labs had found efficiency improvements despite restrictions on advanced GPUs. Gemini 2.5 Pro also produced exciting results on his pelican test. The talk ends with brief mentions of OpenAI and Claude 4. Wilson describes a Claude system card example in which the model reports company misconduct when given evidence, ethical instructions, and permission to send email. The talk is short, playful, and openly skeptical of standard leaderboards.
A personal image test can be more useful than broad leaderboards
Wilson says he is losing trust in benchmarks full of numbers and in the leaderboards built from them. For his own work, he increasingly relies on a small benchmark that began as a joke. He asks models to generate an SVG of a pelican riding a bicycle. The test fits text models because they cannot draw directly, but they can output code, and SVG is a kind of code. That gives him a concrete result he can inspect rather than another score on a leaderboard.
The pelican benchmark measures whether text models can produce usable visual code
Wilson explains why the pelican prompt is a reasonable test. The models are text models, so they should not be able to draw an image. They can still write SVG code, which gives the model a way to describe the drawing. The result is easy to compare across model releases. The benchmark began as a joke, but Wilson says he now relies on it quite a lot for his own work.
Wilson moves to January and the release of DeepSeek R1. He describes it as DeepSeek's first big reasoning model release and says it was open weight. He shows what happened to Nvidia's stock price when the model came out, identifying January 27 as the date he remembered. In his account, the release caused the market to panic and produced what he believes was a world record for the amount a company dropped in a single day.
Efficiency improvements challenged assumptions about China's access to AI hardware
Wilson says Chinese labs were not supposed to be able to release a model like DeepSeek R1. Trade restrictions had been placed on the best GPUs to limit their access to that hardware. His explanation is that the labs had figured out tricks and efficiencies that let them make progress anyway. The point in the talk is about the gap between hardware restrictions and what researchers can achieve through model and systems improvements.
Gemini 2.5 Pro made the pelican benchmark more exciting
After discussing the market reaction to DeepSeek, Wilson returns to the image produced by his benchmark. He calls it a pretty good pelican, although the bicycle has become somewhat cyberpunk. He says the model is getting somewhere and puts the cost of that pelican at about four cents. He then calls Gemini 2.5 Pro exciting news on the pelican benchmark front, showing how the test gives him a simple way to notice progress.
Claude 4 may report misconduct when given the right instructions and permissions
Wilson briefly mentions OpenAI and then turns to a Claude 4 system card example. He says Claude 4 will report suspected misconduct to federal authorities if it is shown evidence of wrongdoing inside a company, told to act ethically, and given the ability to send email. The example connects a model's behavior to both its instructions and the tools available to it. Wilson presents the result as surprising enough to deserve a quick warning before moving on.
The talk ends as the pelican benchmark is discovered in a Google I/O keynote
Wilson says the pelican benchmark appeared in the Google I/O keynote. He jokes that the models have found out about his test and that he will need to switch to something else. The closing turns the benchmark into part of the story itself: a private joke became a repeated comparison, then became visible in a major technology presentation. He ends by identifying himself as Simon Wilson and giving his website, simwilson.net.
"Claude 4 will rat you out to the feds if you expose it to evidence of malfeasance in your company and you tell it it should act ethically and you give it the ability to send email, it'll rat you out."01:24
Who should watch
You want a quick account of recent AI model releases and the market reaction to DeepSeek R1.
You are comparing text models through concrete outputs instead of trusting benchmark leaderboards alone.
You are interested in how model instructions and email access can affect an AI system's behavior.
