Paint.WTF turned CLIP into an AI Pictionary game where users drew text prompts and competed on image-text similarity.
2
The game attracted 120,000 players in its first week and reached seven submissions per second at peak.
3
Users quickly found ways to exploit CLIP, so the team used CLIP to penalize handwriting and block unsafe submissions.
Summary
Joseph Nelson explains how Roboflow built Paint.WTF, an AI Pictionary game powered by GPT-3, a browser canvas, CLIP, and Supabase. GPT-3 generated prompts such as "a giraffe in the Arctic" and "a bumblebee that loves capitalism." Players drew images, and CLIP ranked them by comparing the text embedding of the prompt with the image embedding of each drawing. The game reached 120,000 players in its first week and processed seven submissions per second at peak. Nelson then builds a small version live with Roboflow's inference server. The practical lessons come from letting strangers submit images: users wrote prompt text into their drawings to improve rankings, submitted unsafe content, and found ways to bypass simple checks. The team used CLIP to detect handwriting and penalize it, and compared images against unsafe concepts to block submissions. Nelson is open about CLIP's conservative similarity scores and the need to rescale them for display.
Paint.WTF made CLIP into a competitive drawing game
Paint.WTF generated a prompt with GPT-3, gave the player a Microsoft Paint-style browser canvas, and used CLIP to judge the drawing. CLIP compared the text embedding of the prompt with the image embedding of each submission. The closest match ranked highest on the leaderboard. Prompts included "a giraffe in the Arctic," "an upside down dinosaur," and "a bumblebee that loves capitalism." The game spread through Reddit and Hacker News, reaching 120,000 players in its first week and seven requests per second at peak.
Open-set prompts let users draw ideas outside fixed class lists
Nelson uses the prompt "a bumblebee that loves capitalism" to explain why CLIP enabled a different kind of application. The model could compare a drawing with an abstract text description, even though the team was not selecting from a fixed list of object classes. He connects this to open-form, open-set understanding, where users can provide concepts that were not predefined by the application developer. The game made this useful through simple drawings and playful prompts rather than through a conventional classification interface.
A small CLIP application can be built with a short inference loop
Nelson builds a simplified version of the app using Python, OpenCV, Roboflow's open-source inference server, and the CLIP model. The basic flow embeds the prompt as text, reads an image from a webcam, embeds the image, and calculates similarity. He first demonstrates the inference server with a Rock Paper Scissors model, then swaps in CLIP and displays the similarity score over the image. He says the original application was built in 48 hours and that the live version can be reduced to less than 50 lines of Python.
Users learned that writing the prompt into a drawing improved its score
One raccoon-driving-a-tractor submission ranked 586 out of 10,187, while another user simply wrote "a raccoon driving a tractor" and ranked 81. Nelson's first lesson is that CLIP can read text inside an image. The team addressed this by using CLIP to moderate the drawings. If CLIP judged an image more similar to handwriting than to the requested prompt, the submission was penalized. The game therefore exposed an interaction between the model's image understanding and the rules of the competition.
CLIP scores were conservative and needed rescaling for the interface
Across more than 20,000 submissions, the lowest similarity was about 8 percent and the highest was 48 percent. Nelson says this range made the raw values poor for a public-facing score and for the live demonstration. The rendering code mapped the observed range onto zero to 100, with the lowest value treated as zero and the highest as 100. This made differences easier for players to understand, although the displayed number was a rescaled comparison rather than a direct probability or quality measure.
User-submitted images required a separate safety check
The team asked CLIP to identify unsafe images because some players ignored the prompt and submitted unrelated content. Their rule compared the image's similarity to unsafe concepts with its similarity to the requested prompt. If the unsafe comparison was stronger, the submission was blocked. Nelson says this worked fairly well, while also describing the process as a cat-and-mouse game because users would draw the prompt and hide other content inside the image. The same zero-shot approach could support simple applications such as a CLIP-based "not hot dog" classifier.
Inference infrastructure kept the viral workload running
Nelson credits Roboflow's inference tooling with making deployment easier. The inference stream function handled model serving and was designed around the team's experience with hundreds of millions of API calls and video workloads. He says the software can maximize throughput on target hardware, and his live example ran locally on an M1 at about 15 frames per second. He also points to Roboflow Universe as a source of more than 50,000 pre-trained and fine-tuned models, including the Rock Paper Scissors model used in the demo.
"Strangers on the internet were smart so they'd like draw The Prompt and like sneak some other stuff in and it's this cat and mouse game with folks online."15:10
Who should watch
You are building an app where users upload images and need to understand how a foundation model may be gamed.
You want a compact example of turning image-text embeddings into a product with prompts, rankings, and a leaderboard.
Your prototype depends on serving inference at unpredictable traffic levels and needs practical lessons from a live public launch.
