tldraw's canvas is built from ordinary web components, so it can embed tools such as YouTube, CodeSandbox, Figma, and Excalidraw while keeping its own drawing interactions.
2
Make Real turns a sketch into a working web prototype, then lets users annotate the result on the canvas and send those annotations as the next prompt.
3
tldraw computer uses language models as the execution layer for connected blocks, allowing multimodal inputs, loops, decisions, generated media, and long-running processes.
Summary
Steve Ruiz introduces tldraw as an infinite canvas SDK built with React and demonstrates how it can contain interactive web content, code editors, design tools, and even another tldraw canvas. He then walks through experiments that combine the canvas with AI. Make Real turns wireframes into working applications and lets users draw corrections over the result. Draw Fast generates images from sketches. The main example, tldraw computer, is a graph of blocks that accept inputs, use language-model instructions, and pass outputs to later blocks. Ruiz shows it generating a commercial from text and drawings, combining numbers with visual inputs, and running a loop that creates and classifies songs. He argues that this provides a more intuitive way to design asynchronous, multi-stage computer processes. A final demo has an AI manipulate structured canvas shapes rather than painting pixels. The talk is a practical tour of prototypes, with Ruiz open about their rough edges and their potential for creative, visual programming.
tldraw is an interactive web canvas and an SDK for building on it
Ruiz describes tldraw as both a free whiteboard at tldraw.com and an SDK documented at tldraw.dev. It is built with ordinary web technology, with React underneath. The canvas can contain a YouTube video that remains interactive while users draw over it, as well as an embedded CodeSandbox, Figma, or Excalidraw. Users can even paste tldraw inside itself. The product work includes small interaction details, such as arrows that follow shapes and box corners that stay correctly positioned. Those details are part of what tldraw provides to developers instead of requiring each application to implement them.
Make Real turns sketches into working prototypes and accepts drawn corrections
Make Real began with the observation that people used tldraw for wireframes. With vision models such as GPT-4 with Vision, a screenshot of a wireframe could be sent to a model with an instruction to build it. Ruiz demonstrates a stop-motion application generated from a description, including camera input, image capture, onion skinning, and GIF playback. Because the result remains on the canvas, a user can draw an annotation over the website and use it as the next prompt. When Ruiz asks for a button to become solid, the model changes it. He also sends a screenshot of a bug with the original source and gets a fix.
Draw Fast links a changing sketch to a changing generated image
Draw Fast uses latent consistency models to generate an image from a drawing. Ruiz shows the image changing as the underlying drawing changes, then flattens the result so it can be moved, rotated, and stretched like another canvas object. He says the interaction can work almost in real time in good circumstances, although the live demo is less reliable. The experiment treats the canvas as both the input surface for image generation and the place where the generated result can be edited.
tldraw computer treats a graph of model-driven blocks as a visual program
In tldraw computer, each block accepts inputs and produces outputs. Ruiz gives the graph an AI Engineer conference prompt and a drawing of a top hat with playing cards, then asks it to write a short commercial. One block creates the script, another generates speech, and another creates an image from the text. Each block has a small instruction describing how to inspect its inputs and what output to produce. The output format is passed to the next block, so the graph becomes a chain of model-guided transformations.
The graph can perform computation through language-model interpretation
Ruiz shows a block asked to add its inputs. With 2 and 11, it produces 13, but the calculation is done by a language model rather than code. When one input is an octopus, the prompt tells the model to infer a number, so it interprets the octopus as eight and produces 10. He also uses a camera input and gets an answer based on what the model sees. The point is that the same multimodal mechanism can accept text, images, and other inputs, then produce an output without every operation being predefined as conventional software.
Loops let a visual program run repeated decisions and transformations
Ruiz builds a process that creates a random pop song, adds it to a list, feeds the list back so the next song does not repeat, and asks whether each song is about love. The classification output can be yes, no, or maybe because the system is working with a language model. The loop can continue indefinitely. He also describes a possible process that reads a CSV of email addresses, replies to people who engaged with a product, performs sentiment analysis, and asks for approval before sending another email. He sees the canvas as a useful interface for designing long-lived asynchronous work that could run in parallel.
Structured canvas shapes let AI edit diagrams as objects
Ruiz demonstrates an AI tool that creates a flowchart beginning with AI and ending with engineer, using existing shapes. A request to draw a cat produces structured canvas objects rather than a painted image. That means the user can select and correct the individual shapes, and the model can inspect and modify them later. When Ruiz marks a cat as orange and asks it to blow out a candle, the tool adds the requested action and even produces smoke. He says an SDK with a runtime API makes it easier for AI systems to collaborate with a canvas.
The prototypes are rough, but Ruiz thinks the canvas has room for broader creative software
Ruiz is candid that some of the demonstrations work inconsistently. He describes tldraw computer as popular, though less popular than Make Real, and says it rewards creative experimentation. He points to uses such as turning a child's drawings into pictures and stories, and to external projects that use tldraw as a control or authoring layer for simulations. Observable is building with tldraw, and Ruiz closes by offering the canvas and its tools to people who want to build unusual applications.
"I want a computer that works the way that I thought a computer worked before I knew how a computer works."15:22
Who should watch
You are building a visual editor and want to see how an SDK can combine structured canvas objects with model-generated actions.
You want examples of using sketches and annotations as inputs for application generation, rather than treating an image as the final output.
You are exploring graph-based or asynchronous AI workflows and need a user interface for connecting model-driven steps.
