# Designing AI To Scale Human Thought Jun Yu Tan, Tusk | AI Engineer World's Fair 2025 | 12:24 Source: https://www.youtube.com/watch?v=0tVu-V3_fFw Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/designing-ai-to-scale-human-thought Published: 2025-06-03 Tags: copilots, design, human-in-the-loop, product-strategy ## TL;DR - AI interfaces should keep people responsible for high-judgment work while helping them spot errors and consider alternatives. - Useful augmentation depends on blind spot detection, cognitive partnership, and proactive guidance that arrives at the right time. - Trust should grow through small, contextual interactions, while the product measures whether users become more capable rather than merely more engaged. ## Summary Jun Yu Tan argues that AI products should help people think better instead of taking complex work out of their hands. Automation can handle discrete tasks, but coding, design, and other high-context work still depend on human judgment, experience, and taste. He proposes three interaction patterns: blind spot detection, cognitive partnership, and proactive guidance. Tusk's testing platform illustrates the first pattern by creating and executing tests against code changes, then showing verified problems, assumptions, and possible fixes. Tan also describes a novelty-criticality framework for deciding which suggestions should interrupt a user and which should be batched. Trust should develop progressively and vary by domain. Augmentative products should adapt to users while helping them build skills, with product metrics that track capability growth alongside engagement. The talk is an argument for interfaces that keep users in control and make their reasoning more aware. ## Key ideas ### Augmentation keeps humans responsible for high-judgment work [01:44](https://www.youtube.com/watch?v=0tVu-V3_fFw&t=104s) Tan says many coding and design tasks depend on human experience and taste, and their context cannot always be packed into a prompt. He contrasts an email agent that writes and sends a message with an assistant that brainstorms points, suggests phrasing, checks tone, and leaves the user to review and send. In coding, automation might create a pull request from a Slack message, while augmentation reviews the pull request and points out an edge case or a high-load problem. The user remains in the driver's seat, with AI acting as a thinking partner. Tan compares this relationship to working with a new team member who grows with you rather than an offshore contractor taking over a task. ### Blind spot detection should reveal verified problems at the right moment [03:51](https://www.youtube.com/watch?v=0tVu-V3_fFw&t=231s) Tan defines blind spots as patterns people cannot see in their own thinking. They can be temporal, such as making worse decisions when tired on Friday afternoons, or social, such as giving technical feedback that sounds like personal criticism. The design problem is to challenge someone's thinking without making them defensive. At Tusk, code changes trigger generated unit tests that run against the codebase's test environment. The system surfaces issues it can verify, along with assumptions and possible fixes. Users can review the results with thumbs up or thumbs down and explain their reasoning, so the system learns from those judgments. Tan calls this approach "systematic pessimism," because it searches changed symbols and second-order effects for what could go wrong. ### Suggestion priority should combine novelty with criticality [06:11](https://www.youtube.com/watch?v=0tVu-V3_fFw&t=371s) Tan proposes a novelty-criticality framework for managing noise in code review. High-novelty, high-criticality findings, such as an unfamiliar race condition or accidental customer-data exposure, deserve interruption. High-novelty, low-criticality suggestions can appear gently as explanations or learning moments. Low-novelty, high-criticality items, such as standard security or performance checks, should receive clear warnings without lengthy explanation. Low-novelty, low-criticality items, such as formatting and minor optimizations, should be optional and batched. Tan says people tend to process only three or more meaningful suggestions per review session, so the interface must spend the user's attention carefully. ### Cognitive partnership requires a model of how the user thinks [07:16](https://www.youtube.com/watch?v=0tVu-V3_fFw&t=436s) The second pattern moves beyond a stateless question-answering system. Tan says an augmentative product should build a theory of mind about the user, including whether they learn through analogies, prefer visual explanations, sleep on decisions, or iterate quickly. A code editor could learn a user's refactoring patterns and suggest similar improvements in new contexts. A research assistant could adapt to whether the user synthesizes information chronologically or automatically. The difficulty is personalization without making people feel watched or surveyed. Users should feel understood, while retaining a clear sense of how the system reached its suggestions. ### Proactive guidance depends on timing rather than just accuracy [07:54](https://www.youtube.com/watch?v=0tVu-V3_fFw&t=474s) Tan calls proactive guidance the hardest pattern to get right because the system must decide when to offer help. A good suggestion feels like serendipity instead of an interruption. His examples include a calendar that considers a person's energy patterns when suggesting meeting times and a writing tool that notices when someone is stuck and recommends a break. The product must find a middle ground between waiting for an explicit request and overwhelming the user with unsolicited advice. Timing is part of the interaction itself, so a technically correct suggestion can still be unhelpful if it arrives at the wrong point in the user's work. ### Trust should grow gradually and vary by context [08:29](https://www.youtube.com/watch?v=0tVu-V3_fFw&t=509s) Tan says trust in augmentation systems should be progressive, contextual, and bidirectional. A system should prove itself with low-stakes suggestions before a user relies on it for high-impact decisions. Trust should also depend on the domain. An AI might be strong at code review but poor at design feedback, or it might lack the context needed for a particular decision. Bidirectional trust means the AI learns the user's preferences while the user learns the AI's capabilities. The system should adjust its behavior according to how much the user trusts it in different situations, rather than treating trust as a single permanent setting. ### An augmentative product should help users build skills over time [09:35](https://www.youtube.com/watch?v=0tVu-V3_fFw&t=575s) Tan says augmentation implies skill growth. He suggests visualizing expertise, increasing complexity as competence grows, and changing explanations as the user's understanding develops. The goal is for people to become better at their core work, rather than more dependent on the product. This also changes what teams should measure. Alongside engagement and retention, they should ask whether users are becoming more capable in real situations. Tan applies the same standard to learning apps and other AI tools: time spent with the product should lead to improvement, not only continued use. ### AI's value includes making people more aware of their own thinking [11:17](https://www.youtube.com/watch?v=0tVu-V3_fFw&t=677s) Tan closes by saying AI can help amplify intuition, taste, and creativity in ways earlier technologies did not. Blind spot detection, cognitive partnerships, and proactive guidance all depend on UX that supports trust and thoughtful agency. He describes these systems as mirrors for the mind, showing users patterns and perspectives they habitually miss. The measure of success is therefore larger than productivity. An interface can help someone become more thoughtful and aware of their cognitive patterns while keeping them involved in the work. Tan is also direct that designers are still at the beginning of understanding how to build such systems. ## Notable quotes - "Instead of getting humans to accommodate to AI systems attempting to automate complex tasks and doing so suboptimally, why not spend that compute just helping humans produce high quality work in the first place?" (01:44) - "In a nutshell, many differences arise based on a shift of responsibility of the task itself and our relationship with the AI system that helps us with these tasks." (03:17) - "The magic is in the prioritization because people tend to only process three or more meaningful suggestions per review session." (07:00) - "Trust needs to be progressive, contextual and bidirectional." (08:44) - "The goal is genuine skill enhancement, not just the illusion of it." (09:51) ## Tools & references mentioned - Tusk - ChatGPT - DeepLearning.AI - TeamFeePay - Slack ## Who should watch - You are building an AI product for coding, writing, research, or another high-context task where users still need to exercise judgment. - Your assistant produces too many suggestions and you need a way to decide what should interrupt users, what should be explained, and what should wait. - You want product metrics that show whether people are becoming more capable instead of only measuring usage or retention. ## Related talks - [Build AI Systems for Discernment, Not Approval](https://aietalks.com/talks/build-ai-systems-for-discernment-not-approval) (Angel Ortmann Lee, Duolingo, 25:53) - [The Intelligent Interface](https://aietalks.com/talks/the-intelligent-interface) (Samantha Whitmore & Jason Yuan, New Computer, 18:56) - [The UX of AI: Making AI-Powered Apps Your Users Don't Hate](https://aietalks.com/talks/the-ux-of-ai-making-ai-powered-apps-your-users-dont-hate) (Kathryn Grayson Nanz, Progress Software, 35:59) - [Climbing the Ladder of Abstraction](https://aietalks.com/talks/climbing-the-ladder-of-abstraction) (Amelia Wattenberger, Adept.ai, 16:47) - [Second Order Effects of AI](https://aietalks.com/talks/second-order-effects-of-ai) (Cheng Lou, 21:46)