Human reviewers can rubber-stamp AI alerts even when they are skilled and the alerts are false.
2
A small change in decision framing increased accurate rejections of false cheating signals by 21%.
3
Interfaces should elicit independent judgment and capture nuanced feedback instead of collecting mostly binary approvals.
Summary
Angel Ortmann Lee argues that adding a human to an AI workflow does not guarantee careful judgment. People often accept AI output with little scrutiny, a behavior she calls cognitive surrender. At the Duolingo English Test, experienced proctors accepted fabricated copy-typing alerts half the time, despite scoring above 90% in accuracy calibration. The model and the reviewers were not the main problem. The review interface presented the AI signal in a way that encouraged confirmation. Telling proctors that the alert was preliminary and requiring independent evidence improved accurate decisions by 21%. Lee generalizes this lesson to AI products, writing tutors, and coding agents. Interfaces should expose assumptions, separate different decisions, add friction when stakes are high, and capture edits, overrides, questions, and trade-offs. These interactions create better labels and more useful evaluation data. The talk's practical message is that interaction design shapes both human behavior and the next version of the model.
Lee says people increasingly hand small cognitive tasks to technology. They store phone numbers in contacts, follow GPS routes without examining them, and may accept an AI search summary instead of checking the CDC or WHO. As AI becomes part of these routine decisions, trust increases while caution decreases. She connects this pattern to a Wharton study that calls the result "cognitive surrender": a person gives up deliberation and adopts AI output with minimal scrutiny. In that study, AI assistance improved performance by 25 percentage points when correct, but reduced it by 15 points when wrong. Eighty percent of participants accepted AI answers even when they were incorrect.
Skilled reviewers can still rubber-stamp false AI alerts
The Duolingo English Test is an online English proficiency exam used by 6,000 programs worldwide. Its security process includes identity checks, a locked testing environment, AI monitoring, and human review of video and AI results. Lee describes a copy-typing detector that looks for unusual keystroke patterns associated with transcribing rather than composing. In an experiment, proctors reviewed historical sessions with no cheating, but were shown fabricated alerts at specific moments. Although they normally scored above 90% on calibration metrics, they accepted half of the fake signals and falsely accused test takers at roughly a coin-flip rate. The sessions were historical, so the experiment did not affect customers.
Decision framing changed reviewer behavior without changing the model
Lee says the model had a 1% false-positive rate and the reviewers were experienced, so the team focused on the human-AI interaction. They changed the proctoring guidance in two ways. The AI signal was described as a preliminary alert, while the proctor remained the final decision-maker. The guidance also required independent evidence in the video before upholding a flag. This copy change moved the relevant decision result from 50% to 71%, a 21% increase in accurate rejections of the false signals. Lee's engineering lesson is that the team did not need to retrain the model or redesign the user interface. The wording changed how proctors treated the AI output.
The human review loop is cyclical and produces model data
Lee describes human-in-the-loop systems as cycles rather than straight lines. A model produces an output, the interface shapes human behavior, and that behavior generates data for evaluations and later model work. Engineers cannot directly change how another person thinks, but they can change the interaction to elicit different decisions. Deliberately structured interactions create labels that can become training data and evaluation data. This can form a useful flywheel: better interactions produce better data, which supports better models and later interactions. Poor design creates the opposite cycle. A confident model encourages rubber-stamping, accepted outputs get logged as truth, and the model becomes even more confident while the human is encouraged to defer further.
Lee uses headphone detection to show how one button can hide two separate judgments. The first question is whether the model correctly detected pixels that look like headphones or earbuds. The second is whether the person should receive a violation for that observation. A test taker wearing a hearing aid may produce a true headphone signal without having cheated. If a reviewer selects "no" to avoid penalizing that person, the system may incorrectly record that the model's detection was false. Splitting the interaction into two decisions preserves the detection label and the policy decision separately. It also produces more data and avoids teaching the model that every non-violation is a detection error.
Good AI feedback is tied to the user's actual work
Lee contrasts an overwhelming writing tutor with a Duolingo-style alternative. When she asked an LLM to act as a writing tutor for a short passage written by an English learner, it returned 400 lines containing praise, indirect feedback, and a complete rewrite she had not requested. The feedback was difficult to connect to specific text. The alternative marks the passage directly: green indicates something good, yellow indicates awkward wording, and red indicates a direct mistake. Hovering reveals concise, actionable feedback, and suggestions can be accepted in place. Lee says this resembles how a classmate might mark up an essay. It also lets the learner improve in small steps and gives the system feedback tied to exact parts of the input.
Coding agents should support review instead of demanding approval
Lee identifies two common coding-agent patterns. One produces a large change across many files, which encourages the developer to approve everything and inspect the result later. The other asks for permission after every small change, leading to repeated clicks on "yes." Both patterns turn the developer into a rubber stamp. Lee compares a better agent to a junior developer who can plan, ask useful questions, document design choices, and divide work into reviewable pull requests. The developer should see assumptions and decisions before problems spread. This interaction also creates richer data than accepted-or-rejected blocks. It can record bad assumptions, trade-offs, style preferences, and the developer's preferred approaches.
Lee recommends adding friction where deliberate thought matters. For the high-stakes English Test, review gates and structured pauses can make proctors slow down before deciding, which reduces the chance that they become rubber stamps. The interface should include checkpoints and speed bumps that support clarity. Low-oversight uses, such as casual conversation with an AI, need a lighter interaction with fewer stopping points. The right amount of friction depends on the consequences of the decision. This principle treats review speed as a design choice rather than an automatic virtue. A system should make careful decisions harder to rush and low-risk interactions easy to complete.
Treat edits, overrides, and questions as valuable labels
Lee says every interaction already contains a signal about whether the system matched the user's intent. An approved plan or accepted suggestion may indicate a good fit, while a modification or override may show a problem. A simple yes-or-no record misses the difference between accepting an output and accepting it before manually changing or deleting part of it. Capturing that diff can reveal where the AI fell short and prevent false positive training signals. User questions and requests for explanations also contain information. They may indicate low trust or an incorrect output. Lee recommends defining success metrics and required data before building the system, then using structured inputs, surfaced assumptions, review gates, and explicit feedback at the right points.
"Sometimes the fix is not a better model or more oversight, it's just engineering the interaction itself."25:45
Who should watch
You are building a human-review workflow and want to know whether reviewers are investigating AI outputs or simply approving them.
Your product collects thumbs-up or thumbs-down feedback, but you need richer signals for evaluation and model improvement.
You are designing a coding agent or AI tutor and need interaction patterns that make work easier to review.
