AI-generated unit tests can pass while failing to check whether the application behaves correctly for users.
2
Playwright lets an agent write failing browser tests from feature requirements, then generate code until those tests pass.
3
The developer should spend the most time reviewing and refactoring the code after the agent has made the behavioral tests pass.
Summary
Marlene Mhangami argues that AI-assisted development needs tests based on user behavior, not only tests that cover implementation details. She describes red-green-refactor TDD, where a feature request first produces a failing test, code is then written to pass it, and the result is refactored. Her concern is that an agent can generate self-affirming unit tests that confirm the code it wrote without checking whether the application works. Playwright addresses this by automating browser interactions such as entering search terms, selecting categories, and checking price filters. In the demo, GitHub Copilot CLI uses a feature request for a toy store, the Playwright MCP server, and Work IQ to create and run tests before and after implementation. Mhangami recommends one test per feature, saving screenshots in pull requests, using headless mode when appropriate, and committing before asking an agent to change code. She also discusses Playwright agents for state management and browser checks across mobile and desktop views.
Clean codebases determine whether AI produces useful gains
Mhangami presents a Stanford study of 120,000 developers showing that AI productivity depends on how developers use it. Clean codebases amplify the gains from AI, while unchecked AI-generated code amplifies entropy. In the case study she cites, a team increased the number of pull requests but saw lower code quality and spent more time reworking and refactoring code. Its effective output increased by about 1%, so AI did not improve overall productivity. She recommends practices such as good test coverage, type coverage, documentation, and modularity before relying heavily on AI coding tools.
Red-green-refactor starts with a failing test for the feature
Mhangami describes red-green TDD as a three-stage process. When a feature request arrives, the developer first writes a failing test because the feature does not exist yet. During the green phase, the goal is to get the test passing quickly rather than polish the implementation. The final refactor phase improves the code and applies the team's best practices. She connects this workflow to agentic coding, where an agent can speed up the red and green phases while the developer spends more time reviewing and refactoring what the agent produced.
Unit tests can break when implementation details change
One criticism of TDD is that it can overemphasize unit-test coverage instead of testing the system's behavior. Mhangami gives an order calculation with a discount as an example. If a test is tied directly to a method named calculate, simply renaming that method can break the test even when the feature still works. Tests based on the final price or on a stable contract, such as an API or exported module, are more likely to survive internal refactoring. She recommends Ian Cooper's talk, TDD Where It All Went Wrong, for more on behavior-driven TDD.
AI can write self-affirming tests that miss broken behavior
Mhangami says AI-generated tests can create a false sense of safety. The code coverage tests may pass and the unit-test suite may be green, while the system's actual behavior has not been validated. The agent may write tests that confirm what its own code does instead of tests based on what users are supposed to experience. Her proposed response is to begin with behavioral tests for the requested feature, using Playwright to exercise the application through the browser.
Playwright tests the application through browser interactions
Playwright is an open-source Microsoft testing framework for end-to-end browser testing. Mhangami shows a typical test starting at a toy-store page, locating a search field by its placeholder, and filling it with the word Furby. Playwright supports Python, TypeScript, and C#, and it can run with the browser visible or in headless mode. Because the test follows user actions, it checks the behavior of the application rather than depending on the names or structure of internal methods.
AI makes the red and green phases faster while refactoring stays with the developer
When Playwright is connected to a coding agent, Mhangami expects the agent to generate failing behavioral tests and then produce code that makes them pass. This speeds up the first two phases of red-green-refactor. She recommends that developers spend more time on the refactoring phase, examining the generated code and improving its quality. Agents can connect to Playwright through the Playwright MCP server, the CLI, or Playwright agents. Playwright agents install planner, generator, and healer instruction files.
The toy-store demo turns a feature request into browser tests
In the demo, Mhangami works on a fictional Tailspin Toys application. A product team requests a search bar with simple text search and Azure AI Search for more complex searches, plus sidebar filters for category and price. GitHub Copilot CLI uses Work IQ to bring the request from Microsoft 365 into the terminal. Mhangami then asks Copilot to use red-green TDD and first write failing Playwright tests for each feature. The agent examines the codebase before generating the tests, and she asks it not to commit changes for the example.
The live test checks search, category filters, and price ranges
After the tests and implementation have been generated, Mhangami runs the Playwright tests through the MCP server. The browser opens the correct page, enters search inputs, finds Furby and Simon, clicks category buttons, and checks that the toys in a selected price range are correct. She keeps her hands off the keyboard while Playwright and Copilot perform the interactions. The passing tests give her direct evidence that the app behaves as expected. She then moves on to refactoring the generated code.
Playwright agents and API tests help with state-heavy applications
In the questions, Mhangami recommends Playwright agents when an application has substantial state management. Their agent.md files contain specialized instructions that can help the agent handle state. If browser testing is not the right fit for every part of an application, she suggests testing available APIs directly. She says Playwright can check mobile and desktop views, although it is browser-based for the moment.
"When I run these functionality tests, all I can see actively that, okay, the agent has written this code. The code is working as I expected. The app is working as I expected."15:35
Who should watch
You are using coding agents to generate tests and want to know whether those tests check user-visible behavior.
Your unit-test suite is green, but you are concerned that implementation-focused tests will miss failures in search, filtering, or other browser workflows.
You want a practical TDD workflow that combines GitHub Copilot with Playwright and still leaves code review and refactoring with the developer.
