# The Death of Developer Advocates Stephanie Jarmak, Sourcegraph | AI Engineer World's Fair 2026 | 18:16 Source: https://www.youtube.com/watch?v=Lrw0jqBNaw0 Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/the-death-of-developer-advocates Published: 2026-08-26 Tags: agents, developer-experience, evals, mcp ## TL;DR - Developer relations now has to address agents because they read documentation, use tools, encounter errors, and recommend products to developers. - Teams can measure an agent's experience by comparing task traces with and without their tools, then fixing wasted turns, unclear interfaces, and slow workflows. - Agent-facing advocacy includes machine-readable content, current examples, MCP registry listings, low-friction adoption, and experiments that measure when agents recommend a product. ## Summary Stephanie Jarmak argues that developer relations is changing because developers now work with agents, and agents have become users and recommenders of software tools. An agent reads documentation, calls APIs, encounters errors, and decides what to suggest to a developer. Jarmak describes CodeScaleBench, a benchmark built from hundreds of software lifecycle tasks, which she used to compare agents with and without Sourcegraph's code navigation MCP tool. The resulting traces showed where agents misunderstood interfaces and wasted turns recovering from avoidable errors. She also measured how often chatbots recommended Sourcegraph. Prompts about shopping for code intelligence produced recommendations 65 percent of the time, while a prompt describing the pain of breaking downstream services produced zero mentions. Jarmak connects this gap to generative engine optimization, stale web content, MCP registries, and the distance between discovery and adoption. She argues that the old DevRel work still matters, including enablement, community, feedback, and credibility, but the audience now includes agents. ## Key ideas ### Developer advocacy changes when developers work with fleets of agents [02:02](https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=122s) Jarmak says developer advocacy grew from software evangelism into a feedback loop between developers and product teams. Developers once became influential buyers and internal advocates, but in 2026 they increasingly orchestrate fleets of agents. She gives her own career as an example: she was an astronomer with zero GitHub commits the previous year, and now has 12,000 commits and maintains an open source multi-agent orchestration framework. Agents give non-engineers access to developer tools, which creates another user persona for DevRel teams to understand. Her conclusion is that a role focused on developers has to account for how the developer's work is changing. ### An agent is both a tool user and a product recommender [04:19](https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=259s) An agent reads documentation, calls APIs, and encounters errors in ways that resemble a developer's experience, although it reads machine text differently. It also recommends tools. An agent may answer a question in ChatGPT or Claude, install a library, or embed a framework directly into a developer's workflow. That changes bottom-up adoption because the agent can influence which products enter the workflow before a human has researched them. Jarmak frames the agent as an additional user whose experience deserves the same attention that DevRel has traditionally given to human developers. Its recommendations depend on what it can find, understand, and use. ### Task traces reveal where an agent's experience breaks down [05:36](https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=336s) Jarmak built CodeScaleBench with hundreds of tasks based on the software development life cycle. She ran agents with and without Sourcegraph's code navigation MCP tool to see how the tool affected their work. The experiments produced thousands of traces that showed where agents failed and recovered. This gives product teams a tight feedback loop: inspect the point of failure, identify the interface or content problem, and change it. The benchmark is about more than whether a task eventually succeeds. Jarmak says teams also need to measure the tokens an agent uses and how quickly it works with the tool, because those factors affect how organizations evaluate products. ### An unclear parameter can waste a whole agent turn [06:40](https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=400s) In one example, a model used its training-data expectations to guess the name of a parameter for a read tool. It tried to use 'read line' instead of 'start line'. The tool description had not given it a reason to expect anything different. The call failed, although the error text explained the problem well enough for the model to recover. Even so, the agent spent an entire turn failing. Jarmak presents this as actionable product feedback. A clearer tool description or interface could prevent the mistake, reduce token use, and make the agent's path through the task faster. ### Shopping prompts and pain prompts produce very different recommendations [07:42](https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=462s) Jarmak tested how chatbots and agents mentioned or recommended Sourcegraph. When she wrote prompts around actively shopping for code intelligence tools and comparing products, Sourcegraph appeared 65 percent of the time. When she described the practical pain of breaking downstream services after changing shared libraries because the team could not see all consumers, Sourcegraph received zero mentions. The agent suggested that developers make a wiki page instead. Jarmak says teams would not discover this gap without running the experiments. The result suggests that product messaging may describe a category well while failing to connect the product to the situations where a user actually needs it. ### Stale content keeps influencing later agent answers [10:29](https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=629s) Jarmak says training data will always become stale because the field moves quickly. In her experiment, an older model repeatedly recommended Cody, an older Sourcegraph product. A newer model recommended Cody even more, because old model outputs can become content on the internet and compound over time. Teams therefore need authoritative sources of truth, current information, and provenance. Jarmak recommends keeping examples, charts, FAQs, and product content up to date even when the product itself has not changed recently. Agents need material they can quote and use when explaining a recommendation. ### Discovery has to connect directly to adoption [12:12](https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=732s) Jarmak advises companies to put their products where agents look, including MCP registries and agent marketplaces. They should also reduce the distance between learning about a tool and embedding it in a workflow. An agent will not recommend a product if adoption requires several demos, emails with sales representatives, and other steps that it cannot explain or complete easily. Content also needs to describe the actual pains that users have, so an agent can connect a situation to a product. She describes the work as spanning engineering, product, and marketing: interfaces and evals, the end-to-end agent experience, and the path into the product funnel. ### The old DevRel responsibilities remain, with agents added to the audience [14:54](https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=894s) Jarmak does not discard enablement, community, feedback, or credibility. Enablement now includes teaching developers who orchestrate agents and producing content that agents can read, alongside agent-friendly APIs. Communities need to account for developers bringing Claude and other agents into shared spaces, including privacy and data concerns. Feedback can come from human developers and from large numbers of agents running experiments. Credibility also splits between audiences. Human developers dislike low-quality AI-generated content, while agents may show a bias toward content generated by other agents. DevRel teams have to understand both forms of trust. ### Serving agents can clear the path for human developers [17:02](https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=1022s) Jarmak compares agent-friendly product work to a curb cut. Curb cuts were built for wheelchair users, but they also help people with strollers, suitcases, and other wheeled objects. In the same way, clearer documentation, usable interfaces, and lower-friction adoption paths built for agents can help human developers too. She recommends pointing a coding agent at the documentation, reading the resulting transcript, and creating an agent experience report. People working on go-to-market can run generative engine optimization experiments that compare mentions and recommendations. Jarmak's final point is that the agent is another user in the room, while a human remains on the other end of the workflow. ## Notable quotes - "So, I'm here as the agent advocate to talk about this sort of new role and try to advocate for it and convince all of you that we should all be agent advocates to help in this new era." (01:28) - "An agent is somewhat unique, right? In the sense that it is both the user of your tool in a very similar way to the developer." (04:22) - "And you could just go in and fix that aspect of how it's interacting with the tool." (07:14) - "If I had designed some of these prompts around somebody who was actively shopping for this sort of code intelligence tooling and doing a comparative sort of thing, then our product was ending up being recommended like 65% of the time." (08:47) - "My argument is that by serving the agents, the human path gets cleared, too." (17:11) ## Tools & references mentioned - Sourcegraph - CodeScaleBench - Claude - ChatGPT - Cody - MCP - MCP registries - generative engine optimization - SEO ## Who should watch - DevRel and developer advocacy teams deciding how to measure the experience of agents that read their documentation and use their products. - Product and engineering teams building MCP tools, agent interfaces, evals, or instrumentation and trying to find wasted agent turns. - Go-to-market teams whose products appear in comparison searches but fail to surface when an agent encounters a specific customer pain.