The Eyes Are The (Context) Window to The Soul: How Windsurf Gets to Know You

Sam Fertig, Windsurf13:37 · Jun 2025 · 748 views
Thumbnail for The Eyes Are The (Context) Window to The Soul: How Windsurf Gets to Know You Watch on YouTube
TL;DR
  1. 1

    AI coding tools can generate code easily, but producing code that fits a person's codebase, standards, and preferences requires context.

  2. 2

    Windsurf divides context into user state, such as open files and terminal activity, and codebase state, such as code, documentation, rules, and agent memories.

  3. 3

    Selecting relevant context is more useful than continually enlarging the model's context window, and it also helps control latency.

Summary

Sam Fertig explains why generating code is easier than generating code that fits a particular developer and codebase. A useful suggestion depends on the prompt, the state of the repository, and the user's current behavior in the editor. Windsurf groups this information into heuristics, such as cursor position, open-file history, clipboard contents, and terminal activity, plus hard evidence, such as code, documentation, rules, and generated memories. Fertig argues that context selection matters more than simply making context windows larger. The system needs to find relevant information quickly as repositories grow, while keeping latency low. He names several Windsurf features and search approaches, including embeddings, memories, rules, plain-text search, remote indexing, reranking, and parsing. In the questions, he says editor information is sent to Windsurf's servers for processing, but the servers are stateless, and Windsurf does not store or train on users' data.

Key ideas
01:20

Surface facts need context before they say much about a person

Fertig opens by describing himself with facts such as working at Windsurf, liking Slack emojis, wearing turtlenecks, exaggerating his title, and speaking at the AI World's Fair. Those facts alone do not tell the audience much. With different framing, the same observations can suggest that he is power hungry, prioritizes Slack over work, or enjoys sharing with friends. He then adds the idea of hard evidence, such as trusted colleagues saying that he spends all day on Slack. The example establishes the talk's model of context: observations become more useful when combined with framing and evidence.

05:25

Coding agents must generate code that fits its surroundings

Fertig says generating code is no longer difficult. A developer can use tools such as Windsurf or Augment to create a simple interface around a model and return an answer from a prompt. The harder task is producing code for a particular person and an existing large codebase. The output needs to follow organizational policies and standards, match personal coding preferences, and fit the surrounding code. This is why a suggestion can feel as if Windsurf is inside the developer's head. When Cascade recommends an approach that matches what the developer was already thinking, Fertig attributes that feeling to context.

07:01

Windsurf separates context into user behavior and codebase evidence

Windsurf's context model has two buckets. The first contains heuristics about the user's behavior and current intent. Fertig gives examples including code above and below the cursor, open files, the order in which files were opened and closed, clipboard contents, terminal state, and recent activity in the IDE. The second contains hard evidence from the environment, including code, documentation, user-provided rules, agent-generated memories, and other repository contents. He summarizes the relevant input as the prompt plus codebase state plus user state.

08:56

Choosing relevant context matters more than making the window larger

Fertig says teams spent a period trying to make model context windows larger so they could put more information into each call. That helped to a point, but Windsurf found better results by optimizing which context entered the window. This approach also addresses latency. For an AI coding agent, response time is part of the product experience rather than a minor preference. The system therefore has to select useful information instead of sending increasingly large amounts of repository content to the model.

09:36

Repository growth makes fast context retrieval a systems problem

Finding relevant context remains difficult as a codebase grows, even though the general problem has become easier across AI products. Fertig connects Windsurf's approach to the company's background in GPU optimization and its earlier identity as Exafunction. He says Windsurf also gives users tools to find and supply context themselves. The named capabilities include embedding search, memories, rules, custom workspaces, app mentions, plain-text search, a knowledge base, multimodal input, Riptide, MCP, remote indexing, reranking, and parsing.

12:38

Windsurf's context claims come with a stateless-server privacy explanation

During questions, Fertig says Windsurf processes information about the user in the editor, rather than having access to the user's whole operating machine. The information is sent to Windsurf's servers, which he describes as stateless and used for pass-through transactions. He says Windsurf does not store users' data and does not train on it. He directs people seeking security and legal guarantees to the company's website or the Windsurf booth.

"Our magic formula, right? What is relevant output when it comes to coding agents? It's your prompt plus the state of your codebase plus the user state."08:31
Who should watch
  • You are building an AI coding tool and need to decide what repository and editor information should enter a model call.
  • Your coding assistant produces plausible code but misses local conventions, project rules, or the user's current intent.
  • You are evaluating context retrieval, indexing, and privacy behavior for an AI coding product or an enterprise deployment.