# Building Context-Aware Reasoning Applications with LangChain and LangSmith

Harrison Chase, LangChain | AI Engineer Summit 2023 | 18:54

Source: https://www.youtube.com/watch?v=cwjs1WAG9CM
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/building-context-aware-reasoning-applications-with-langchain-and-langsmith
Published: 2023-11-01
Tags: agents, context-engineering, evals, observability

## TL;DR
- Language models need surrounding systems that provide current information, tools, memory, and other relevant context.
- Reasoning applications range from fixed chains to routers and looping agents, with different tradeoffs between control and flexibility.
- Teams need tooling for orchestration, data debugging, prompt iteration, evaluation, and collaboration as these applications become more complex.

## Summary
Harrison Chase argues that language models are only one part of a useful AI application. On their own, they lack current information, cannot run a user's code, and do not remember earlier interactions. Applications must provide context through instructions, examples, retrieval, or fine-tuning, then give the model a way to reason through a task. He describes a progression from single model calls to fixed chains, routers, looping agents, and more autonomous systems. Each architecture has different tradeoffs. Chains give developers control, while agents can react to unexpected inputs. Chase explains why LangChain helps teams prototype these systems and why LangSmith helps inspect tool calls, model calls, prompts, and retrieved data. He also discusses evaluation, where teams often lack both datasets and useful metrics, and collaboration between engineers, data specialists, product managers, and subject-matter experts.

## Key ideas
### A language model needs a larger system to become a useful assistant
[00:35](https://www.youtube.com/watch?v=cwjs1WAG9CM&t=35s)
Chase says models such as ChatGPT are only the starting point. By themselves, they do not know about current events, cannot run code written by the user, and do not remember previous interactions. A personalized assistant therefore needs a surrounding system that supplies relevant context and gives the model ways to act on it. The product may feel magical to the user, but Chase describes the underlying work as a substantial engineering task. LangChain is intended to help teams close the gap between the visible product experience and the system required to produce it.

### Instructions, examples, retrieval, and fine-tuning provide different kinds of context
[01:50](https://www.youtube.com/watch?v=cwjs1WAG9CM&t=110s)
Chase separates several ways to bring context to a model. Instruction prompting tells it how to behave, much like an employee handbook. Few-shot examples are useful when behavior is hard to describe, such as tone or structured output. Retrieval-augmented generation gives the model text to base its answer on, like an open-book test. Fine-tuning changes the model's weights and can help when a team needs far more examples than it can place in a prompt, especially for tone and structured data parsing.

### Reasoning systems become more flexible as the model chooses more of the process
[05:20](https://www.youtube.com/watch?v=cwjs1WAG9CM&t=320s)
The simplest application is a single model call that determines the output. A chain adds multiple known steps, such as a model call followed by an API call and another model call. A router lets the model choose which branch to use, prompt, model, or tool. A looping agent lets the model select actions repeatedly and decide when to stop. Chase contrasts these systems with more autonomous agents, where much of the planning, action selection, and validation is left implicit rather than written as distinct states in the application code.

### Chains and agents have different tradeoffs
[10:55](https://www.youtube.com/watch?v=cwjs1WAG9CM&t=655s)
Chase does not present one reasoning architecture as universally superior. Chains give developers more control over the sequence of steps. Agents can react more dynamically to unexpected inputs and edge cases. He describes plan, execute, and validate as a common structure for state-machine-style applications. An autonomous agent may be asked to handle those activities itself, deciding when to plan, act, validate, or finish. He also warns that the field is changing quickly, so today's categories may become less useful as systems develop.

### Debugging the full context is a central engineering problem
[10:35](https://www.youtube.com/watch?v=cwjs1WAG9CM&t=635s)
As applications grow, developers need to see what happens inside them. LangChain helps teams prototype different architectures, while LangSmith provides visibility into the exact sequences of tool calls and language model calls. The same applies to data. Teams must load, transform, and transport information, then inspect whether the right documents came from a vector store and whether they were formatted clearly for the model. Prompt debugging also requires seeing the fully assembled input, including system instructions, examples, retrieved context, chat history, and earlier agent steps.

### Evaluation starts with building data and accepting imperfect metrics
[14:18](https://www.youtube.com/watch?v=cwjs1WAG9CM&t=858s)
Chase says LLM applications are often easy to turn into a working MVP without first creating a dataset, because the models can learn zero-shot. That makes later evaluation harder. Teams can build evaluation data by labeling examples, collecting production traffic, or generating examples with language models. Traditional quantitative metrics often work poorly for long, unstructured outputs, so teams still use human 'vibe checks' to build intuition. Chase is positive about LLM-assisted evaluation and recommends tracking direct feedback such as thumbs up or down alongside indirect signals such as link clicks or confused replies.

### Building these applications will involve several kinds of specialists
[16:56](https://www.youtube.com/watch?v=cwjs1WAG9CM&t=1016s)
Chase expects larger systems to involve AI engineers, data engineers, data scientists, product managers, and subject-matter experts. Context awareness often needs data engineering work to bring the right information into the application. The reasoning behavior is often shaped through prompts, where product managers or domain experts may be well placed to specify how the application should behave. He says the best skill set for an AI engineer role is still unclear and that collaboration between these groups remains an unsolved design problem.

## Notable quotes
- "By themselves they're just the start." (00:35)
- "If you don't provide that context no matter how good the language model is it's not going to be able to figure out what to do." (01:50)
- "Chains are really good because you have more control over the sequence of steps that are taken." (11:15)
- "The second Big Challenge in evaluation is lack of metrics." (15:29)
- "I think like the main thing that I want to leave you all with is that the big thing that we believe is that it's still really really early on in this journey." (18:01)

## Tools & references mentioned
- ChatGPT
- LangChain
- LangSmith
- GPT-4
- GPT-4V
- Claude
- AutoGPT
- Voyager paper
- Robust Intelligence
- Kensho

## Who should watch
- You are building an application around a language model and need to decide between a chain, router, or agent.
- Your retrieval or prompting pipeline works in demos, but you cannot inspect the exact data, prompts, and model calls reaching the model.
- You are defining an evaluation process for an LLM product without much labeled data or a reliable automatic metric.

## Related talks

- [Build an AI Research Agent](https://aietalks.com/talks/build-an-ai-research-agent) (Apoorva Joshi, MongoDB, 27:33)
- [Creating Agents that Co-Create](https://aietalks.com/talks/creating-agents-that-co-create) (Karina Nguyen, OpenAI, 24:22)
- [From Systems of Record to Systems of Context](https://aietalks.com/talks/from-systems-of-record-to-systems-of-context) (Omri Bruchim & Tomer Ast, monday.com, 15:58)
- [Building Applications with AI Agents](https://aietalks.com/talks/building-applications-with-ai-agents) (Michael Albada, Microsoft, 15:50)
- [Connecting the Dots with Context Graphs](https://aietalks.com/talks/connecting-the-dots-with-context-graphs) (Stephen Chin, Neo4j, 17:39)