An agent can choose among a robot's existing movement policies from a natural-language request.
2
Scout uses separate agents for environmental reasoning, conversation, and voice interaction.
3
A robot connected to an agent can collect training episodes while people direct it manually and observe its behavior.
Summary
Sandhya Subramani demonstrates Scout, a four-legged robot with a Raspberry Pi, a SIM card, and a 4G connection. Scout already has preset movement policies, but an agent layer lets it interpret natural-language requests and choose which policy to call. Subramani uses AWS's open-source Strands Agents framework to connect the robot as a hardware tool in about five lines of code. Scout runs agents for environmental reasoning and communication through Telegram and a web app. A voice agent exists but is disabled during the talk. The architecture separates the agent's decision about what to do from the policy's decision about how to do it. Some processing and policy execution can happen in the cloud, while other calls run on the edge. Subramani also uses Scout as a way to gather training data by manually moving it, creating episodes, and studying how it responds to instructions. The live demo is unreliable, with Scout falling over and performing unexpected actions, which makes the limits of the setup clear.
Scout can answer questions that were not part of its original training
Scout runs on a Raspberry Pi located in New York City and communicates through a SIM card and a 4G connection. Subramani asks it how many people it can see, even though she says it was never trained to answer that question. Scout checks its front camera and reports two people, one near the speaker monitor and another further to the right. It can also respond to direct commands such as turning on its headlights, saying hello, or spinning around. The response takes a second or two because the robot is sending the request through its connected system.
An agent can select a robot policy from a natural-language instruction
Scout already has tools and preset functions for actions it knows how to perform. Subramani adds an agentic layer called Strands Agents, an open-source framework built by AWS. The agent receives the natural-language request, decides which tool to call, and selects an available programmable policy. This is similar to giving a software agent access to tools, except the tool is a robot with hardware actions. The agent does not invent the movement itself. It chooses among policies that the robot already has.
The robot can be connected to an agent with a small amount of code
Subramani shows a short Strands example that imports the agent, calls the robot tool, and assigns the robot to the available tools. The request can then be phrased as 'pick up the red cube', assuming Scout has that capability. Her point is that the integration does not require a large application around every new instruction. The agent can interpret the request and call one of the robot's existing functions. The available policies still determine what physical actions are possible.
Scout runs separate agents for reasoning and communication
Scout has three Strands agents working at the same time. The thinker agent continually assesses the environment and considers what to do next. A communication agent lets Subramani converse with Scout in natural language through Telegram and a web app, then ask it to take actions. A voice agent can also listen and respond, but she disables it because it would treat her stage remarks as commands and interrupt her. These agents give the robot different ways to perceive, decide, and communicate while sharing the same hardware.
The system separates the agent decision from the trained policy
Subramani describes four layers. The agent receives an instruction and decides which policy to call. A policy provider supplies the relevant robot policy, which can be trained through collected data and simulated data. That policy becomes a vision-language-action model and runs either in a simulation environment or on real hardware. The final output occurs in the physical robot. Her concise division is that the agent decides what to do, while the policy decides how to do it.
The architecture uses Strands Agents on the edge and in the cloud. Subramani says training the vision-language-action models and policies can happen in the cloud with AgentCore, while direct calls on the edge let the robot execute functions and policies faster. Strands can decide which side to call. This arrangement supports training from collected information while keeping some runtime actions close to the robot. The policy may therefore live in a simulation environment, on a hardware chip, or in another hardware environment connected through the system.
Larger robot policies could reduce the need for task-specific fine-tuning
Subramani imagines vision-language-action models growing as large as large language models and containing broad information about how to act. In that future, a robot might receive one general model and know what to do without the current process of fine-tuning existing policies. She describes the present agent-and-policy setup as a step toward that possibility. For now, the agent can extend a robot beyond its fixed task list only by choosing among the capabilities and policies that have already been provided.
Scout can provide data for improving its own behavior
Subramani can manually move Scout, direct it through a desired route, and create training episodes. She can record how it responds and reasons after receiving different questions. That information can help her improve its behavior. Her configuration uses Strands Agents with Anthropic Claude Opus 4.8 as the language-model brain, a system prompt describing the robot's rules and tools, and OpenAI Realtime for voice. She also adds safety information and guardrails. In the live demo, Scout attempts unexpected actions, falls over, and turns off its headlights, showing the gap between selecting a function and getting a reliable physical result.
"Every time I need a confidence boost, I can be the only person in the room and ask this guy who's the prettiest of them all, and he will always choose me as the answer."16:45
Who should watch
You are building language-model agents that need to control physical devices through existing APIs or policies.
Your robot can perform a fixed set of actions, but you want people to address it with natural-language instructions.
You need a way to collect robot episodes while testing how an agent chooses and combines available behaviors.
