Humanoid robots need physical AI because tasks in fields such as healthcare, construction, and manufacturing require operating devices in the real world.
2
GR00T N1 combines a slower System 2 planner with a fast System 1 controller that operates at almost 120 hertz.
3
GR00T N1 uses foundation knowledge across embodiments, then applies an embodiment-specific action decoder to produce robot motion.
Summary
Annika Brundyn and Aastha Jhunjhunwala introduce GR00T N1, NVIDIA's open Vision-Language-Action foundation model for humanoid robots. They explain why physical AI needs different data from language models. Robot data is scarce and expensive, so the training mix combines teleoperated robot trajectories, human videos, synthetic simulation, and generated variations of collected data. The model uses two connected systems. System 2 breaks down complex tasks and plans at a slower pace, while System 1 turns that plan into fast motor actions. Images, language, robot state, and noisy action data pass through encoders and a diffusion transformer before an embodiment-specific action decoder produces motion. The speakers also compare imitation learning with reinforcement learning and explain why GR00T N1 uses both. Its foundation-model design allows the same learned knowledge to be adapted to different robot bodies and downstream tasks.
Physical AI is needed for work that chatbots cannot perform
Annika Brundyn argues that industries such as leisure, hospitality, healthcare, construction, transportation, and manufacturing cannot be handled by ChatGPT alone. Their tasks involve instruments and devices in the physical world. She describes the challenge as making the intelligence found in language models operable outside software. Humanoid form helps because the world is built for people. A specialist barista robot may make espresso well, but that does not mean it can cook rice. A robot that needs to handle many tasks can work more easily in human environments if its body resembles ours.
Physical AI follows a data, training, and deployment lifecycle
The speakers describe three stages for building physical AI: collect or generate data, train a model, and deploy it on the robot or at the edge. NVIDIA calls this the three-computer problem because each stage has different computing needs. Simulation needs a system such as an OVX Omniverse machine. Training uses systems such as DGX to process large amounts of data. Deployment needs a smaller, efficient model that can run on an edge device such as AGX. Project GR00T includes the infrastructure, software, and research around robotics, while this talk focuses on its foundation model.
Robot training data has to be assembled from several imperfect sources
The GR00T team created a data pyramid because there is no internet-scale dataset of robots performing tasks. At the top is real robot data, often collected through human teleoperation with devices such as an Apple Vision Pro and gloves. This data provides successful task trajectories, but it is expensive and limited by human and robot fatigue. At the bottom are large collections of human videos, such as cooking tutorials. These videos are not structured for robots, but still contain useful information. Synthetic simulation can produce much more data, although high-quality environments require substantial labor and skill. The team also studies how video generation models can multiply scarce human teleoperation trajectories.
GR00T N1 maps perception and language to embodiment-specific actions
A model input includes an image observation, robot state, and language prompt. For example, the instruction can be to pick up an industrial object and place it in a yellow bin. The output is an action trajectory, which the robot represents as floating-point vectors controlling its joints. The architecture encodes robot state and noisy action data, then combines those tokens with vision and text tokens in a diffusion transformer. An action decoder converts the resulting tokens into vectors that a particular robot body can execute. The decoder can target a humanoid hand, a humanoid body, or an industrial robot arm.
The dual-system design separates planning from fast control
GR00T N1 applies the System 1 and System 2 idea from Daniel Kahneman's Thinking, Fast and Slow to robotics. System 2 acts as the planner. It breaks a complex task into simpler steps and works slowly. System 1 executes those steps quickly, operating at almost 120 hertz. The two systems are trained together in a coherent architecture. Aastha Jhunjhunwala says earlier approaches trained components independently, which could create disagreement between them. Co-training lets the full stack be optimized together.
Imitation learning and reinforcement learning solve different training problems
Imitation learning teaches a robot by having it copy a human expert and minimize the difference between its behavior and the expert trajectory. It has a clear target, but depends on expensive expert data. Reinforcement learning instead uses trial and error and maximizes a reward, without requiring a gold-standard demonstration. Its main difficulty is the gap between simulation and the real world. GR00T N1 uses both approaches. Imitation learning supplies demonstrations, while reinforcement learning can explore behaviors without the same expert-data bottleneck.
A foundation model can transfer knowledge across robot bodies and tasks
The speakers describe GR00T N1 as a generalist model that can be adapted to different embodiments and downstream tasks. Its base model contains knowledge learned from different robot bodies and data sources. The embodiment-specific action decoder translates that knowledge into motion for a chosen robot. Demonstrations show pick-and-place tasks in a kitchen, an intentionally romantic scene involving champagne and flowers, and an industrial task involving two robots. These examples are presented as extensions of a model that can be trained for other tasks and environments.
"None of them can be solved by ChatGPT alone. They require operating instruments and devices in the physical world."01:34
Who should watch
Engineers who understand language or vision models and want a beginner-friendly explanation of how those ideas reach robot control.
Teams deciding whether to use teleoperation, simulation, generated video, imitation learning, or reinforcement learning for robot training data.
Anyone evaluating whether a general robot model can transfer behavior across different robot bodies and tasks.
