Skydio is moving drones from tools that people carry and pilot into infrastructure that can launch, fly, complete tasks, and return to docks with limited supervision.
2
The autonomy stack divides work between the drone and the cloud, using edge systems for immediate actions and cloud systems for heavier models, longer-term planning, and fleet-wide learning.
3
Bargoti argues that fully end-to-end learning is promising for physical systems, but its failure modes are still hard to observe and its reliability is difficult to guarantee.
Summary
Suchet Bargoti presents Skydio's approach to operating many drones through a single high-level interface. He begins with a live demonstration: launching a docked drone near San Mateo, starting another in Colorado, tracking a car, and sending the fleet back to dock. He then describes drones as infrastructure used by utilities, public safety agencies, and construction companies. The talk covers the engineering needed to make that practical, including edge and cloud autonomy, low-bandwidth video, maps that act as world models, fleet-based map updates, and tracking through occlusion. A visual language model can interpret a request such as finding a white Jeep, call drone APIs, and follow the object without a hand-coded rule for that specific case. Bargoti is careful about the limits of fully end-to-end learning. Physical systems need very high reliability, while end-to-end systems make it difficult to see what went wrong or provide guarantees. Skydio therefore combines learned behavior with explicit world models and tools.
Drones are becoming infrastructure that can act without a pilot beside each vehicle
Bargoti describes Skydio's shift from drones as hobby devices, to tools carried by workers, to infrastructure that is available at fixed locations. The company has thousands of docked drones deployed around the country with power utilities, public safety organizations, and construction companies. In his example, a user launches a drone from a laptop, gives it instructions, and then starts another drone in Colorado while the first one is still flying. The intended interface is a high-level request, possibly a Slack bot saying that something happened and a drone should be sent, rather than a dedicated pilot controlling every flight.
The live demo shows why one operator can manage a fleet
The demonstration combines several operations that would normally require separate attention. Bargoti starts a dock opening in Colorado for a possible power-line fire, lets the dock perform its safety checks, returns to the first drone, and asks it to track a car. The tracking continues while he starts another drone at headquarters. He then tells all the drones to pause and return to dock. The point is that the operator can issue objectives while the autonomous systems handle flight, safety checks, tracking, and landing. Bargoti says the systems shown are already used in production rather than being only a concept.
Infrastructure use cases depend on reliability across difficult environments
Bargoti gives two examples of why docked autonomous drones are useful. A utility inspection found a pole burning from the inside before it could fall and create a fire risk. In San Francisco, Skydio works with SFPD to follow a stolen car from the air, allowing officers to position themselves for an intervention instead of starting a high-speed chase. The docks are intended to operate day and night, in rain and sunshine, including very cold conditions in Alaska and extreme heat in Texas. Bargoti says the systems need reliability down to 99.9999 percent and that about 16 million people live within two miles of this infrastructure.
A fleet needs an autonomy stack that spans hardware, software, cloud, and the user interface
Bargoti says Skydio controls the hardware, software, cloud, user interface, and autonomy, which lets the company work on the full system rather than only one model. The stack must handle perception in cloudy conditions, at high altitude and speed, and in rain. It must navigate cities, plan at large scale, and track objects through occlusion. He frames the problem as moving from first-person interaction with one vehicle to a multi-agent view in which a user commands a fleet with an objective. That shift is needed because rising numbers of emergency calls and alerts would otherwise require more skilled pilots.
Each flight can add information to Skydio's training process. Bargoti describes collecting data during operations, sanitizing it so private information is removed, checking what customers have agreed to share, and then comparing intended behavior with what actually happened. Those examples can return to learning agents and reinforcement-learning systems for retraining and evaluation before a new version is sent back into operation. The fleet therefore provides repeated observations of real environments and failures. Bargoti compares part of this process to Google Street View, while stressing that privacy and customer data handling are part of the workflow.
Immediate control stays on the drone while heavier reasoning can run in the cloud
The autonomy stack splits work between the edge device and cloud servers. Immediate autonomous actions happen on the drone, where low latency matters. The cloud can run GPUs and inference engines for heavier processing and longer-term planning. This division creates a communication problem because video and telemetry must reach the cloud over sometimes limited networks. Bargoti describes work on encoding information into smaller sizes and decoding it into clear, higher-quality video under the same network conditions. That network layer supports the cloud-based autonomy without making every decision depend on a perfect connection.
Maps give the fleet a shared world model that can be updated by its own flights
Bargoti uses maps as an example of a world model. A drone combines prior information such as buildings with vector data for power lines and roads, then uses the resulting map for planning and navigation. The map supports global planning rather than only reacting to the next obstacle. Maps can become outdated, though. He shows a construction site that was missing from the existing representation. As drones fly, they observe changes and send data back to the map-syncing process. The updated map can then be distributed so the rest of the fleet has the newer view of the world.
Cloud VLMs can make slower, broader tracking decisions after edge tracking starts
The drone can perform a minimal form of object tracking on the edge, while the cloud handles more demanding tracking and reasoning. A cloud model can consider the full map and use heavier models, including visual language models. Bargoti gives a response rate of roughly 1 to 2 seconds for this higher-level feedback, compared with 7 to 10 hertz for faster processing, and says that slower response can still be enough for broad movement decisions. The system also needs to keep tracking objects behind occlusions, such as a vehicle disappearing behind a building and reappearing elsewhere.
End-to-end learning remains limited by observability and reliability requirements
Bargoti describes the attraction of giving a system raw sensor data and receiving the desired action, such as where the drone should point or fly. Skydio is testing reinforcement learning and related end-to-end approaches, but physical systems need very high reliability. When an end-to-end system fails, it is difficult to observe what caused the failure and difficult to provide reliability guarantees. His approach is to decide which parts can use learned behavior and which need a world-model representation or explicit tools. Search and rescue is an example where hard-coded branching can become unwieldy, with instructions to search certain areas, look under objects, turn on thermal imaging, and try another location.
