Coding agents can write and benchmark CUDA kernels when they have hardware context, examples, and reusable project files.
2
Skills turn difficult zero-shot engineering tasks into few-shot workflows by giving agents versioned instructions, examples, and tests.
3
Open repositories, compute, storage, and tracking let multiple agents run machine learning experiments in parallel as an automated research lab.
Summary
Ben Burtenshaw argues that coding agents should move beyond application code into AI systems engineering. He presents three increasingly autonomous examples. First, an agent writes and benchmarks CUDA kernels, then distributes them through Hugging Face's kernels repositories. Second, an agent fine-tunes a language model end to end from a prompt using Hugging Face infrastructure. Third, a research lab uses separate researcher, planner, worker, reviewer, and reporter agents to propose, run, assess, and report training experiments. Burtenshaw's main technique is to give agents file-based skills containing instructions, examples, scripts, and references. This changes a zero-shot task into a few-shot workflow. He also argues that open primitives matter because agents need access to the underlying data and tools. Trackio stores experiment data in an open format, while Hugging Face provides storage, tracking, and compute for the jobs.
Coding agents can move into AI systems engineering
Burtenshaw says coding agents can now tackle difficult engineering work in systems engineering and machine learning engineering. He proposes moving closer to the silicon as agents become common in ordinary software development. The talk has three progressively more autonomous examples. The first is an interactive agent that writes a CUDA kernel. The second is a zero-shot prompt that asks an agent to train a language model on Hugging Face. The third is a multi-agent research setup that operates like an automated AI lab. The examples shift from a human working with one agent to several agents running experiments with less direct intervention.
Custom kernels improve model speed by using the GPU more efficiently
A model running on a GPU executes its work through kernels written for particular hardware. Custom kernels can target a specific math operation and use hardware features that are unavailable elsewhere, but writing and installing them requires knowledge of CUDA, hardware generations, software versions, and a large compatibility matrix. Burtenshaw divides deep learning efficiency into compute, memory, and overhead. He says memory is usually the bottleneck, because a modern GPU can perform enormous amounts of computation while data still has to move through memory. Kernels such as FlashAttention increase arithmetic intensity by doing more sums for each read and write, keeping more of the GPU's capacity in use.
The Hugging Face kernels library turns optimized kernels into shareable repositories
Hugging Face's kernels library provides a way to distribute custom kernels as repositories on the Hub. Each repository has a TOML file that records supported hardware, CUDA versions, and other software requirements. This lets users check whether a kernel matches their GPU instead of dealing with compatibility by guesswork. Burtenshaw describes kernel writers as a category alongside model publishers. In his example, a kernel generated for Qwen3 8B on an H100 produced a 94% speedup. He presents that result as a compatibility demonstration rather than a state-of-the-art result, since a model may have a kernel that is not tuned for the hardware available to a user.
Skills give agents examples and tools for specialized work
Burtenshaw describes skills as file-based context. Files can be opened when needed, versioned, and stored in source control, and agents can use them in the same way. A kernel skill can contain examples of how to write and use kernels, plus scripts for benchmarking and testing them. He says this changes a task from zero-shot to few-shot because the agent receives examples of the expected work. Hugging Face is placing maintained skills inside projects, where project maintainers can update them, while the separate Hugging Face skills repository contains more experimental examples. The project-owned approach gives agents instructions that track the software they are meant to operate.
Burtenshaw introduces Upskill as an open source library for generating skills, generating evaluations for those skills, and comparing models on the same task. This creates a way to test whether a skill works and to choose a cheaper or smaller model for repeated use. He gives comparisons in which GPT-OSS, Kimi, and Haiku differ in accuracy and token use. The point is to measure model behavior under the same skill rather than choosing a model without testing. A team can use the results to iterate on the skill and reduce the cost of an agent workflow while keeping the task definition fixed.
An agent can fine-tune a language model from a single instruction
The second example asks an agent to fine-tune Qwen3 6B on a chain-of-thought dataset and improve the model's chain-of-thought behavior. Burtenshaw says the workflow is integrated with the Hugging Face Hub, including the ability to run the needed GPUs there, and it uses Hugging Face CLI skills. He also mentions an Onslaught version that uses optimized models and can be cheaper. The example is presented as an end-to-end task: the user provides the instruction, while the agent handles the training pipeline. Burtenshaw points viewers to a blog post and existing examples rather than describing a custom system built from scratch.
A research lab can divide model experiments among specialized agents
Burtenshaw builds on Andre Karpathy's Auto Research project, where Claude Code changes a nanoGPT training script, runs experiments, and keeps improvements that raise the measured result. Burtenshaw distributes that single-agent loop across several roles. A researcher searches HF Papers or arXiv and turns papers into hypotheses. A planner keeps a queue of jobs. Workers implement hypotheses as training-script changes. A reporter monitors the jobs and maintains a dashboard. The experiments run as separate branches and jobs, with scores stored in the main branch. This structure lets agents explore several proposed changes instead of having one agent inspect and modify the project one step at a time.
Open data and compute let agents run and inspect experiments
The research system is implemented in OpenCode in the example, with similar implementations in Codex and Claude available in the repository. The planner, researcher, reviewer, and worker agents use templates containing the current state, previous jobs, successful and failed experiments, and parameters they can change. Workers launch jobs through HF Jobs and submit patches back to the repository. The reporter uses Trackio to collect metrics, events, warnings, and notifications. Trackio stores data in Parquet, so an agent can read the underlying data directly and create a different view, such as a Gantt chart. Burtenshaw says the open data layer matters because agents are not limited to the dashboard's built-in visualizations.
Agents need exposed primitives rather than fully hidden abstractions
Burtenshaw's final point is that agents work well with open primitives. He names Trackio and kernels as tools that expose enough of their internals for agents to control them directly. Abstract APIs can be useful, but a layer that agents cannot access becomes a ceiling. He argues that the Hugging Face Hub already has the basic pieces for these workloads: storage, tracking, and compute. With verifiable experiments such as model training or CUDA kernel generation, an agent system can run work, measure it, and preserve the results in repositories. The claim depends on having inspectable files and data, rather than handing the agent an opaque service.
