inference
115 talks
Judge a Transaction by the Company It Keeps
Turning My Obsidian Vault Into a Local AI Engineer
From 15% to 90% GPU Utilization: Fix the Data Pipeline, Not the Model
Is Speculative Decoding Worth It? Profiling vLLM on NVIDIA Blackwell
What Is an Inference Engine, Anyway?
What Makes Open Models Fast in Production
Lessons from Generating 12 Trillion Synthetic Tokens
Are LLM Performance Benchmarks Reliable?
Large clusters for small models
Operating Distributed Inference Systems at Scale
Routing LLM Inference in Production: From Engine Signals to Policy
The Frontier AI Inference Cloud for Agents
Two Bugs That Hid in Plain Sight: A vLLM Debugging Detective Story
Vertical Mobility: Inference from MVP to Trillion-Parameter Workloads
Weight Folding, CUDA Streams, and the Bug That Made My Model Speak Backwards
What's New in Inference Engineering
Deep dive on LLM Inference at Scale
Can LLMs Write Fast Multi-GPU Kernels?
KV Cache-Aware Routing and P/D Disaggregation on Kubernetes
Preferences Over Benchmarks: Model Routing
Generative Video at the Speed of Light
Voice agents with Realtime Video
Taking Reinforcement Learning Cross Datacenter
Compression at the Edge
The State of Model Routing
Agents at Scale: Inside MiniMax's Model and the Infrastructure Behind It
The Messy Reality of Scale: Synthetic Data and Pre-Training
Notion's Token Town
The Desktop Frontier
Stop Renting Your Cognitive Infrastructure
Special Topics in Kernels, RL, Reward Hacking in Agents
Computer-Use 2.0: Agents Just Got Multi-Cursor
From fork() to Fleet: Designing an Agent Sandbox Cloud
State of the Union: Why Local, Why Now
Sovereign Escape Velocity: Ownership with Open Models
GPU Cloud Deployment Without Leaving Your IDE
Road to 5 Million Tokens: Breaking Barriers in Long Context Training
Under 5 minutes to a deployed LLM endpoint
Text Diffusion
What Lies Beneath the API
Engineering voice agents: Latency, quality, and scale
Run Frontier AI at Home
Scaling the Next Paradigm of Heterogeneous Intelligence
Prompt to Pipeline: Building with Google's Gen Media Stack
Fast Models Need Slow Developers
Why MLX
Two Roads to Durable Agents: Replay vs. Snapshot
Voice AI: When Is the "Her" Moment?
Why TTS Models Now Look Like LLMs
How Transformers Finally Ate Vision
Accelerating AI on Edge
The Small Model Infrastructure Nobody Built (So We Did)
Gemma 4 Deep Dive
Gemma, DeepMind's Family of Open Models
Running LLMs on your iPhone: 40 tok/s Gemma 4 with MLX
Running LLMs Locally: Practical LLM Performance on DGX Spark
Efficient Reinforcement Learning
Building Cursor Composer
Compilers in the Age of LLMs
Context Platform Engineering to Reduce Token Anxiety
AI Engineer Paris 2025, Day 2
The Next Unicorns: 7 Top AI Startups from the HF0 Residency
Hacking the Inference Pareto Frontier
Infrastructure for the Singularity
Why We Don't Need More Data Centers
Serving Voice AI at $1/hr: Open-Source, LoRAs, Latency, Load Balancing
The Unofficial Guide to Apple's Private Cloud Compute
Introduction to LLM serving with SGLang
Latent Space Paper Club: AIEWF Special Edition (Test of Time, DeepSeek R1/V3)
The Rise of Open Models in the Enterprise
What Every AI Engineer Needs to Know About GPUs
Dream Machine: Scaling to 1m users in 4 days
RL for Autonomous Coding
Thinking Deeper in Gemini
Trends Across the AI Frontier
Optimizing Inference for Voice Models in Production
Foundry Local: Cutting-Edge AI Experiences on Device with ONNX Runtime/Olive
From Mixture of Experts to Mixture of Agents with Super Fast Inference
How fast are LLM inference engines anyway?
Serving Voice AI at Scale
Fun Stories from Building OpenRouter and Where All This Is Going
The Geopolitics of AI Infrastructure
GPU-less, Trust-less, Limit-less: Reimagining the Confidential AI Cloud
Luminal: Search-Based Deep Learning Compilers
The End of Awkward AI Transcriptions
How to Build Your Own AI Data Center in 2025
Finetuning: 500M AI Agents in Production with 2 Engineers
Lessons from building GenAI based applications
The AI developer experience doesn't have to suck: why and how we built Modal
Customized, production-ready inference with open source models
Accelerating Mixture of Experts Training With Rail-Optimized InfiniBand Networking in Crusoe Cloud
System Design for Next-Gen Frontier Models
LLM Quality Optimization Bootcamp
Llama 3 at 1,000 tok/s on the SambaNova AI Platform
Accelerate your AI journey with Azure AI model catalog
Substrate Launch: the API for modular AI
Mastering LLM Inference Optimization From Theory to Cost Effective Deployment
AI Frontiers in Trust and Safety: Combatting Multifaceted Harm on Tinder at Scale
Decoding Mistral AI's Large Language Models
No More Bad Outputs with Structured Generation
Breaking AI's 1-GHz Barrier
Making Open Models 10x Faster and Better for Modern Application Innovation
From Model Weights to API Endpoint with TensorRT-LLM
Build enterprise generative AI apps using Llama 3 at 1,000 tokens/s on the SambaNova AI platform
More Nodes Is All You Need
Unlocking Developer Productivity across CPU and GPU with MAX
Llamafile: Bringing AI to the Masses with Fast CPU Inference
Keynotes & Multimodality Track
GPUs & Inference Track
Keynotes & CodeGen Track
Open Questions for AI Engineering
Harnessing the Power of LLMs Locally
Lessons from the First Viral CLIP App
AI Engineering 201: Inference
Building AI For All