# Why MLX

Prince Canuma, Neywa Labs | AI Engineer Europe 2026 | 23:10

Source: https://www.youtube.com/watch?v=zTLJNHj0DeQ
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/why-mlx
Published: 2026-05-11
Tags: edge, inference, multimodal, open-source

## TL;DR
- MLX lets developers run vision, audio, language, and multimodal models locally on Apple Silicon devices.
- On-device AI can support accessibility, offline systems, voice applications, video generation, and robots that need to respond without a cloud connection.
- Turbo Quant reduces KV cache use by four times and makes million-token context possible on compatible local hardware.

## Summary
Prince Canuma presents MLX as an Apple Silicon framework for running AI models locally, with examples across vision, audio, language, multimodal input, video, and robotics. His motivation comes from his father's blindness and the unreliable internet and poor subscription options available where his father lives in Africa. MLX supports accessibility tools that describe surroundings, voice control, speech-to-speech pipelines, and native Swift applications. Canuma also demonstrates real-time object detection, background blur, local image analysis, community-built visual reasoning systems, chained video generation, and a voice application. He describes a robot using MLX Audio and MLX Vision for perception and real-time voice cloning. The talk also covers practical limits. MLX uses the GPU rather than Apple's Neural Engine, and local models do not yet match the performance of leading cloud models. He closes with Turbo Quant, which reduces KV cache requirements by four times and enables up to one million tokens of context on device, depending on the model and hardware.

## Key ideas
### On-device AI matters when connectivity and subscriptions are unreliable
[01:03](https://www.youtube.com/watch?v=zTLJNHj0DeQ&t=63s)
Canuma connects his interest in local AI to his father's blindness and his life in Africa, where internet access is less reliable and subscription plans are poor. Cloud compute did not solve the accessibility problem he wanted to address. He concluded that an assistant running on the device could help his father read and understand his surroundings without depending on a network connection. He also frames local inference as a way to avoid paying for some cloud subscriptions, leaving the user with the cost of electricity instead.

### MLX brings a PyTorch-like array framework to Apple Silicon
[02:15](https://www.youtube.com/watch?v=zTLJNHj0DeQ&t=135s)
Canuma discovered MLX on GitHub in 2023 and describes it as an array framework for Apple Silicon, comparable in role to PyTorch or TensorFlow for that hardware. He says the project had reached more than 1.5 million downloads and more than 4,000 ported models at the time of the talk. The community supports new open-source models quickly, including Gemma 4. Canuma says models can run on Macs, iPhones, and iPads, and that even very large models can run on older M1 hardware with reasonable speeds in some cases.

### Vision and omni models can help people understand and control their surroundings
[03:15](https://www.youtube.com/watch?v=zTLJNHj0DeQ&t=195s)
Canuma first built goggles that described objects in front of a user, then moved the idea to phones with MLX VLM. A user can point an iPhone at something and ask what is there. Omni models add audio input, allowing someone who cannot type easily to speak commands, control the camera, and ask about the surrounding scene. He positions this combination as an accessibility tool, especially for users who can speak but cannot rely on text input.

### Audio models support fast speech, transcription, and custom voice pipelines
[05:35](https://www.youtube.com/watch?v=zTLJNHj0DeQ&t=335s)
Canuma describes Marvis, a custom text-to-speech model that generates audio in less than 100 milliseconds. MLX Audio also supports real-time speech-to-text and speech-to-speech, so an application can listen and respond aloud. Developers can use Python or Swift, with Swift intended for fully native applications. The speech pipeline is modular: developers choose an automatic speech recognition model, a language model, and a text-to-speech model. That lets the same application be adjusted for different Apple Silicon hardware, from a first-generation M1 to newer machines.

### Local vision models can perform real-time detection without an internet connection
[07:59](https://www.youtube.com/watch?v=zTLJNHj0DeQ&t=479s)
The first live demo uses Roboflow's RFDetector model to analyze a camera feed on Canuma's Mac. It detects objects continuously, including a glass, although it labels the glass as wine. He then shows native background blur and real-time mask detection. In a second demo, MLX VLM runs Gemma 4 locally through a chat interface, analyzes an image, and describes details from it. Canuma turns off the internet during the demonstration and says the model uses the machine's GPU.

### Community projects extend MLX into security, video, and native voice applications
[12:00](https://www.youtube.com/watch?v=zTLJNHj0DeQ&t=720s)
Canuma shows grounded visual reasoning systems that detect fires or selected objects in video without internet access. He describes possible uses such as home security and reviewing dash-cam footage on a MacBook. Another project uses MLX Video to generate cartoons from text prompts on device. The creator chains successive video generations so the output can form a coherent story rather than being one isolated clip, and Canuma says the system can run on a MacBook with 16 GB of video RAM. He also shows a native application called Locally that speaks to users through MLX Audio and Marvis TTS.

### MLX can give robots local perception and real-time voice interaction
[15:39](https://www.youtube.com/watch?v=zTLJNHj0DeQ&t=939s)
Canuma describes using MLX Audio and MLX Vision with a Richie Mini robot. The robot receives audio and camera input, then responds using locally powered capabilities. In the demonstration, it performs real-time voice cloning of the Jarvis voice from Iron Man. Canuma's broader example is an agent that can hear, see, and speak on an iPhone, iPad, Mac, or robot. The point is practical: a robot can respond without sending every interaction to a remote service.

### MLX uses the GPU, while model quality and hardware still set limits
[17:03](https://www.youtube.com/watch?v=zTLJNHj0DeQ&t=1023s)
In the questions, Canuma explains that MLX uses the GPU rather than the Apple Neural Engine. Developers who want Neural Engine support need Core ML, which he says is currently difficult to use, partly because of private API issues. He recommends Mac top for watching GPU and CPU usage during inference. For omni models, he names the e variants of Gemma 4 and Qwen 3 Omni. He is direct about quality limits: local models do not yet match Claude 3 or Claude 4.5 Opus, so applications need realistic performance expectations.

### Turbo Quant cuts KV cache use and enables very long local context
[20:34](https://www.youtube.com/watch?v=zTLJNHj0DeQ&t=1234s)
Canuma says his recent Turbo Quant implementation reduces KV cache or RAM use by four times. He gives an example in which a full model needs almost 1 GB of KV cache. With Turbo Quant, that requirement is reduced by four times while maintaining similar quality in the comparison he describes. He also says the approach can make one-million-token context available fully on device, depending on the model and hardware. At around 300,000 tokens of context, he reports that throughput nearly doubles in his results.

## Notable quotes
- "I thought on device is the future." (02:15)
- "You can now build fully native applications enabled by audio intelligence as well as vision intelligence." (06:53)
- "MLX uses the GPU, not the neural engine." (17:03)
- "You can run inference on hundreds of images in parallel." (20:35)
- "By using Turbo Quant you can reduce that by 4x." (21:52)

## Tools & references mentioned
- MLX
- Apple Silicon
- PyTorch
- TensorFlow
- Meta
- Google
- Gemma 4
- MLX VLM
- LM Studio
- Liquid AI
- Marvis
- MLX Audio
- MLX Video
- Roboflow
- RFDetector
- Core ML
- Mac top
- Qwen 3 Omni
- Turbo Quant
- Richie Mini
- Iron Man Jarvis

## Who should watch
- You are building an AI feature for an iPhone, iPad, Mac, or Apple Silicon device and want examples of local vision, language, and audio inference.
- Your product must keep working with unreliable connectivity, or you are building a robot or agent that cannot depend on a cloud round trip.
- You want practical guidance on MLX model choices, GPU monitoring, Core ML limitations, and Turbo Quant's effect on long context.

## Related talks

- [Running LLMs on your iPhone: 40 tok/s Gemma 4 with MLX](https://aietalks.com/talks/running-llms-on-your-iphone-40-tok-s-gemma-4-with-mlx) (Adrien Grondin, Locally AI, 10:51)
- [Foundry Local: Cutting-Edge AI Experiences on Device with ONNX Runtime/Olive](https://aietalks.com/talks/foundry-local-cutting-edge-ai-experiences-on-device-with-onnx-runtime-olive) (Emma Ning, Microsoft, 22:52)
- [Accelerating AI on Edge](https://aietalks.com/talks/accelerating-ai-on-edge) (Chintan Parikh & Weiyi Wang, Google DeepMind, 23:58)
- [The Desktop Frontier](https://aietalks.com/talks/the-desktop-frontier) (Ahmad Osman, Osmantic, 18:02)
- [Run Frontier AI at Home](https://aietalks.com/talks/run-frontier-ai-at-home) (Alex Cheema, EXO Labs, 1:45:02)