Gemma 4 spans on-device effective 2B and 4B models, a 26B mixture-of-experts model, and a 31B dense model.
2
Interleaved local and global attention, grouped query attention, and per-layer embeddings reduce the memory and compute costs of running the models.
3
Variable image budgets, native audio input, and support for function calling and structured JSON make the models suitable for multimodal and agentic applications.
Summary
Cassidy Hardin presents Gemma 4 as a four-model family covering local devices and larger reasoning workloads. The 31B dense model and 26B mixture-of-experts model support long context, coding, function calling, structured JSON, and agentic workflows. The smaller effective 2B and 4B models accept text, images, and audio while producing text, with design choices aimed at phones and laptops. Hardin explains the architecture behind the efficiency claims: local attention is interleaved with global attention, grouped query attention reduces the cost of global layers, and per-layer embeddings move a large embedding table from VRAM to flash memory. The vision system accepts variable aspect ratios and resolutions, so developers can spend more image tokens on OCR or object recognition. Audio uses a tokenizer and a conformer encoder for speech recognition and translation. Developers can self-host through Hugging Face, Kaggle, or Ollama, or use larger models through AI Studio and Vertex AI.
Gemma 4 covers local devices and larger reasoning workloads
Cassidy Hardin describes four Gemma 4 sizes. The effective 2B and 4B models target phones, iPads, and laptops. The larger options are a 26B mixture-of-experts model and a 31B dense model. The 26B model uses 3.8 billion parameters during a forward pass, while the 31B model has a 256K context length and supports thinking, function calling, and structured JSON outputs. Hardin says the larger models rank in the top six of open-source models on the LM Arena, and that the 31B model ranked number three on the global arena AI leaderboard.
Apache 2.0 licensing makes Gemma easier to integrate into development
Gemma 4 moves to an Apache 2.0 license. Hardin says the change was deliberate, so everyday developers can use the models through the development lifecycle, from initial testing to deployment. The talk presents this licensing change alongside the model release, rather than as a separate deployment product. Her stated aim is for developers to integrate Gemma into their own development process and build within the Gemma ecosystem.
Local and global attention trade memory cost against context range
Gemma 4 interleaves local and global attention layers. The larger models use a 5:1 ratio, while the effective 2B model uses 4:1. Local layers attend through a sliding window of 1,024 tokens in larger models and 512 tokens in smaller models. Global layers attend to all preceding tokens, and the final layer is always global. Hardin says local attention improves efficiency, while global attention remains expensive because it needs access to the full preceding sequence.
Grouped query attention reduces the cost of global layers
Gemma 4 uses grouped query attention to reduce the memory and inference cost of attention. In local layers, two queries share key and value heads. In global layers, eight queries share them. The model compensates for the reduced number of key and value heads in global layers by doubling their length from 256 to 512. Hardin says the eight-to-one grouping improves global-layer efficiency without a large increase in serving cost or memory use.
The MoE model activates eight of 128 experts on each pass
The 26B Gemma model replaces the standard feedforward network with a mixture-of-experts layer. It has one shared router expert and 128 total experts. Eight experts are selected on each forward pass, and each expert is a small feedforward neural network. The shared expert runs on every pass and is three times the size of the regular experts. This gives the model a larger set of representational parameters while limiting the parameters used for an individual inference.
Per-layer embeddings move storage pressure from VRAM to flash memory
Hardin explains that the effective 2B and 4B models have more representational parameters than the number needed to operate them. The effective 2B model uses 2.3 billion parameters while having 5.1 billion representational parameters. Per-layer embeddings give every layer its own embedding table, with an embedding dimension of 256. Gemma stores these tables in flash memory instead of VRAM, which Hardin identifies as a major constraint on phones and laptops. At the end of each decoder block, the model looks up the per-layer embedding and projects it to the full embedding size.
Variable image shapes let developers choose the image token budget
Gemma 4 vision encoders support variable aspect ratios and variable resolutions across five resolutions. Developers can choose the resolution and soft token budget for images. Higher budgets suit OCR and spatial object recognition, while smaller budgets suit applications that use fewer multimodal features. Hardin contrasts this with Gemma 3's pan-and-scan approach, which split one image into multiple square images. Gemma 4 can process a varying number of patches based on the supplied image. A 3-by-3 patch grid becomes one embedding, and a token budget of 280 corresponds to 2,520 patches.
Audio input combines a tokenizer with a conformer encoder
The effective 2B and 4B models add audio input for speech recognition and translation. Raw audio first passes through a mel spectrogram, which is divided into mel chunks and downsampled through two convolutional layers. The result is a sequence of audio soft tokens. Those embeddings go into a 305-million-parameter conformer that processes embeddings rather than tokens. The conformer follows patterns from the dense and MoE models, with an added convolutional layer.
Developers can self-host Gemma or use hosted larger models
Hardin gives two ways to start. Developers can download and self-host the models through Hugging Face, Kaggle, and Ollama. The 31B and 26B models also have cloud-hosted options through AI Studio and Vertex AI. She connects the hosted models with prototyping, agentic workflows, and testing function calling. The smaller models are positioned for local use, while the larger models target more complex reasoning, coding, and agentic tasks.
