# Z.ai GLM 4.6: What We Learned From 100 Million Open Source Downloads

Yuxuan Zhang, Z.ai | AI Engineer CODE 2025 | 19:39

Source: https://www.youtube.com/watch?v=m6MF1OR_9kM
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/z-ai-glm-4-6-what-we-learned-from-100-million-open-source-downloads
Published: 2025-11-22
Tags: long-context, multimodal, open-models, reinforcement-learning

## TL;DR
- Z.ai built GLM 4.6 through staged data expansion, moving from broad pre-training to repository-level code, synthetic reasoning, long-context examples, and agent trajectories.
- The SLIME framework switches between tightly coupled and decoupled training depending on whether tasks are short or involve slow, multi-step environments.
- Z.ai found that single-stage reinforcement learning preserved 64,000-token capabilities better than shorter-to-longer RL stages, while clean data and token-weighted coding loss improved results further.

## Summary
Yuxuan Zhang presents GLM 4.6 as the latest result of Z.ai's open-source model program, which had passed 100 million downloads across more than 65 released models. He explains a training pipeline that starts with 15 trillion general tokens, then adds code, reasoning, repository-level context, synthetic reasoning traces, long-context data, and agent trajectories. The talk focuses on SLIME, Z.ai's reinforcement learning framework for agent tasks. It keeps training and inference together for fast short tasks, then separates them when external environments make some trajectories slow. Zhang also describes a two-stage difficulty curriculum, a single-stage 64,000-token RL setup, and token-weighted loss for coding. The multimodal section covers GLM 4.5V's native image resolution, video time tokens, and computer-control abilities. He closes with deployment through open-source inference frameworks, Z.ai's hosted service, and a coding plan.

## Key ideas
### Z.ai's open-source model family had passed 100 million downloads
[00:01](https://www.youtube.com/watch?v=m6MF1OR_9kM&t=1s)
Z.ai began releasing open-source models in 2022, starting with the G30B, and expanded into language, vision, image-generation, and video-generation systems. Zhang says the company had released more than 65 models by 2025. Downloads across Hugging Face, ModelScope, and other platforms had passed 100 million, while GitHub contained more than 51,500 community projects related to GLM models. He presents this ecosystem as community-driven, with GLM 4.6 as the latest flagship release.

### GLM 4.6 was trained by extending context and adding increasingly structured data
[05:08](https://www.youtube.com/watch?v=m6MF1OR_9kM&t=308s)
The pipeline starts with about 15 trillion tokens from web pages, books, Wikipedia, and multilingual material. Z.ai then adds roughly 7 trillion tokens of code and reasoning data, including open-source reports, mathematics, science, and step-by-step reasoning. Mid-training uses repository-level code with multiple files, issues, and pull requests packed into long contexts, reaching 32,000 tokens. The model receives about 500 billion synthetic reasoning tokens, followed by about 100 billion long-context and agent tokens. GLM 4.6 reaches a 200,000-token context window and is trained on multi-step tool calls, search, and code execution.

### SLIME uses different RL system designs for short tasks and agent environments
[07:48](https://www.youtube.com/watch?v=m6MF1OR_9kM&t=468s)
Zhang introduces SLIME as an in-house reinforcement learning framework built on an inference stack and released as open source. For short math or code-completion tasks, training and inference run on the same GPUs, so a new batch can be sampled immediately after an update. Agent tasks can involve browsers, backend APIs, and external responses, so forcing every worker to move at the same speed wastes GPU time. SLIME therefore supports a coupled mode and a decoupled, synchronized mode. In the latter, agent workers write trajectories to a shared buffer while the training side consumes them at its own pace.

### The training system trades precision and speed across different parts of the RL loop
[10:22](https://www.youtube.com/watch?v=m6MF1OR_9kM&t=622s)
Z.ai keeps the main training process in BF16 for stability, while using blockwise FP8 quantization after policy updates and sending the lower-precision version to workers. Zhang says this lets the system run data generation and inference with higher output while retaining BF16 for training. The shared data buffer separates the agent environments from the trainer when needed. Slow tasks therefore do not block the complete training pipeline, while short tasks can still use immediate sampling and updates.

### A difficulty curriculum gives RL useful rewards before exposing the model to hard problems
[10:49](https://www.youtube.com/watch?v=m6MF1OR_9kM&t=649s)
Z.ai uses a two-stage curriculum rather than keeping one fixed dataset throughout training. Stage one contains medium-difficulty problems, where rewards vary enough to provide a meaningful learning signal. Stage two moves to extremely hard problems. Zhang says that with 512 samples, the model can still occasionally produce a correct solution on these problems, and the resulting curve continues to improve. The comparison in his slide shows the hard-problem switch outperforming a method that stays at medium difficulty.

### Single-stage 64,000-token RL protects long-context ability
[12:07](https://www.youtube.com/watch?v=m6MF1OR_9kM&t=727s)
Zhang describes a common multi-stage approach that increases reasoning length through stages such as 16,000, 32,000, 48,000, and finally 64,000 tokens. Z.ai found that a model already trained with 64,000-token supervised fine-tuning can forget long-context ability when RL starts with shorter outputs. The final 64,000-token stage does not fully recover that loss. Z.ai instead starts RL directly at 64,000 tokens in a single stage, and Zhang says this approach clearly outperforms the multi-stage curve.

### Token-weighted loss improves coding RL and discourages short template answers
[12:35](https://www.youtube.com/watch?v=m6MF1OR_9kM&t=755s)
For coding, Z.ai compares sequence-mean loss with a token-weighted loss that averages over tokens. Zhang says the token-weighted version converges faster and more steadily. It also reduces the chance that the model generates a very short template answer simply to obtain a reward. In scientific reasoning experiments on GPQA, Z.ai compares a small set of expert-verified questions with mixed-quality data. The smaller clean dataset performs better, leading Zhang to say that data quality matters more than raw size for this setting.

### GLM 4.5V keeps native image shape and adds time markers for video
[14:02](https://www.youtube.com/watch?v=m6MF1OR_9kM&t=842s)
GLM 4.5V supports image and video understanding, with the visual input kept close to its original resolution and aspect ratio instead of being forced into a fixed square. Zhang says this matters for screenshots, long vertical images, and slides. For video, the model receives a time-index token after each frame so it can distinguish temporal order. The model also supports GUI-agent actions through mouse, keyboard, touch, browsers, computers, and mobile environments.

### Z.ai offers open weights, hosted access, and coding integrations
[16:07](https://www.youtube.com/watch?v=m6MF1OR_9kM&t=967s)
Users can run GLM 4.6 and GLM 4.5V with their open weights through inference frameworks. Zhang names vLLM, SGLang, LlamaFactory, and MS-SWIFT as part of the surrounding tooling, and says integrations were available around release. Because GLM 4.6 has more than 355 billion parameters, Z.ai also provides access through its website and API. The company offers a GLM coding plan that connects the model with tools and plugins for coding assistants, alongside technical reports, Discord, GitHub, and community events.

## Notable quotes
- "2025 is our open source year." (00:58)
- "So the nice thing is even if some tasks super slow they don't block the whole training pipeline." (09:55)
- "We start directly with 64,000 token and train in one single stage." (12:35)
- "For scientific reasoning data quality really matters more than raw size." (13:52)

## Tools & references mentioned
- GLM 4.6
- GLM 4.5V
- SLIME
- LMSYS Chatbot Arena
- GPT-5
- Claude 4.5
- Claude 4
- GPT-4o
- DeepSeek-V3.2
- Qwen
- Kimi K2
- vLLM
- SGLang
- Hugging Face
- ModelScope
- LlamaFactory
- MS-SWIFT
- GPQA
- Z.ai
- GitHub
- Discord

## Who should watch
- You are training an open model and need concrete choices for long-context data, coding data, and reinforcement learning stages.
- You are building coding or browser agents whose trajectories can take very different amounts of time.
- You need to run a large open model and want to compare local inference frameworks with hosted access.

## Related talks

- [Lessons from Trillion Token Deployments at Fortune 500s](https://aietalks.com/talks/lessons-from-trillion-token-deployments-at-fortune-500s) (Alessandro Cappelli, Adaptive ML, 18:35)
- [The Wild World of AI: 6 Months That Changed Everything](https://aietalks.com/talks/the-wild-world-of-ai-6-months-that-changed-everything) (Simon Wilson, 01:57)
- [Stop Making Models Bigger, Make Them Behave](https://aietalks.com/talks/stop-making-models-bigger-make-them-behave) (Kobie Crawford, Snorkel, 20:56)
- [Real ROI: Lessons from Enterprises That Have Already Succeeded with LLMs at Scale](https://aietalks.com/talks/real-roi-lessons-from-enterprises-that-have-already-succeeded-with-llms-at-scale) (Raza Habib, Humanloop, 20:01)
- [AI Engineering 201: The Rest of the Owl](https://aietalks.com/talks/ai-engineering-201-the-rest-of-the-owl) (Charles Frye, Full Stack LLM Bootcamp, 56:57)