Hugging Face moved model search from regex queries to Atlas Search, which uses Apache Lucene and autocomplete over precomputed search tokens.
2
MongoDB stores Hub metadata while model files live in S3, and a hidden analytics node handles heavy queries away from production traffic.
3
The Hub scales Kubernetes deployments from 10 to 500 pods, then adds nodes with CastAI, while KEDA uses request queues and event loop utilization for application-level scaling.
Summary
Arek Borucki explains how Hugging Face operates the Hub as its catalog grew to 3 million public models, 1 million datasets, and more than 14 million users. Search became the hardest part because an approach that worked at 20,000 models produced latency problems at a much larger scale. The Hub now tokenizes model names when they are inserted, stores those tokens in a read-optimized MongoDB collection, and uses MongoDB Atlas Search with Apache Lucene. MongoDB holds metadata, while model artifacts are stored in S3, allowing the two workloads to scale separately. A seven-node cluster spreads reads, with a hidden replica reserved for reports and heavy queries. Borucki also describes the planned move from replica sets to sharding, plus two layers of Kubernetes scaling. HPA adjusts pods, CastAI adds nodes, and KEDA is being adopted to scale on request rate and event loop utilization.
Search becomes the main scaling problem as the catalog grows
Hugging Face grew from 20,000 models to 3 million, alongside 1 million datasets and more than 14 million users. At 20,000 models, even an unindexed query was fast enough that users would not notice. At 3 million, the same method created slow results, and users expected search to feel instant. Borucki says P99 latency matters more than P50 because even 1% of 14 million users would mean 140,000 people encountering slow search. The team responded by denormalizing read data, optimizing MongoDB collections, moving to Apache Lucene-based full-text search, and improving autoscaling.
Metadata and model artifacts use separate storage paths
A Hub request reaches the API running on Kubernetes and then MongoDB Atlas, which is the source of truth for metadata. MongoDB stores repository, model, dataset, bucket, space, configuration, billing, and access-control information. It does not store the model files themselves. Artifacts such as tokenizer files, card assets, and configuration files are kept in cloud object storage such as AWS S3. This division lets Hugging Face scale metadata, binary storage, and compute independently. Each part can be tuned for its own workload instead of forcing database capacity to grow with the size of the stored artifacts.
Precomputed tokens make model-name autocomplete practical
For a search such as "llama", the Hub reads from a separate denormalized MongoDB collection used for reads and listings. Model names are tokenized when a model is inserted, rather than during each query. A name such as "meta-llama/llama 3.1 8b" is split into smaller tokens and stored in an array in the MongoDB document. Atlas Search, using Apache Lucene underneath, applies autocomplete to those tokens. This avoids rebuilding the searchable representation for every request and keeps the search path focused on a collection designed for read traffic.
Atlas Search replaced a regex-based query and ranking path
The earlier implementation used MongoDB's find method with a regex operator over the search-token arrays. Results were sorted by trending score, which Borucki describes as being calculated every five minutes from downloads and likes. That approach worked while the dataset was small, but regex searches developed latency problems as the catalog grew. The replacement uses an aggregation pipeline with MongoDB's $search operator, pointing to an Atlas Search index backed by Apache Lucene. The query uses the model-search autocomplete index and still sorts by trending score. Borucki says this removed the search-bar latency problems.
A hidden replica absorbs reporting and heavy database work
The Hub uses a seven-node MongoDB cluster. Writes, including inserts, deletions, and updates, go to the primary, while reads can be spread across other machines. One hidden analytics node replicates data from the primary but is not visible to the application driver. Hugging Face connects to it directly for reporting and heavy queries. Secondaries handle queries that do not need the latest data, complex aggregations, change streams, and ad hoc or experimental work. Queries that require strong consistency remain on the primary. Borucki's rule is that the primary should focus on operations only it can perform, while other work moves elsewhere.
Sharding will split the catalog across independently replicated pieces
Borucki says a single MongoDB replica set will eventually be insufficient for the Hub's growth, so the next step is sharding. A replica set keeps the full dataset on every node, while a sharded cluster stores only part of the dataset on each shard. Each shard has its own primary and secondaries. Additional shards can be added, and MongoDB's balancer distributes data across them. A shard key must be selected, and Borucki notes that this is not a trivial decision. Sharding is intended to scale CPU, memory, storage, reads, and writes horizontally.
The Hub uses separate layers for pod and node autoscaling
The Hub runs on Kubernetes and its deployment can scale from 10 to 500 pods depending on traffic. Horizontal Pod Autoscaler adds or removes pods based on CPU and memory thresholds. If new pods are pending because the cluster has no free nodes, CastAI adds Kubernetes nodes so the scheduler can place them. Borucki describes this as two layers: deployment-level scaling for pods and infrastructure-level scaling for machines. This avoids requiring the team to overprovision the cluster for every possible traffic spike.
KEDA can scale on the request queue that CPU metrics miss
Hugging Face plans to migrate from HPA to KEDA, Kubernetes Event-driven Autoscaling. HPA watches CPU and memory, while KEDA can use application metrics such as requests per second and event loop utilization. Borucki gives the case of a pod with low CPU but a high request queue. HPA may not react to that condition, while KEDA can see the queue through an application-level signal. The goal is to make scaling respond to the actual work waiting to be processed instead of relying only on resource utilization.
"Primary should focus on what only primary can do. Anything else can be pushed to different machines."16:23
Who should watch
You operate a model or dataset catalog whose search latency is rising as the index grows.
Your database receives a mix of latency-sensitive production queries, reports, aggregations, and change-stream workloads.
You are tuning Kubernetes autoscaling and need signals that reflect request queues or event-loop pressure rather than CPU and memory alone.
