DGX Spark lets developers run and benchmark open-source LLMs locally with 128 GB of unified memory and NVIDIA's production software stack.
2
Quantization has a large effect on performance: the 14B NVFP4 model reached 20.19 tokens per second, compared with 8.40 tokens per second for the 14B base model.
3
Local compute fits steady workloads, privacy-sensitive data, and rapid prototyping, while the same workflow can later move to a data center or cloud.
Summary
Mozhgan Kabiri chimeh presents a practical benchmark of open-source language models running on an NVIDIA DGX Spark. Her setup uses vLLM inside an NVIDIA-optimized container, with an automated harness that isolates environments, runs three warm-ups, logs GPU metrics, and stores versioned results. She measures completion throughput and time to first token across models from 1.5B to 14B parameters. The 1.5B instruct model reaches 61.73 tokens per second. A 14B NVFP4 model reaches 20.19 tokens per second, while the 14B base model reaches 8.40. She also reports that the 14B NVFP4 model produces its first token 3.4 times faster than the unoptimized 14B model. The talk argues that unified memory capacity only determines what can fit. Memory bandwidth and quantization determine how usable the model feels. Local hardware makes sense for steady workloads, privacy-sensitive data, and rapid prototyping, with a path to later deployment in the cloud or a data center.
Local infrastructure can reduce dependence on shared cloud and data-center resources
Mozhgan Kabiri chimeh describes two practical problems for AI developers: running out of memory and lacking the right software stack. Sending work to a shared cloud or data center can introduce cost concerns, data-residency issues, uncertain latency, and delays when workloads compete for infrastructure. She argues that some of this workflow should move closer to the developer. The goal is not to replace cloud systems. It is to make development and iteration possible on a local machine before scaling elsewhere.
DGX Spark combines local capacity with NVIDIA's production software stack
DGX Spark uses the GB10 Grace Blackwell superchip, which combines CPU and GPU resources with unified memory. The system has 128 GB of unified memory and supports NVFP4. Kabiri chimeh says this allows models of up to around 200 billion parameters to run locally. It also uses the NVIDIA AI software stack found in production environments, so developers can move workflows from a desktop to a data center or cloud with minimal changes.
A reproducible benchmark needs the same protocol for every model
Kabiri chimeh serves quantized models of different sizes with vLLM inside an NVIDIA-optimized container. Her automated harness uses Docker for environment isolation, three mandatory warm-up runs, and background GPU metric logging at one-second intervals. Each execution gets a unique directory based on a timestamp and sanitized model ID. The stored artifact includes metadata, the endpoint response, and benchmark text results. This gives each run a record that can be checked later rather than leaving only a chart of final numbers.
Time to first token captures the responsiveness users actually feel
The benchmark measures end-to-end behavior, but Kabiri chimeh focuses on time to first token because it determines whether an application feels responsive. Her code handles the streaming response from the vLLM server and timestamps the first returned chunk. This differs from calling an API and waiting for the complete result. The measurement therefore captures the delay before a user sees the model begin responding, which can matter even when total completion time is acceptable.
Model size creates a large throughput cost on the local system
The completion-throughput results show a steep decline as models grow. The 1.5B instruct model reaches 61.73 tokens per second. The 14B NVFP4 model reaches 20.19 tokens per second despite being nearly ten times larger. The 14B base model reaches only 8.40 tokens per second. Kabiri chimeh uses these results to show that local performance depends on the model and its precision format, rather than on hardware capacity alone.
Kabiri chimeh compares the 14B base and 14B NVFP4 models in both throughput and startup responsiveness. The NVFP4 model reaches 20.19 tokens per second, compared with 8.40 for the base model. It is also 3.4 times faster to first token than the unoptimized 14B base model. Her conclusion is that on Blackwell hardware, quantization format can matter as much as the hardware itself when developers choose a model for local use.
Memory capacity determines what fits, while bandwidth affects how it runs
The 128 GB unified memory can hold very large models, with DGX Spark supporting models of up to around 200 billion parameters according to the talk. That capacity does not guarantee high throughput. Kabiri chimeh explains that performance is governed by how efficiently data moves through memory. NVFP4 improves the amount of model capability available per byte, which helps a 14B model feel closer to a much smaller model in responsiveness.
Local compute fits prototyping and sensitive steady workloads
Kabiri chimeh recommends DGX Spark for steady-state workloads, privacy-sensitive data, and rapid prototyping. Developers can build and fine-tune locally with the same software stack used for deployment elsewhere. She points viewers to build.nvidia.com/spark for the playbooks and software stack used in the benchmarks. Her proposed workflow is to iterate locally, then scale to a data center or cloud when the application is ready.
