A tenant sharing an H100 can infer a model's architecture, decoding speed, and workload category by timing its own GPU memory reads.
2
The demonstrated attack does not read victim memory or recover prompts and answers, but it can identify timing patterns created by model layers and speculative decoding.
3
Operators can reduce exposure through tenant placement, explicit choices between MIG and NVLink-connected sharding, and monitoring for repeated probing kernels.
Summary
Branislav Brzak demonstrates timing side channels against shared H100 systems. With MIG, an attacker reads and times only its own memory, repeating the probe while another tenant serves a model. GPU-wide memory barriers affect every L2 partition, so the victim's activity creates small latency changes in the attacker's trace. Frequency analysis reveals token rate and layer-related patterns. Brzak shows that these patterns can distinguish model architectures, including related models with similar decoding speeds. When speculative decoding is enabled, changes in draft-token acceptance also expose workload categories such as prose, code, JSON, and retrieval. A separate experiment across NVLink finds a weaker signal that mainly reveals tempo. The attack does not recover prompts or generated text. Brzak closes with operational responses: separate untrusted tenants from sensitive models, understand the communication and isolation costs of MIG and NVLink, and inspect GPU workloads for repeated probes.
Brzak says MIG isolates compute units, memory capacity, bandwidth quotas, fault errors, and L2 cache partitions. A GPU-wide memory barrier still affects every L2 partition, however. When the victim is busy, the attacker's own operations become slightly slower. The attacker needs no root access or software exploit. It repeatedly times loads from a small buffer in its own memory. The median difference between an idle tenant and one serving an LLM is about three microseconds. That small change is enough to collect a useful trace over time.
The probe observes timing without reading victim data
The attacker's vector is deliberately narrow. It reads only its own memory and times only its own reads. It never touches the victim's memory. The trace contains timing effects caused by shared hardware operations, rather than the victim's data or output. Brzak separates what the experiment can infer from what it cannot recover. The attack can expose the model, response speed, busy periods, token counts, answer length, and some workload categories. It cannot reveal the actual prompts, answers, or contents.
Token timing reveals decoding speed and layer structure
A live H100 trace shows bursts associated with generated tokens. In one example, the frequency spectrum has a fundamental at 81 Hz, corresponding to token rate, with harmonics from the structure of each token's processing. A token trace contains layer invocations as well as extra processing bumps. Speed alone narrows the model but can confuse models with nearly identical rates. Brzak gives Qwen 8B and Llama 3 8B as an example whose token rates differ by only 1.6 percent. Layer-related timing adds a more distinctive signal.
Model depth can be fitted without a trained classifier
The method estimates layer counts by measuring how strongly candidate depth signatures appear in the timing trace. It checks candidates from 16 to 48 layers and chooses the best fit. Brzak says the implementation is under 30 lines of code and has no training set, labels, or configuration. Across 22 traces, the mean distance from an integer was 0.028, which supports the layer-count interpretation. The approach can work on models it has not seen before, including proprietary or unreleased models, because it tests structural signatures rather than matching a learned lookup table.
Related models remain distinguishable through architectural timing
Models that share a base architecture can have similar per-layer timing and still produce distinguishable traces. Brzak discusses Llama 3 8B, Llama 3.1 8B, and DeepSeek R1 based on Llama 8B. Their layer structure remains similar, while small timing and harmonic differences provide additional information. The method also identifies fine-tunes and distillations that retain the base model's architecture and layer count. Brzak reports that only two of the least confident answers in the tested set were wrong.
Speculative decoding can expose workload categories
The same timing channel can classify what the model is doing when speculative decoding is enabled. Brzak tests prose, code, JSON, and retrieval over windows of the previous five seconds. The mechanism is that text predictability changes how many drafted tokens are accepted at each step, which changes step duration. Across 11 tested configurations, the signal appeared with different strengths. The experiment identified workload categories through majority voting, but Brzak makes the condition explicit: speculative decoding is an optional setting and is not the default.
A second experiment places the attacker on another H100 in an eight-GPU server connected by an NVSwitch fabric. The attacker probes memory located on a different GPU over NVLink, so it does not need to share the victim's physical die. The path is much noisier. Brzak says the layer signal falls below the noise, leaving mostly decoding tempo. In the tested setup, five models were identified with 79.2 percent accuracy. Models with similar speeds blurred together, showing why the NVLink result is weaker than the MIG result without being useless.
Operators can detect the probe and choose isolation boundaries
Brzak describes a probe that relaunches one kernel about 3,000 times against the same 40 kilobytes of memory. It returns only about one kilobyte to the CPU because the timing itself carries the information. His product, Delty, detects this pattern by monitoring kernel launches across GPUs and sending suspicious workloads through an alert pipeline. He recommends keeping untrusted tenants away from sensitive models, then choosing between isolation and communication needs. MIG prevents peer-to-peer communication and tensor parallelism, while NVLink-based sharding permits communication but leaves timing signals.