https://hiren.me/tags/nvlink/ Recent content in NVLink on Hugo en Fri, 25 Sep 2026 11:10:00 -0700 https://hiren.me/posts/inference-disaggregation-part-2/ Fri, 25 Sep 2026 11:10:00 -0700 https://hiren.me/posts/inference-disaggregation-part-2/ <p><a href="https://hiren.me/posts/inference-disaggregation-part-1/" >Part 1</a> showed why you&rsquo;d split prefill and decode onto different GPUs: a long prefill on the same GPU makes everyone who is already decoding there wait. Splitting has a price, though. The KV cache that prefill builds has to move to the GPU that does decode. The <a href="https://hiren.me/posts/watching-a-kv-cache-grow-part-3/" >offload post</a> (part 3 of Watching a KV cache grow) predicted that moving a 32k-token cache would add about 3% to prefill over a 400 Gb/s link and about 12% over 100 Gb/s. This post measures the fastest case, within one machine, on real hardware.</p>