Posts

October 3, 2026 Inference in production (part 5): workload types
September 30, 2026 Inference in production (part 4): rolling out a new model
September 30, 2026 Inference in production (part 3): when hardware goes bad
September 30, 2026 Inference in production (part 2): the gateway, the router and the autoscaler
September 29, 2026 Inference in production (part 1): disaggregated inference on B200 and GB200 NVL72
September 25, 2026 Inference disaggregation (part 2): moving the cache
September 25, 2026 Inference disaggregation (part 1): why split prefill and decode
September 24, 2026 Watching a KV cache grow (part 3): move it or rebuild it?
September 24, 2026 Watching a KV cache grow (part 2)
September 24, 2026 Watching a KV cache grow (part 1)