Tag: Inference

• October 8, 2026 Watching a KV cache grow (part 4): reusing it
• October 8, 2026 Speculative decoding: guessing tokens to use idle compute
• October 8, 2026 Batching: how one GPU serves many users, and what a token costs
• October 7, 2026 Kernels: what they are and why inference cares
• October 6, 2026 Sizing inference: from traffic to a GPU count
• October 5, 2026 Inference engines (part 3): the stacks you can't see
• October 5, 2026 Inference engines (part 2): deploying one
• October 5, 2026 Inference engines (part 1): what an engine does and where it sits
• October 4, 2026 Weight precision (part 2): how FP4 works, and what it costs
• October 4, 2026 Weight precision (part 1): how it affects serving

• 1
• 2
• ›
• »