https://hiren.me/tags/rollout/ Recent content in Rollout on Hugo en Wed, 30 Sep 2026 12:30:00 -0700 https://hiren.me/posts/inference-in-production-part-4/ Wed, 30 Sep 2026 12:30:00 -0700 https://hiren.me/posts/inference-in-production-part-4/ <p>The first three parts followed a request through <a href="https://hiren.me/posts/inference-in-production-part-1/" >prefill and decode pools</a>, <a href="https://hiren.me/posts/inference-in-production-part-2/" >the router and autoscaler in front of them</a>, and <a href="https://hiren.me/posts/inference-in-production-part-3/" >what happens when the hardware under them fails</a>. This part is about changing what runs on all of it: a new model, a new checkpoint of the same model, or a new engine version.</p> <p>A rollout does to every replica, on purpose, what the last two posts measured happening by accident. Each old replica is shut down, like part 3&rsquo;s failures, and each new one pays <a href="https://hiren.me/posts/inference-in-production-part-2/#why-a-new-replica-takes-minutes" >part 2&rsquo;s cold start</a>. A rollout goes well when users don&rsquo;t notice either. The sources are project docs and operators&rsquo; own write-ups; the part about draining I checked on Modal.</p>