NVIDIA Snapshot restores initialized GPU workers instead of rebuilding them
NVIDIA Snapshot captures initialized CPU and GPU state so an inference worker can skip repeated weight loading, compilation and graph capture. On the vendor’s B200 tests with VAST NFS storage, restores took 3.6–42.7 seconds across the tested configurations. Those measurements exclude machine provisioning and container-image loading, so they describe worker restoration rather than the entire time to acquire new serving capacity.
NVIDIA Technical Blog · Nov 10, 2026
Why it matters
Reusing initialized execution state can reduce a different part of startup latency than caching a model’s weight files alone.
More context
The open-source Kubernetes component uses snapshot resources and a small workload pause/resume handshake. Try a supported inference-engine example on matching GPU models and drivers. The benchmark releases the request KV cache before checkpointing; it is not part of the artifact. Budget checkpoint storage and GPU infrastructure separately.
Keep in mind
Compatibility and storage bandwidth constrain restores. These are vendor measurements, not a general recovery-time guarantee or proof that arbitrary running GPU applications can be migrated unchanged.
Original read