[WIP] feat: Add storage checkpoint index for KV offloads#668
Draft
albertoperdomo2 wants to merge 7 commits into
Draft
[WIP] feat: Add storage checkpoint index for KV offloads#668albertoperdomo2 wants to merge 7 commits into
albertoperdomo2 wants to merge 7 commits into
Conversation
github-actions
Bot
requested review from
hyeongyun0916,
liu-cong,
sagearc and
yankay
June 15, 2026 13:11
albertoperdomo2
force-pushed
the
feat/storage-kv-cache-index
branch
from
June 15, 2026 13:23
6b934e1 to
404fbea
Compare
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
albertoperdomo2
force-pushed
the
feat/storage-kv-cache-index
branch
from
June 28, 2026 15:28
25edee1 to
c223ae8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a write-path-only storage checkpoint index so the Go indexer can track which KV blocks exist on shared storage (FS backend). This is the foundation for storage-aware scoring, pre-fetch signals... and any work related to it.
Python side:
StorageEventPublishernow requires ablock_sizeparameter, populated fromhash_block_size * gpu_blocks_per_filein the FS backend manager. This value is emitted in field[4]ofBlockStoredevents so the Go side can derive stride at runtime without duplicated configuration.block_size=0explicitly (it only emits removal events).Go side:
StorageIndexinterface (AddCheckpoint,HasCheckpoint,RemoveCheckpoint,Clear,SetStride,Stride) backed by a Cuckoo filter inpkg/kvcache/kvblock/storage_index.go.Indexercreates and exposes theStorageIndexwhenStorageIndexConfig.Enabledis true.Pool.processEventBatch()detects storage-tier events viaisStorageTier()and routes them to the storage index, bypassing the normal engine-key-to-request-key resolution path. Storage events carry canonical request-key checkpoints directly.BlockStoredevent:event.BlockSize / tokenProcessor.BlockSize(). Thread-safe initialization viastorageStrideMumutex with consistency validation on subsequent events.storageIndexis nil, storage events are silently dropped and never fall through into GPU/CPU processing.SHARED_STORAGEpod identifier is excluded from scoring to prevent it from appearing as a routable pod.Testing
Unit tests covered, now benchmarking for potential regressions (WIP).
Related Issues
Closes #520