Skip to content

[Bug] Incorrect chunk hit bitmask indexing in UCMBlendConnector._get_req_chunk_hit when prefix cache partially hits #906

Description

@SuperMarioYL

Describe the bug

In UCMBlendConnector._get_req_chunk_hit (ucm/integration/vllm/blend_connector.py, lines 222-259), the chunk lookup results are concatenated with prefix cache results and then sliced by global block indices (start_blk_idx:end_blk_idx), but the two hash sequences use fundamentally different hash semantics, causing an index misalignment when both prefix cache and chunk cache have partial hits.

Root Cause Analysis

Two different hash spaces are mixed:

  1. prefix_block_hashes (line 231) — generated via chained hashing where each block hash depends on all previous blocks (positional).
  2. req_chunks_hashes (line 244) — generated per-chunk independently, each chunk starts fresh from self._seed (content-based).

The problematic sequence:

# Line 244: slice req_chunks_hashes starting from pc_hit_blocks
chunk_lookup_results = self.store.lookup(req_chunks_hashes[pc_hit_blocks:])

# Line 247: concatenate prefix hits + chunk lookup results
chunk_lookup_results = pc_lookup_results[:pc_hit_blocks] + chunk_lookup_results

# Lines 249-252: slice by global block index
chunk_meta.store_hits = chunk_lookup_results[
    chunk_meta.start_blk_idx : chunk_meta.end_blk_idx
]

When pc_hit_blocks > 0, the req_chunks_hashes[pc_hit_blocks:] slicing assumes that chunk hashes and prefix block hashes are indexed in the same linear space. But since chunk hashes are independent (not chained from the prefix), position pc_hit_blocks in req_chunks_hashes does not correspond to position pc_hit_blocks in the overall block sequence.

Expected behavior

The store_hits bitmask for each ChunkMetaData should accurately reflect which blocks exist in the store, using correctly aligned indices.

Actual behavior (when prefix partially hits + chunk cache also hits)

The store_hits bitmask is shifted by the misalignment between the two hash spaces, potentially causing:

  1. Loading non-existent blocks — the connector tries to load blocks at wrong store keys, resulting in errors or silent data corruption.
  2. Skipping existing blocks — cached blocks that should be reused are missed, wasting computation.
  3. Applying delta-rope to wrong blocks_post_process_chunk_cache operates on misaligned KV cache, producing incorrect inference output.

This is likely related to #867 (incorrect token generation after prefix cache hit).

Suggested investigation

  1. Compare req_chunks_hashes generation (in _build_blend_meta or caller) with prefix_block_hashes generation to confirm the hash semantic mismatch.
  2. Add logging to dump prefix_block_hashes, req_chunks_hashes, pc_hit_blocks, and the resulting chunk_lookup_results for a request with both prefix and chunk hits.
  3. Consider using chunk-relative indices rather than global block indices for slicing after concatenation, or ensure both hash sequences are in the same space.

Environment

Code review of develop branch at commit f531458.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions