Skip to content

KAFKA-20219: Headers store with caching enabled should reject queries#21941

Merged
mjsax merged 5 commits intoapache:trunkfrom
zheguang:zheguang-KAFKA-20219
Apr 8, 2026
Merged

KAFKA-20219: Headers store with caching enabled should reject queries#21941
mjsax merged 5 commits intoapache:trunkfrom
zheguang:zheguang-KAFKA-20219

Conversation

@zheguang
Copy link
Copy Markdown
Contributor

@zheguang zheguang commented Apr 2, 2026

For headers-store with caching enabled, we want to keep IQv2 disabled until we implement IQv2 across the board. Thus, we need to introduce CachingKeyValueStoreWithHeaders and just forward any query to the underlying store, instead of re-using the existing query implementation on CachingKeyValueStore.

Reviewers: Matthias J. Sax matthias@confluent.io

@zheguang zheguang marked this pull request as draft April 2, 2026 10:31
@github-actions github-actions Bot added triage PRs from the community streams tests Test fixes (including flaky tests) small Small PRs labels Apr 2, 2026
@zheguang zheguang changed the title KAFKA-20219: Show key-value store with cache accepts key query but rejects range q… KAFKA-20219: Show headers key-value store with cache accepts key query but rejects range Apr 2, 2026
@github-actions github-actions Bot removed the triage PRs from the community label Apr 3, 2026
@github-actions github-actions Bot removed the small Small PRs label Apr 4, 2026
@zheguang zheguang changed the title KAFKA-20219: Show headers key-value store with cache accepts key query but rejects range KAFKA-20219: Headers store with caching enabled should reject queries Apr 4, 2026
@zheguang zheguang marked this pull request as ready for review April 4, 2026 11:39
Copy link
Copy Markdown
Member

@mjsax mjsax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Overall LGTM.

But it seems, we are missing a test for IQv2 that verifies that we indeed get an error-result back for kv-header-store case with caching enanbled?

final File dir = TestUtils.tempDirectory();
final Properties props = StreamsTestUtils.getStreamsConfig();
final InternalMockProcessorContext<String, String> context = new InternalMockProcessorContext<>(
final InternalMockProcessorContext<String, String> context = spy(new InternalMockProcessorContext<>(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be easier to just pass in the ThreadCache into the constructor of InternalMockProcessorContext instead of using a spy?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Done. Thanks for the suggestion! :)

@mjsax mjsax added kip Requires or implements a KIP ci-approved labels Apr 6, 2026
zheguang and others added 2 commits April 7, 2026 08:17
Co-authored-by: Matthias J. Sax <mjsax@apache.org>
Comment on lines +561 to +564
@ParameterizedTest
@ValueSource(booleans = {true, false})
public void shouldReturnUnknownQueryTypeForKeyQueryOnHeadersStore(final boolean cachingEnabled) {
final TimestampedKeyValueStoreWithHeaders<String, String> store = headersStoreMaybeWithCache(cachingEnabled);
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing a test for IQv2 that verifies that we indeed get an error-result back for kv-header-store case with caching enanbled

This is covered by this test when cachingEnabled = true... right?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. Sounds correct.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, maybe you meant IQv2StoreIntegrationTest... let me look into adding an integration test there

@zheguang
Copy link
Copy Markdown
Contributor Author

zheguang commented Apr 7, 2026

Hm... the test failure IQv2StoreIntegrationTest.verifyStore is due to mismatch between caching wrapper and the db supplier: KeyValueStoreMaterializer builds with headers by default but with a supplier of RocksDBStore, prior to
e9eb75f.

@mjsax
Copy link
Copy Markdown
Member

mjsax commented Apr 8, 2026

Hm... the test failure IQv2StoreIntegrationTest.verifyStore is due to mismatch between caching wrapper and the db supplier: KeyValueStoreMaterializer builds with headers by default but with a supplier of RocksDBStore, prior to
e9eb75f.

Not sure what you exactly mean by this? Jenkins is green. Did also re-run IQv2StoreIntegrationTest on trunk plus this branch rebased on trunk, and both passed.

@mjsax mjsax merged commit 579a4e0 into apache:trunk Apr 8, 2026
23 checks passed
mjsax added a commit that referenced this pull request Apr 8, 2026
…#21941)

For headers-store with caching enabled, we want to keep IQv2 disabled
until we implement IQv2 across the board. Thus, we need to introduce
CachingKeyValueStoreWithHeaders and just forward any query to the
underlying store, instead of re-using the existing query implementation
on CachingKeyValueStore.

Reviewers: Matthias J. Sax <matthias@confluent.io>

---------

Co-authored-by: Matthias J. Sax <mjsax@apache.org>
@mjsax
Copy link
Copy Markdown
Member

mjsax commented Apr 8, 2026

Thanks for the PR. Merged to trunk and cherry-picked to 4.3 branch.

@zheguang
Copy link
Copy Markdown
Contributor Author

zheguang commented Apr 8, 2026

Hm... the test failure IQv2StoreIntegrationTest.verifyStore is due to mismatch between caching wrapper and the db supplier: KeyValueStoreMaterializer builds with headers by default but with a supplier of RocksDBStore, prior to
e9eb75f.

Not sure what you exactly mean by this? Jenkins is green. Did also re-run IQv2StoreIntegrationTest on trunk plus this branch rebased on trunk, and both passed.

Never mind, just a test failure I saw before I merged trunk. Thanks for the review!

nileshkumar3 pushed a commit to nileshkumar3/kafka that referenced this pull request Apr 15, 2026
…apache#21941)

For headers-store with caching enabled, we want to keep IQv2 disabled
until we implement IQv2 across the board. Thus, we need to introduce
CachingKeyValueStoreWithHeaders and just forward any query to the
underlying store, instead of re-using the existing query implementation
on CachingKeyValueStore.

Reviewers: Matthias J. Sax <matthias@confluent.io>

---------

Co-authored-by: Matthias J. Sax <mjsax@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-approved kip Requires or implements a KIP streams tests Test fixes (including flaky tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants