feat: Implement stateful retry and resumable stream logic for TopicMessageQuery#2171
Conversation
Codecov Report❌ Patch coverage is @@ Coverage Diff @@
## main #2171 +/- ##
==========================================
+ Coverage 93.99% 95.04% +1.04%
==========================================
Files 163 163
Lines 10445 10514 +69
==========================================
+ Hits 9818 9993 +175
+ Misses 627 521 -106 🚀 New features to boost your workflow:
|
Up to standards ✅🟢 Issues
|
|
Hi @manishdait, This pull request has had no commit activity for 10 days. Are you still working on it?
If you're no longer working on this, please comment Reach out on discord or join our office hours if you need assistance. From the Python SDK Team |
|
Hi @manishdait, this is CronInactivityBot 👋 This pull request has had no new commits for 21 days, so I'm closing it and unassigning you from the linked issue to keep the backlog healthy. If you're no longer interested, no action is needed. Tip: You can comment If you'd like to continue working on this later, feel free to comment |
83d14a9 to
631d51e
Compare
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughTopicMessageQuery subscription now tracks state across retries, resumes from the last received message, handles chunked responses, applies gRPC-aware retry decisions with exponential backoff, and coordinates thread-safe call cancellation via SubscriptionHandle. ChangesTopic Message Query Subscription Reliability
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. 📋 Issue PlannerLet us write the prompt for your AI agent so you can ship faster (with fewer bugs). View plan for ticket: ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 6
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: ea309a58-41ae-4f9a-9ef9-8bb283f770f5
📒 Files selected for processing (4)
src/hiero_sdk_python/query/topic_message_query.pysrc/hiero_sdk_python/utils/subscription_handle.pytests/unit/subscription_handle_test.pytests/unit/topic_message_query_test.py
631d51e to
43f9b2c
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 1454aa79-c860-4bc3-8f79-1b3fde0bbdc6
📒 Files selected for processing (4)
src/hiero_sdk_python/query/topic_message_query.pysrc/hiero_sdk_python/utils/subscription_handle.pytests/unit/subscription_handle_test.pytests/unit/topic_message_query_test.py
43f9b2c to
45c80fb
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (4)
src/hiero_sdk_python/query/topic_message_query.py (1)
61-62:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winType annotation mismatch:
_error_handlerdeclared asCallable[[], None]but used withExceptionargument.Line 62 declares
_error_handlerasCallable[[], None], but:
set_error_handler(line 85) expectsCallable[[Exception], None]_on_error(line 118) takeserr: Exception- The handler is invoked with
self._error_handler(e)at line 243This inconsistency means type checkers will report errors, and users following the type hint at line 62 would provide incompatible handlers.
🐛 Proposed fix
self._completion_handler: Callable[[], None] | None = self._on_complete - self._error_handler: Callable[[], None] | None = self._on_error + self._error_handler: Callable[[Exception], None] | None = self._on_errortests/unit/topic_message_query_test.py (3)
152-154:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winChunk ordering is not verified — test would pass even if chunks are assembled in wrong order.
Using
inonly checks presence, not position. If the implementation incorrectly orders chunks, this test won't detect it.🐛 Proposed fix to verify correct ordering
assert len(received_messages) == 1 - assert b"chunk-1" in received_messages[0].contents - assert b"chunk-2" in received_messages[0].contents + # Verify chunks are assembled in correct order + contents = received_messages[0].contents + chunk1_pos = contents.find(b"chunk-1") + chunk2_pos = contents.find(b"chunk-2") + assert chunk1_pos != -1, "chunk-1 not found in assembled message" + assert chunk2_pos != -1, "chunk-2 not found in assembled message" + assert chunk1_pos < chunk2_pos, "Chunks assembled in wrong order: chunk-1 should come before chunk-2"As per coding guidelines, tests should catch regressions in chunk ordering behavior.
167-178:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winMissing assertion that
on_erroris invoked after retries exhaust.When max_attempts are exhausted with retryable errors,
on_errorshould be called. The test creates anon_error=MagicMock()but never verifies it was invoked, so regressions in terminal error handling would go undetected.🐛 Proposed fix
def test_retry_logic_on_retryable_error(mock_client, error): """Test that the query retries on retryable errors but stops after max_attempts.""" query = TopicMessageQuery(topic_id="0.0.123").set_max_attempts(2).set_max_backoff(0.5) mock_client.mirror_stub.subscribeTopic.side_effect = [error, error] - handle = query.subscribe(mock_client, on_message=MagicMock(), on_error=MagicMock()) + on_error = MagicMock() + handle = query.subscribe(mock_client, on_message=MagicMock(), on_error=on_error) handle._thread.join(timeout=2.0) + assert not handle._thread.is_alive(), "Thread should have terminated after retries exhausted" assert mock_client.mirror_stub.subscribeTopic.call_count == 2 + on_error.assert_called_once_with(error)As per coding guidelines, "Tests must provide useful error messages when they fail for future debugging."
220-224:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winTiming-dependent sleep can cause flaky tests in CI.
time.sleep(0.2)followed byassert handle._thread.is_alive()is scheduler-dependent. In slow CI environments, the thread may not have started processing yet, or conversely, the sleep may be insufficient. Use an event to synchronize on first message receipt before cancellation.🐛 Proposed fix using event synchronization
+import threading + def test_subscription_cancellation(mock_client): """Test that cancelling a handle stops the subscription thread.""" query = TopicMessageQuery(topic_id="0.0.123") def infinite_stream(): while True: yield mirror_proto.ConsensusTopicResponse(message=b"ping") time.sleep(0.1) mock_call = MagicMock() mock_call.__iter__.return_value = infinite_stream() mock_client.mirror_stub.subscribeTopic.return_value = mock_call - on_message = MagicMock() + first_message_seen = threading.Event() + on_message = MagicMock(side_effect=lambda _: first_message_seen.set()) handle = query.subscribe(mock_client, on_message=on_message) - time.sleep(0.2) - assert handle._thread.is_alive() + assert first_message_seen.wait(timeout=2.0), "Expected at least one streamed message before cancellation" + assert handle._thread.is_alive(), "Subscription thread should be alive before cancel" handle.cancel()As per coding guidelines, "No timing-dependent or unseeded random assertions."
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: cc2d52be-f082-42a4-b517-4b8426de7c85
📒 Files selected for processing (4)
src/hiero_sdk_python/query/topic_message_query.pysrc/hiero_sdk_python/utils/subscription_handle.pytests/unit/subscription_handle_test.pytests/unit/topic_message_query_test.py
|
Hi, this is WorkflowBot.
|
45c80fb to
bb49b14
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (4)
tests/unit/topic_message_query_test.py (4)
383-408:⚠️ Potential issue | 🟠 Major | ⚡ Quick winReplace timing-dependent sleep with event-based synchronization.
The
time.sleep(0.2)at line 400 followed byis_alive()assertion at line 401 is scheduler-dependent and can flap in CI under high load or slow schedulers. Use an event to deterministically wait for the first message delivery before asserting thread liveness and performing cancellation.⏱️ Proposed fix using threading.Event
+import threading import time from datetime import datetime, timezone @@ def test_subscription_cancellation(mock_client): """Test that cancelling a handle stops the subscription thread.""" query = TopicMessageQuery(topic_id="0.0.123") def infinite_stream(): while True: yield mirror_proto.ConsensusTopicResponse(message=b"ping") time.sleep(0.1) mock_call = MagicMock() mock_call.__iter__.return_value = infinite_stream() mock_client.mirror_stub.subscribeTopic.return_value = mock_call - on_message = MagicMock() + first_message_seen = threading.Event() + on_message = MagicMock(side_effect=lambda _: first_message_seen.set()) handle = query.subscribe(mock_client, on_message=on_message) - time.sleep(0.2) - assert handle._thread.is_alive() + assert first_message_seen.wait(timeout=1.0), "Expected at least one message before cancellation" + assert handle._thread.is_alive(), "Subscription thread should be alive before cancel()" handle.cancel() handle._thread.join(timeout=1.0) assert not handle._thread.is_alive() mock_call.cancel.assert_called()As per coding guidelines, "No timing-dependent or unseeded random assertions."
54-80:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winVerify fluent interface contract: setters must return
self.The test validates internal state but doesn't assert that each setter returns
self, which is required for method chaining. If a setter is accidentally changed to returnNone, chaining would break but this test would pass.🔗 Proposed fix
def test_topic_message_query_initialization(): """Test initializing the query with various parameter types and setters.""" start = datetime(2023, 1, 1, tzinfo=timezone.utc) def mock_complete(): pass def mock_error(e): pass - query = ( - TopicMessageQuery() - .set_topic_id("0.0.123") - .set_start_time(start) - .set_limit(5) - .set_chunking_enabled(True) - .set_completion_handler(mock_complete) - .set_error_handler(mock_error) - ) + query = TopicMessageQuery() + assert query.set_topic_id("0.0.123") is query, "set_topic_id should return self" + assert query.set_start_time(start) is query, "set_start_time should return self" + assert query.set_limit(5) is query, "set_limit should return self" + assert query.set_chunking_enabled(True) is query, "set_chunking_enabled should return self" + assert query.set_completion_handler(mock_complete) is query, "set_completion_handler should return self" + assert query.set_error_handler(mock_error) is query, "set_error_handler should return self" assert query._topic_id.topicNum == 123 assert query._start_time.seconds == int(start.timestamp()) assert query._limit == 5 assert query._chunking_enabled is True assert query._completion_handler == mock_complete assert query._error_handler == mock_errorAs per coding guidelines, "Assert fluent setters return
self".
304-306:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winChunk ordering is not verified — test passes even if chunks are assembled backwards.
The test uses
inat lines 305-306, which only checks presence. If the implementation concatenateschunk-2beforechunk-1, this assertion would still pass.🔍 Proposed fix to verify correct ordering
assert len(received_messages) == 1 - assert b"chunk-1" in received_messages[0].contents - assert b"chunk-2" in received_messages[0].contents + # Verify chunks are assembled in correct order + contents = received_messages[0].contents + chunk1_pos = contents.find(b"chunk-1") + chunk2_pos = contents.find(b"chunk-2") + assert chunk1_pos != -1, "chunk-1 not found in assembled message" + assert chunk2_pos != -1, "chunk-2 not found in assembled message" + assert chunk1_pos < chunk2_pos, "Chunks assembled in wrong order: chunk-1 must precede chunk-2"As per coding guidelines, tests must catch regressions in ordering behavior.
345-355:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winVerify terminal error callback and thread termination after retries exhaust.
The test passes
on_error=MagicMock()at line 350 but never asserts it's invoked when max_attempts are exhausted. The implementation should callon_errorwith the final error before stopping. Additionally,join(timeout=2.0)at line 352 should be followed by an assertion that the thread actually terminated—if it times out, the subscription count assertion at line 354 may pass spuriously.🧪 Proposed fix
def test_retry_logic_on_retryable_error(mock_client, error): """Test that the query retries on retryable errors but stops after max_attempts.""" query = TopicMessageQuery(topic_id="0.0.123").set_max_attempts(2).set_max_backoff(0.5) mock_client.mirror_stub.subscribeTopic.side_effect = [error, error] - handle = query.subscribe(mock_client, on_message=MagicMock(), on_error=MagicMock()) + on_error = MagicMock() + handle = query.subscribe(mock_client, on_message=MagicMock(), on_error=on_error) handle._thread.join(timeout=2.0) + assert not handle._thread.is_alive(), "Subscription thread should have terminated after retries exhausted" assert mock_client.mirror_stub.subscribeTopic.call_count == 2 + on_error.assert_called_once_with(error)As per coding guidelines, "Tests must provide useful error messages when they fail for future debugging" and should verify error handler contracts.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: e10049ac-fd87-4829-a382-583a10b742f7
📒 Files selected for processing (4)
src/hiero_sdk_python/query/topic_message_query.pysrc/hiero_sdk_python/utils/subscription_handle.pytests/unit/subscription_handle_test.pytests/unit/topic_message_query_test.py
763e14e to
7b8a3bb
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (3)
src/hiero_sdk_python/query/topic_message_query.py (1)
176-177:⚠️ Potential issue | 🟠 MajorDon't let a resumed query fall back to unlimited streaming.
Proto field:
ConsensusTopicQuery.limit(#4). Issue type: Wrong default. Oncestate.countreaches the caller's cap, the retry path sendslimit = 0, but the mirror schema defines zero/unset as "return messages indefinitely", so a reconnect after the cap is reached can over-deliver instead of stopping. Stop locally oncestate.count >= self._limit, or avoid emitting a retry request withlimit=0. (github.com)
As per coding guidelines, "Compare the SDK class against the proto schema."Also applies to: 227-259
src/hiero_sdk_python/utils/subscription_handle.py (1)
20-26:⚠️ Potential issue | 🟠 MajorCancel the active call after releasing
_lock.Both
_set_call()andcancel()invokecall.cancel()inside the critical section. That makes the lock cover an external call and can stall the other thread trying to register or cancel the stream. Capture the call reference while locked, then invokecancel()once the mutex is released.Also applies to: 28-34
tests/unit/topic_message_query_test.py (1)
397-405:⚠️ Potential issue | 🟡 MinorMake this cancellation test event-driven.
time.sleep(0.2)plus an immediate liveness check is scheduler-dependent and can flap in CI. Wait for the first streamed message or forsubscribeTopic()to be entered before asserting the thread is alive and callingcancel(). As per coding guidelines, "No timing-dependent or unseeded random assertions."
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: eeaf40a6-b73f-4be7-8c19-c131fa2cfddf
📒 Files selected for processing (4)
src/hiero_sdk_python/query/topic_message_query.pysrc/hiero_sdk_python/utils/subscription_handle.pytests/unit/subscription_handle_test.pytests/unit/topic_message_query_test.py
aceppaluni
left a comment
There was a problem hiding this comment.
LGTM, just make sure to resolve the open conversations
Signed-off-by: Manish Dait <daitmanish88@gmail.com>
Signed-off-by: Manish Dait <daitmanish88@gmail.com>
Signed-off-by: Manish Dait <daitmanish88@gmail.com>
Signed-off-by: Manish Dait <daitmanish88@gmail.com>
Signed-off-by: Manish Dait <daitmanish88@gmail.com>
Signed-off-by: Manish Dait <daitmanish88@gmail.com>
Signed-off-by: Manish Dait <daitmanish88@gmail.com>
Signed-off-by: Manish Dait <daitmanish88@gmail.com>
Signed-off-by: Manish Dait <daitmanish88@gmail.com>
Signed-off-by: Manish Dait <daitmanish88@gmail.com>
6a0d38c to
79ac8d1
Compare
|
👋 Hi @manishdait! Great work completing a Advanced issue! 🎉 Thanks for your contribution! 🚀 Here are some issues you might want to explore next: 🌟 Stay connected: Happy coding! 🚀 |
Description:
This PR introduces fixes for flaky
TopicMessageQuerye2e tests by implementing retry logic and error handling.Changes Made:
SubscriptionStateto track last_message and count. on retry, the query resumes fromlast_message.consensusTimestamp + 1nsto prevent message loss or duplication._should_retry logicto identify retryable gRPC errors (e.g.,UNAVAILABLE,RESOURCE_EXHAUSTED) and specificRST_STREAMinternal errors.SubscriptionHandleto safely manage gRPC call cancellation across threads.Related issue(s):
Fixes #1796
Notes for reviewer:
Checklist