[pull] master from DataDog:master#636
Merged
Merged
Conversation
* kafka_actions: harden reset-consumer-offsets action - Abort before altering offsets when the consumer group still has active members (describe_consumer_groups pre-flight check) so the Kafka NON_EMPTY_GROUP error never surfaces as a cryptic partial failure. - Inspect per-partition TopicPartition error codes in the alter_consumer_group_offsets result; previously only group-level futures were checked, letting silent per-partition failures through. - Add reset_to: earliest|latest support: resolves the log-start or high-watermark via list_offsets at runtime, then commits the concrete offset. alter_consumer_group_offsets does not accept symbolic offsets. - Validate that partition and offset are non-negative; reject entries that set both offset and reset_to or neither. - Clarify in spec.yaml that the cluster field must be the Kafka-internal UUID (from AdminClient.list_topics().cluster_id), not a human-readable name; update examples and error messages accordingly. - Log the full offset list in the pre-execution warning for audit trail. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * kafka_actions: add changelog entry for PR #24165 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * kafka_actions: fix ruff formatting Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * kafka_actions: fix ruff import sort in remaining test files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix update_consumer_group_offsets in kafka_actions The confluent-kafka AdminClient.alter_consumer_group_offsets() API takes a single list of ConsumerGroupTopicPartitions, not (group, partitions) as two positional arguments, which raised "takes 2 positional arguments but 3 were given". ConsumerGroupTopicPartitions is also exported from confluent_kafka, not confluent_kafka.admin. Additionally, future.result() returns a single ConsumerGroupTopicPartitions object rather than a list of partitions, so iterate its topic_partitions to collect per-partition errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * kafka_actions: add timestamp-based offset reset and sentinel offsets Replace reset_to: earliest/latest with offset: -2/-1 sentinel values and add a timestamp field that resolves all partitions of a topic to the first offset at or after the given millisecond timestamp, with automatic partition discovery when partition is omitted. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * kafka_actions: remove unrelated cluster-UUID doc changes from spec.yaml Reverts stray documentation claiming the 'cluster' field for update_consumer_group_offsets must be the Kafka-internal UUID rather than a human-readable name. That claim is unrelated to this PR's hardening work and inconsistent with every other action's cluster docs/examples (prod-kafka-1). * kafka_actions: sync changelog entry with current update_consumer_group_offsets behavior * kafka_actions: restore autogenerated file headers and fix formatting Regenerated config_models were missing their license headers (a stray regeneration artifact), and kafka_client.py / test_unit.py had two small ruff formatting/import-order issues. * kafka_actions: address round-1 review feedback on offset-reset hardening Adds request timeouts and error handling around list_offsets/describe_consumer_groups calls, de-dups overlapping offset targets, guards against both offset and timestamp being set, and splits update_consumer_group_offsets into smaller resolver methods. Extracts a shared partition-discovery helper and de-dups the config.py partition validation. Adds unit test coverage for the client-internal sentinel/timestamp/ per-partition-error/active-member logic and parameterizes the offsets validation tests. Cleans up doc formatting in spec.yaml and splits the changelog entry into .fixed/.added to reflect the net-new sentinel/timestamp capabilities. * [kafka_actions] Address round-2 review feedback Batch the timestamp-fallback offset lookup instead of issuing one list_offsets RPC per partition, pass the admin client into the resolver helpers consistently, wrap the describe_consumer_groups result with the same log-and-reraise pattern as the other admin calls, and tighten a couple of doc/type-hint/test gaps flagged by the second review pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * [kafka_actions] Fix import grouping in test files (ruff I001) CI's ruff check flagged missing blank-line separation between third-party and first-party import groups across the test suite. * [kafka_actions] Address round-3 review nits Type-hint the admin param on the offset resolver helpers, and key the fallback test's list_offsets stub on OffsetSpec type instead of request length so it stays correct if a second fallback partition is added. * [kafka_actions] Fix import grouping (ruff I001) Restore the blank line between third-party and first-party import groups after adding the OffsetSpec import. * [kafka_actions] Trim redundant offsets doc and merge changelog entries Remove the duplicated offset/timestamp usage block from the action description (already covered in the per-field docs), and consolidate the two changelog fragments into a single added entry. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )