Kafka: add numPartitions and replicationFactor metadata for auto topic creation#4441
Open
Akshitguptaa wants to merge 3 commits into
Open
Kafka: add numPartitions and replicationFactor metadata for auto topic creation#4441Akshitguptaa wants to merge 3 commits into
Akshitguptaa wants to merge 3 commits into
Conversation
Signed-off-by: akshitguptaa <akshitguptaa29@gmail.com>
Signed-off-by: akshitguptaa <akshitguptaa29@gmail.com>
Signed-off-by: akshitguptaa <akshitguptaa29@gmail.com>
Akshitguptaa
force-pushed
the
feat/kafka-numpartitions
branch
from
1981428 to
cebce8e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds
numPartitionsandreplicationFactormetadata fields to the Kafka pubsub and binding components. WhennumPartitionsis set (>0), Dapr creates the topic via the Kafka admin API with the given partition count before the first publish/subscribe on that topic, instead of relying only on broker auto-create.Admin client is only created when
numPartitionsis actually set, so existing components with no config change see no behavior or connection difference.numPartitionsis not supported withauthType: awsiamand fails fast at init with a clear error instead of failing on every publish.Issue reference
Please reference the issue this PR will close: #3110
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:
Note: We expect contributors to open a corresponding documentation PR in the dapr/docs repository. As the implementer, you are the best person to document your work! Implementation PRs will not be merged until the documentation PR is opened and ready for review.