feat(kafka): migrate to apache/kafka image with KRaft mode#233
Merged
Conversation
Replace the confluentinc/cp-kafka image with the official apache/kafka image which runs in KRaft mode by default, eliminating the need for Zookeeper. Key changes: - Use apache/kafka:3.9.0 as the default image - Configure KRaft mode with combined broker/controller roles - Use fixed host port (random 29000-29999) for advertised listeners since apache/kafka requires knowing the advertised address at startup - Add with_topics/2 to automatically create topics on startup - Add bootstrap_servers/1 and port/1 helper functions - Simplify configuration by removing Zookeeper-related options - Update tests to match new API Breaking changes: - Removed consensus_strategy option (only KRaft is supported) - Removed zookeeper_port, zookeeper_host options - Removed broker_port, broker_id options (use kafka_port, node_id) - Removed topic_partitions option (use with_topics/2 instead) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Member
Author
|
@Argonus ☝️ 🙈 (used ralph with claude code and it worked without interruption for more than half an hour) |
Member
Author
|
replaces #232 |
Member
Author
|
i think ill just merge this, so we have a new upgraded version of kafka for version 2.X.X which is yet to be released. |
Member
Author
|
but do add your comments and/or suggestions/fears with this change @Argonus |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
confluentinc/cp-kafkaimage with the officialapache/kafkaimage which runs in KRaft mode by defaultwith_topics/2Changes
New Features
apache/kafka:3.9.0as the default Docker imagewith_topics/2function to automatically create topics on container startupbootstrap_servers/1helper to get the connection stringport/1helper to get the mapped portBreaking Changes
consensus_strategyoption (only KRaft is supported now)zookeeper_port,zookeeper_hostoptionsbroker_port,broker_idoptions (usekafka_port,node_idinstead)topic_partitionsoption (usewith_topics/2for topic creation)Implementation Notes
apache/kafkaimage requires knowing the advertised listener address at startup time, before dynamic port mapping is knownTest plan
🤖 Generated with Claude Code