Skip to content

Support consumer, producer, listener and template specific Kafka Admin properties #38830

Description

@hemantmaersk

The current implementation of KafkaProperties in Spring Boot allows for the configuration of different producer and consumer instances, enabling the reading from different cluster and writing to altogether different Kafka cluster using various user IDs and passwords. However, a limitation exists wherein only one admin configuration can be set for both producers and consumers. This constraint becomes problematic in various scenarios requiring cluster id. Observability, is one such instance.

To address this limitation, developers should be empowered to configure different Admin instances for producers and consumers independently. This modification becomes crucial for scenarios where distinct cluster IDs are required for enhanced observability.

To summarise,The current functionality do empower developers is to read from different cluster by configuring spring.kafka.consumer and can write to altogether different cluster using spring.kafka.producer however there is only one admin instance that can be configured for both via spring.kafka.admin

Basically it would be great if we could have something like below

@ConfigurationProperties(prefix = "spring.kafka")
public class KafkaProperties {

	/**
	 * Comma-delimited list of host:port pairs to use for establishing the initial
	 * connections to the Kafka cluster. Applies to all components unless overridden.
	 */
	private List<String> bootstrapServers = new ArrayList<>(Collections.singletonList("localhost:9092"));

	/**
	 * ID to pass to the server when making requests. Used for server-side logging.
	 */
	private String clientId;

	/**
	 * Additional properties, common to producers and consumers, used to configure the
	 * client.
	 */
	private final Map<String, String> properties = new HashMap<>();

	/**
	 * Configuration for Kafka consumers.
	 */
	private final Consumer consumer = new Consumer();

	/**
	 * Configuration for Kafka producers.
	 */
	private final Producer producer = new Producer();

	/**
	 * Configuration for Kafka Admin.
	 */
	private final Admin producerAdmin = new Admin();

	/**
	 * Configuration for Kafka Admin.
	 */
	private final Admin consumerAdmin = new Admin();
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions