You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: enhance Kerberos support and streamline configuration
- Improved Kerberos integration for Kafka and other services, reducing redundancy across components.
- Removed Schema Registry-specific Kerberos configuration, leveraging Kafka Kerberos setup instead.
- Added programmatic support for configuring Kerberos principals, keytabs, and krb5.conf.
- Introduced dynamic Kafka port mapping and improved advertised listener handling for host-side clients.
- Updated Spark example and extensions to support Kerberos-secured Kafka communication.
- Enhanced documentation with revised Kerberos setup and usage instructions.
Copy file name to clipboardExpand all lines: README.md
+4-3Lines changed: 4 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -155,7 +155,7 @@ class MyIntegrationTest {
155
155
}
156
156
```
157
157
158
-
Kerberos can be enabled per service. Hadoop/HDFS, Hive Metastore, Kafka, Schema Registry, and Kafka UI expose dedicated Kerberos switches; set`kerberos = true` to start the shared KDC and then enable auth for the services that should use it.
158
+
Kerberos can be enabled per service. Hadoop/HDFS, Hive Metastore, Kafka, and Kafka UI expose dedicated Kerberos switches. Set`kerberos = true` to start the shared KDC and then enable auth for the services that should use it. Schema Registry can still be enabled with Kafka Kerberos; it uses Kafka's internal plaintext listener while host clients keep using Kerberos.
159
159
160
160
```kotlin
161
161
@BigDataTest(
@@ -166,12 +166,13 @@ Kerberos can be enabled per service. Hadoop/HDFS, Hive Metastore, Kafka, Schema
166
166
hiveMetastoreKerberos =true,
167
167
kafka =true,
168
168
kafkaKerberos =true,
169
-
schemaRegistry =true,
170
-
schemaRegistryKerberos =true,
169
+
kafkaPort =19092,
171
170
)
172
171
classMyKerberosIntegrationTest
173
172
```
174
173
174
+
For host-side Kafka Kerberos clients, configure a fixed `kafkaPort` so the broker can advertise the same host port that clients use.
175
+
175
176
JUnit tests can also route container logs to the main test process console or to files:
@@ -349,12 +346,12 @@ Kerberos can be enabled globally and per service. Set `kerberos = true` to start
349
346
hiveMetastoreKerberos = true,
350
347
kafka = true,
351
348
kafkaKerberos = true,
352
-
schemaRegistry = true,
353
-
schemaRegistryKerberos = true,
354
349
)
355
350
class MyKerberosIntegrationTest
356
351
----
357
352
353
+
For host-side Kafka Kerberos clients, prefer an explicit fixed `kafkaPort`. Kafka must advertise the same host and port that the client uses during the SASL/GSSAPI exchange; with a random host port the framework cannot know that advertised port before the broker starts. When Schema Registry is enabled with Kafka Kerberos, it uses Kafka's internal plaintext listener while host clients keep using the Kerberos listener.
354
+
358
355
Default principals are defined in `BigDataTestKitOptions`:
359
356
360
357
[cols="1,2",options="header"]
@@ -371,15 +368,32 @@ Default principals are defined in `BigDataTestKitOptions`:
The `KerberosMaterialExtension` publishes reusable client and service material to extension output:
378
+
379
+
[source,toml]
380
+
----
381
+
[kerberosMaterial]
382
+
enabled = true
383
+
----
384
+
385
+
Common output keys are:
386
+
387
+
* `kerberos-material.realm`
388
+
* `kerberos-material.kdc`
389
+
* `kerberos-material.krb5-conf`
390
+
* `kerberos-material.client.principal`
391
+
* `kerberos-material.client.password`
392
+
* `kerberos-material.client.keytab`
393
+
* `kerberos-material.kafka.service-name`
394
+
* `kerberos-material.kafka.principal`
395
+
* `kerberos-material.kafka.keytab`
396
+
383
397
== Extensions Module
384
398
385
399
The `extensions` module is for config-driven setup that should not live in `core` or `junit5`. It depends on heavier client libraries where needed and runs lifecycle hooks against a started `BigDataTestKit`.
0 commit comments