feat - kafka support#243

Open

msmygit wants to merge 56 commits into

feat/kafka_support

Commits on Jun 12, 2026

Initial plan for adding Kafka support
Madhavan
committed
Phase 1 - implemented
Madhavan
committed
Phase 2: Core Abstraction Layer
Madhavan
committed
Phase 3: Pulsar Implementation
Madhavan
committed
Phase 4: Kafka Implementation - week 1
Madhavan
committed
Phase 2: Core Abstraction Layer - Week 2-3 Deliverables. deleted ph 3 & 4
Madhavan
committed
Phase 3 - week1 - Pulsar Impl
Madhavan
committed
Phase 3 - weeks2&3 - Pulsar Impl
Madhavan
committed
Phase 4 - Kafka implementation initial
Madhavan
committed
Phase 4 - kafka impl basic docs updates
Madhavan
committed
Phase 5: Kafka Integration Tests & CI Workflows - Implementation
Madhavan
committed
Remove Made with Bob comment on files
Madhavan
committed
Phase 1: CI Stabilization - Disable Kafka tests and add resource limits
Madhavan
committed
ci failures
Madhavan
committed
backfill-ci failure fix attempt
Madhavan
committed
attempt to fix backfill-ci
Madhavan
committed
temp comment jdk17 from matrix
Madhavan
committed
attempt to remove kafka premature
Madhavan
committed
backfill cli and ci fix attempt
Madhavan
committed
bob created mess to fix backfill-ci job attempt 5
Madhavan
committed
phase 3 impl + refactor
Madhavan
committed
interim
Madhavan
committed
interim fixes;slf4j reverts
Madhavan
committed
interim phase3 fix attempts
Madhavan
committed
interim fixes backfill-ci failures
Madhavan
committed
interim backfill CI fixes
Madhavan
committed
s
Madhavan
committed
s
Madhavan
committed
interim backfill-ci fixes
Madhavan
committed
interim ci fixes
Madhavan
committed
Fix ClassCastException by reverting NativeSchemaWrapper and CassandraSource to original implementation
Madhavan
committed
Update BOB_CONTEXT_SUMMARY with ClassCastException fix details
Madhavan
committed
backfill-ci fail fix attempt
Madhavan
committed
Fix connector test failures: Revert to direct Pulsar API usage
Madhavan
committed
ci failures fixes
Madhavan
committed
CI Failure fix attempt
Madhavan
committed
fix(kafka_support): correct SSL keystore/truststore mapping and restore infinite producer timeout
Madhavan
committed
feat(kafka): complete & validate agent-side dual-provider (Pulsar+Kafka) CDC
Madhavan
committed
feat(kafka): add Kafka Connect source connector (events -> Cassandra -> data)
Madhavan
committed
fix(backfill): repair messaging-abstraction backfill (SPI discovery + sender construction)
Madhavan
committed
fix(connector): make assertMapsEqual tolerant of shaded/non-shaded Avro Utf8
Madhavan
committed
fix(backfill): forward Docker API version to the e2eTest task and backfill-ci
Madhavan
committed
fix(backfill): discover dsbulk codec providers in the NAR CLI-extension path
Madhavan
committed
ci: expand Pulsar/Kafka test matrices and restore 360m timeout
Madhavan
committed
fix(agent): reject invalid messagingProvider and validate provider config
Madhavan
committed
docs(kafka): publish Kafka/Confluent user docs and regenerate agent params
Madhavan
committed
feat(backfill): support Kafka as a backfill destination + e2e + CI
Madhavan
committed
docs(backfill): document Kafka as a backfill destination
Madhavan
committed
fix(backfill-ci): pass cassandraFamily to the Kafka e2e so the right cassandra.yaml is mounted
Madhavan
committed
fix(backfill-ci): wait on CQL readiness for the no-agent Cassandra node
Madhavan
committed
fix(backfill-ci): wait on the CQL port, not a log line, for the no-agent node
Madhavan
committed
fix(backfill-ci): revert to CQL-log wait and make the c4 logback log to stdout
Madhavan
committed
fix(connector): make testSchema tolerant of non-shaded Avro and JSON whole-number doubles
Madhavan
committed
fix(connector): type assertGenericMap keys as Object for shaded/non-shaded Avro
Madhavan
committed
fix(connector): normalize non-shaded Avro collections nested inside a shaded UDT
Madhavan
committed
chore(ci): Bump HardNorth/github-version-generate@v1.4.1 to use Nodejs 24+ in CI jobs
Madhavan
committed