Skip to content

[FLINK-37244] Added scylladb tests, documentation and README#40

Open
andrii-kysylevskyi wants to merge 1 commit intoapache:mainfrom
andrii-kysylevskyi:FLINK-37244-scylladb-support-tests-and-docs
Open

[FLINK-37244] Added scylladb tests, documentation and README#40
andrii-kysylevskyi wants to merge 1 commit intoapache:mainfrom
andrii-kysylevskyi:FLINK-37244-scylladb-support-tests-and-docs

Conversation

@andrii-kysylevskyi
Copy link
Copy Markdown

Summary

This PR adds comprehensive ScyllaDB support to the Flink Cassandra Connector, validating ScyllaDB as a drop-in replacement for Apache Cassandra. All connector features (sources, sinks, batch formats, exactly-once semantics) have been tested against ScyllaDB 2025.1.4 (latest open source version).

Motivation

ScyllaDB is a high-performance, drop-in replacement for Apache Cassandra that maintains full CQL protocol compatibility.

Implementation Approach

Following @echauchot's guidance to "add ITests on ScyllaDB using testContainers" without modifying production code, this PR introduces:

Test Infrastructure (4 new files)

  1. ScyllaDBTestEnvironment: Test environment using ScyllaDB containers (319 lines)
  2. ScyllaDBTestContext: Test context factory for source tests (162 lines)
  3. ScyllaDBSourceITCase: Integration tests for source operations (414 lines)
  4. ScyllaDBConnectorITCase: Integration tests for sink operations and batch formats (795 lines)

Implementation Strategy

To avoid modifying the existing Cassandra test infrastructure, the new ScyllaDB tests mirror the structure of their Cassandra counterparts. All ScyllaDB test files are standalone copies with ScyllaDB-specific configuration:

  • ScyllaDBTestEnvironment (319 lines): Duplicates CassandraTestEnvironment with ScyllaDB-specific container configuration (image: scylladb/scylla:2025.1.4, flags: --reactor-backend=epoll --smp 1, increased timeouts)
  • ScyllaDBTestContext (162 lines): Duplicates CassandraTestContext to provide type-compatible test context factory for source tests
  • ScyllaDBSourceITCase (414 lines): Duplicates CassandraSourceITCase with all 11 test methods running against ScyllaDB
  • ScyllaDBConnectorITCase (795 lines): Duplicates CassandraConnectorITCase with all 20+ sink and batch format tests running against ScyllaDB

Original PR that got stale, can be found here

As suggested, inviting @dannycranmer or @zentol for a review. Thanks a lot in advance!

@boring-cyborg
Copy link
Copy Markdown

boring-cyborg Bot commented Oct 30, 2025

Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)

@andrii-kysylevskyi
Copy link
Copy Markdown
Author

@Poorvankbhatia, I noticed you have reviewed a couple of PR's in this repository. Is there any chance you could take a look at this one when you have some time? Thank you!

@Poorvankbhatia
Copy link
Copy Markdown
Contributor

@Poorvankbhatia, I noticed you have reviewed a couple of PR's in this repository. Is there any chance you could take a look at this one when you have some time? Thank you!

Ack. I will review it.

@andrii-kysylevskyi
Copy link
Copy Markdown
Author

Hi @Poorvankbhatia - just following up on this in case it got lost.
Would you get a chance to take a look when you have time?

Thank you in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants