Skip to content

[SPARK-54999] Re-enable ConstraintTests by moving into IcebergTests#401

Draft
dongjoon-hyun wants to merge 1 commit into
apache:mainfrom
dongjoon-hyun:SPARK-54999
Draft

[SPARK-54999] Re-enable ConstraintTests by moving into IcebergTests#401
dongjoon-hyun wants to merge 1 commit into
apache:mainfrom
dongjoon-hyun:SPARK-54999

Conversation

@dongjoon-hyun

@dongjoon-hyun dongjoon-hyun commented May 29, 2026

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR re-enables the constraint test coverage (SPARK-51207, SPIP: Constraints in DSv2) that was disabled in SPARK-54998, by moving the four tests out of the disabled ConstraintTests suite into IcebergTests and switching them from USING ORC (DSv1) to USING ICEBERG (DSv2):

  • primary_key, foreign_key, unique, and check now create local.db.* Iceberg tables and run under the existing SPARK_ICEBERG_TEST_ENABLED guard, like the other IcebergTests.
  • The spark.version.starts(with: "4.1") guard is kept so the constraint syntax (a Spark 4.1+ feature) runs only on the Spark 4.1.2 Iceberg CI job and is skipped on the Spark 4.0.2 one.
  • Tests/SparkConnectTests/ConstraintTests.swift is removed.
  • Fix the foreign_key test to reference the parent table (tableName1, which has id) instead of the table being created.

Note that this is possible because we moved to Apache Iceberg 1.11.0.

Why are the changes needed?

ConstraintTests was disabled in SPARK-54998 because Spark 4.1.1+ started throwing on constraint syntax for DSv1 (ORC) tables after SPARK-54761, and the original plan was to revisit this with DSv2 tables once Apache Iceberg supported Spark 4.1. Iceberg now ships a Spark 4.1 runtime (iceberg-spark-runtime-4.1_2.13:1.11.0, already used by CI), so the constraint syntax can be validated against DSv2/Iceberg tables as originally intended.

Does this PR introduce any user-facing change?

No. This is a test-only change.

How was this patch tested?

Pass the CIs.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 4.8)

@dongjoon-hyun
dongjoon-hyun marked this pull request as draft May 29, 2026 20:20
@dongjoon-hyun

Copy link
Copy Markdown
Member Author

This is still a Draft.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant