Skip to content

feat(test): add Pact contract tests covering Namespace and Table APIs#341

Closed
XuQianJin-Stars wants to merge 1 commit into
lance-format:mainfrom
XuQianJin-Stars:feature/pact-contract-test
Closed

feat(test): add Pact contract tests covering Namespace and Table APIs#341
XuQianJin-Stars wants to merge 1 commit into
lance-format:mainfrom
XuQianJin-Stars:feature/pact-contract-test

Conversation

@XuQianJin-Stars
Copy link
Copy Markdown
Contributor

@XuQianJin-Stars XuQianJin-Stars commented May 9, 2026

Summary

Introduce a consumer-driven contract testing suite for lance-namespace
based on the Pact framework pact-foundation. The suite covers both
Namespace and Table REST APIs (11 operations × success + 4xx error
scenarios) and verifies three language clients (Java / Python / Rust)
against the Spring Boot reference provider.

Problem

lance-namespace ships an OpenAPI spec plus auto-generated clients in Java,
Python and Rust, but so far there is no automated mechanism to guarantee
that:

  1. Every client actually speaks the exact wire protocol the server implements.
  2. Error responses (ErrorResponse) keep a consistent 4-field shape across
    languages and versions.
  3. Breaking changes in request/response schemas are caught before they
    reach downstream consumers (pylance, rust callers, Spark/Ray integrations, …).

Typical symptoms without contract tests:

  • A field rename on the server silently breaks a Python/Rust client at runtime.
  • 4xx error bodies drift between languages (missing error, different casing,
    optional instance, etc.), making retries / user-facing messages inconsistent.
  • Each client re-invents ad-hoc HTTP mocks, so regressions are only caught in
    integration environments.

Solution

A complete Pact-based contract test suite, plus a small contract-pack folder
that hosts the generated pact files, plus CI wiring that runs all three
consumer suites and the Spring Boot provider verification on every PR.

APIs under contract (success + 4xx for each)

Group Operations
Namespace API ListNamespaces, DescribeNamespace, CreateNamespace, DropNamespace, NamespaceExists
Table API ListTables, DescribeTable, TableExists, DropTable, RegisterTable, DeregisterTable

Consumer interaction count

Client File Interactions
Java NamespaceApiPactTest.java 22 (@Pact)
Python test_namespace_api_pact.py 10
Python test_table_api_pact.py 12
Rust namespace_api_tests.rs 10
Rust table_api_tests.rs 12
Total 66

Error body DSL

A reusable 4-field ErrorResponse shape (error, code, type, instance)
is enforced for every 4xx interaction, exposed as:

  • Java: ErrorResponseDsl.java
  • Python: error_response_dsl.py
  • Rust: inline matchers under tests/

Provider verification

PactProviderTest boots an isolated PactTestApplication (Spring Boot,
pact profile, no external deps) and replays consumer pacts via
MockMvcTestTarget + @PactFolder, with provider states served by
PactStateController using InMemoryNamespaceFixtures.

CI pipeline (.github/workflows/pact.yml)

5 jobs, triggered on push and pull_request:

  1. consumer-java — Consumer Contract Tests (Java Apache Client)
  2. consumer-python — Consumer Contract Tests (Python urllib3 Client)
  3. consumer-rust — Consumer Contract Tests (Rust reqwest Client)
  4. provider-verify — Provider Verification Tests (Spring Boot Server),
    downloads pact-files-java/python/rust artifacts and replays them
  5. can-i-deploy — Can I Deploy Check (main branch gate)

.pre-commit-config.yaml wires ci/pact_state_guard.sh so any new
@State("...") string must be whitelisted before commit.

Verification

  • ✅ Java Consumer + Provider — 22 consumer pacts generated; provider
    verification replays all interactions, 0 failures
  • ✅ Python Consumer — 22 tests pass locally (pact-python)
  • ✅ Rust Consumer — 22 tests pass locally (pact_consumer + tokio)
  • ✅ Sample pacts can be dropped under contract-pack/sample-pacts/ for
    offline provider verification without a live Pact Broker.

How to run locally

  • Java consumer (generates pacts under java/lance-namespace-pact-tests/target/pacts)
    cd java && make test-pact-consumer
  • Java provider verification (replays pacts via @PactFolder + MockMvcTestTarget)
    cd java && make test-pact-provider
  • Python consumer
    cd python && make test-pact-tests
  • Rust consumer
    cd rust && make test-pact-tests

close: #340
google doc: https://docs.google.com/document/d/1Wju0Ema10OkuDLs8tYzZ0xXqihX65k-OYabZbnc5NbI/edit?usp=sharing

@github-actions github-actions Bot added enhancement New feature or request python Python features java Java features rust Rust features labels May 9, 2026
… across Java/Python/Rust clients

Introduce a consumer-driven contract testing suite for lance-namespace
based on the Pact framework. The suite covers both Namespace and Table
REST APIs (11 operations × success + error scenarios) and verifies three
language clients (Java / Python / Rust) against the Spring Boot reference
provider.

APIs under contract (success + 4xx error for each):
  Namespace API : ListNamespaces, DescribeNamespace, CreateNamespace,
                  DropNamespace, NamespaceExists
  Table API     : ListTables, DescribeTable, TableExists, DropTable,
                  RegisterTable, DeregisterTable

Contract Pack (contract-pack/):
- README.md                          : contract pack overview
- sample-pacts/                      : drop-in folder for pre-built Pact
                                       v3/v4 JSON to enable offline
                                       provider verification without a
                                       live Pact Broker

Consumers:
- Java  (java/lance-namespace-pact-tests)
    * NamespaceApiPactTest.java : 22 interactions using PactDslJsonRootValue
                                  / PactDslJsonBody matchers
    * ErrorResponseDsl.java     : reusable 4-field ErrorResponse body DSL
- Python (python/lance_namespace_pact_tests)
    * tests/pact_tests/test_namespace_api_pact.py : 10 interactions
    * tests/pact_tests/test_table_api_pact.py     : 12 interactions
    * error_response_dsl.py / conftest.py         : shared fixtures
- Rust  (rust/lance-namespace-pact-tests)
    * tests/namespace_api_tests.rs : 10 async interactions (tokio)
    * tests/table_api_tests.rs     : 12 async interactions (tokio)

Provider (java/lance-namespace-pact-tests, provider package):
- PactProviderTest.java           : MockMvcTestTarget + @PactFolder offline verify
- PactStateController.java        : @State hook endpoints
- InMemoryNamespaceFixtures.java  : deterministic in-memory fixtures
- PactNamespaceController.java    : pact-profile Namespace REST impl
- PactTableController.java        : pact-profile Table REST impl
- PactTestApplication.java        : isolated Spring Boot test application
- PactValidationConfig.java       : request/response validation beans
- EmptyBodyFilter.java            : normalize empty-body edge cases

CI tooling (ci/):
- naming_lint.sh             : consumer/provider name whitelist validator
- openapi_pact_diff.py       : OpenAPI ↔ Pact field coverage checker
- pact_consistency_check.py  : cross-consumer matcher matrix tool
- pact_state_guard.sh        : pre-commit @State string whitelist guard

Build & workflow integration:
- .github/workflows/pact.yml   : CI pipeline running Java/Python/Rust
                                 consumer tests, provider verification,
                                 and a main-branch can-i-deploy gate
- .pre-commit-config.yaml      : wires pact_state_guard into pre-commit
- java/pom.xml + lance-namespace-pact-tests/pom.xml
                               : new pact-tests Maven module + pact-publish profile
- java/Makefile, python/Makefile, rust/Makefile
                               : pact targets for each language
- python/lance_namespace_pact_tests/pyproject.toml
                               : pact-python test package
- rust/lance-namespace-pact-tests/Cargo.toml + src/lib.rs
                               : pact_consumer-based Rust test crate
- rust/Cargo.toml + Cargo.lock : workspace member registration

Verification:
- Java   Consumer + Provider : 22 consumer pacts generated; provider
                               verification replays 45 interactions, 0 failures
- Python Consumer            : 22 tests pass locally (pact-python)
- Rust   Consumer            : 22 tests pass locally (pact_consumer + tokio)
- Sample pacts can be dropped under contract-pack/sample-pacts/ for
  offline provider verification without a live Pact Broker.
@XuQianJin-Stars XuQianJin-Stars force-pushed the feature/pact-contract-test branch from 25cabda to f039a88 Compare May 10, 2026 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request java Java features python Python features rust Rust features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Test] Introduce Pact consumer-driven contract testing for Namespace & Table APIs across Java/Python/Rust clients

1 participant