Skip to content

fix(source-mssql): Allow CDC syncs without primary keys#78292

Draft
devin-ai-integration[bot] wants to merge 3 commits into
masterfrom
devin/1779288003-fix-mssql-cdc-pkless-discovery
Draft

fix(source-mssql): Allow CDC syncs without primary keys#78292
devin-ai-integration[bot] wants to merge 3 commits into
masterfrom
devin/1779288003-fix-mssql-cdc-pkless-discovery

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot commented May 20, 2026

What

Resolves https://github.com/airbytehq/oncall/issues/12666:

Fixes source-mssql CDC discovery for SQL Server CDC-enabled tables that do not have a physical primary key. In version 4.4.7, those streams only advertised full refresh and did not expose CDC metadata fields, which blocked incremental CDC connection creation for valid all-changes capture tables.

Requested by API User via /ai-fix.

How

  • Override source-mssql Airbyte stream creation so CDC streams are decorated with CDC metadata and advertise incremental support even when no primary key is discovered.
  • Keep sourceDefinedPrimaryKey empty and isResumable=false for PK-less CDC streams so the catalog does not imply dedup/resumable primary-key semantics.
  • Add source-mssql regression coverage for CDC discovery of a PK-less stream.
  • Bump source-mssql from 4.4.7 to 4.4.8 and update the connector changelog.

Declarative-First Evaluation

source-mssql is a Kotlin/Gradle connector, not a manifest-only or low-code connector. There is no declarative YAML manifest or custom Python component involved in this fix.

Breaking Change Evaluation

Not breaking. This does not remove or rename fields, change field types, change the primary key or cursor for existing incremental streams, remove streams, change config spec fields, or change state format. It only restores incremental CDC discovery metadata for CDC-enabled tables that previously lacked it.

Review guide

  1. airbyte-integrations/connectors/source-mssql/src/main/kotlin/io/airbyte/integrations/source/mssql/MsSqlSourceOperations.kt
  2. airbyte-integrations/connectors/source-mssql/src/test/kotlin/io/airbyte/integrations/source/mssql/MsSqlSourceOperationsTest.kt
  3. airbyte-integrations/connectors/source-mssql/metadata.yaml
  4. docs/integrations/sources/mssql.md

User Impact

Users can configure CDC incremental syncs for SQL Server tables that are CDC-enabled without a physical primary key. Those streams expose the CDC cursor and CDC metadata fields while remaining append-only/no-dedup from a primary-key perspective.

Test plan

  • ./gradlew :airbyte-integrations:connectors:source-mssql:test --tests io.airbyte.integrations.source.mssql.MsSqlSourceOperationsTest
  • ./gradlew :airbyte-cdk:bulk:toolkits:bulk-cdk-toolkit-extract-jdbc:test --tests io.airbyte.cdk.discover.JdbcAirbyteStreamFactoryTest
  • ./gradlew :airbyte-integrations:connectors:source-mssql:assemble
  • Built local airbyte/source-mssql:dev image with :airbyte-integrations:connectors:source-mssql:dockerBuildImage after generating Docker build args.
  • Local CDC discover repro against SQL Server 2022: airbyte/source-mssql:4.4.7 discovered the PK-less CDC table with only full_refresh and no CDC metadata; local airbyte/source-mssql:dev discovered the same table with full_refresh + incremental, _ab_cdc_cursor, CDC metadata fields, and empty primary key.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

Devin session

Important

Active progressive rollout warning for source-mssql.

  • (Click to Approve:) Bypass the active progressive rollout warning for source-mssql in the PR comment here.

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link
Copy Markdown
Contributor

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • 🛠️ Quick Fixes
    • /format-fix - Fixes most formatting issues.
    • /bump-version - Bumps connector versions, scraping changelog description from the PR title.
      • Bump types: patch (default), minor, major, major_rc, rc, promote.
      • The rc type is a smart default: applies minor_rc if stable, or bumps the RC number if already RC.
      • The promote type strips the RC suffix to finalize a release.
      • Example: /bump-version type=rc or /bump-version type=minor
    • /bump-progressive-rollout-version - Alias for /bump-version type=rc. Bumps with an RC suffix and enables progressive rollout.
  • ❇️ AI Testing and Review (internal link: AI-SDLC Docs):
    • /ai-prove-fix - Runs prerelease readiness checks, including testing against customer connections.
    • /ai-canary-prerelease - Rolls out prerelease to 5-10 connections for canary testing.
    • /ai-review - AI-powered PR review for connector safety and quality gates.
  • 📝 AI Documentation:
    • /ai-docs-review - AI-powered documentation review for PRs with connector changes.
    • /ai-create-docs-pr - Creates a documentation PR for connector changes, stacked on the current PR.
  • 🚀 Connector Releases:
    • /publish-connectors-prerelease - Publishes pre-release connector builds (tagged as {version}-preview.{git-sha}) for all modified connectors in the PR.
  • ☕️ JVM connectors:
    • /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
      Example: /update-connector-cdk-version connector=destination-bigquery
  • 🐍 Python connectors:
    • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
    • /poe source example lock - Alias for /poe connector source-example lock.
    • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
    • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.
  • ⚙️ Admin commands:
    • /force-merge reason="<REASON>" - Force merges the PR using admin privileges, bypassing CI checks. Requires a reason.
      Example: /force-merge reason="CI is flaky, tests pass locally"
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

Detected source-mssql Active Rollout: true

Important

Active progressive rollout warning for source-mssql.

To bypass this warning, click on the matching checkbox in the PR description. Look for the checkbox text:

(Click to Approve:) Bypass the active progressive rollout warning for source-mssql in the PR comment

  • Rollout version: 4.3.0-rc.10
  • Rollout state: initialized

Version on master Branch: 4.4.7

  • RC marker on master branch: false

PR Description Checkbox Status

  • Bypass checkbox checked: false

ℹ️ More Information

Show/hide details...

🤔 What happens if this PR is merged

Checking the checkbox will allow the PR to merge, but it does not necessarily stop the active rollout by itself. The result of the PR merging depends on what connector version is published.

Expected outcomes by type of version number change:

If connector version is not modified in this PR...

No new connector version should be released, and the active rollout should continue unchanged.

If the connector version changes from RC to non-RC (GA) version...

The merged PR may publish the GA version and make it default for eligible non-pinned actors. The existing rollout is not stopped immediately by the merge, but the rollout worker can later cancel it as superseded when finalizing.

If the connector version increments to a higher `-rc` version...

The merged PR may publish a new release candidate and replace the active RC marker. The previous incomplete rollout is canceled without unpinning, and a new rollout is created for the subsequent RC version.

🔁 How to rerun this check

To rerun the check, simply check and uncheck the box, or else modify the PR description and/or title in any way.

Alternatively, you can find the Active Progressive Rollout CI workflow and manually rerun it (although this is generally slower than the above methods).


This comment will be updated as PR and/or rollout status changes.

Workflow run

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

source-mssql Connector Test Results

 13 files   13 suites   45s ⏱️
117 tests 117 ✅ 0 💤 0 ❌
227 runs  227 ✅ 0 💤 0 ❌

Results for commit ae1f587.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

Deploy preview for airbyte-docs ready!

Project:airbyte-docs
Status: ✅  Deploy successful!
Preview URL:https://airbyte-docs-1y8mmvero-airbyte-growth.vercel.app
Latest Commit:ae1f587

Deployed with vercel-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant