Skip to content

Disable Sql*ReliabilityTests on Azure#3823

Merged
mdaigle merged 2 commits into
mainfrom
dev/mdaigle/disable-flaky-sqlconnectionreliabilitytest
Dec 3, 2025
Merged

Disable Sql*ReliabilityTests on Azure#3823
mdaigle merged 2 commits into
mainfrom
dev/mdaigle/disable-flaky-sqlconnectionreliabilitytest

Conversation

@mdaigle
Copy link
Copy Markdown
Contributor

@mdaigle mdaigle commented Dec 2, 2025

Description

These tests are flaky, primarily on ubuntu22. Disabling for azure avoids running the tests on ubuntu22, but maintains some test coverage via local server test stages.

Issues

#3821

Copilot AI review requested due to automatic review settings December 2, 2025 18:58
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Dec 2, 2025
@mdaigle mdaigle added the Area\Tests Issues that are targeted to tests or test projects label Dec 2, 2025
@mdaigle mdaigle marked this pull request as ready for review December 2, 2025 18:58
@mdaigle mdaigle requested a review from a team as a code owner December 2, 2025 18:58
paulmedynski
paulmedynski previously approved these changes Dec 2, 2025
@paulmedynski paulmedynski added this to the 7.0.0-preview3 milestone Dec 2, 2025
@paulmedynski paulmedynski moved this from To triage to In review in SqlClient Board Dec 2, 2025
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses flaky test issues on Azure environments (primarily ubuntu22) by restricting SqlConnectionReliabilityTest test methods to run only on non-Azure SQL Server instances. The tests rely on specific error codes (like error 4060) that are not reliably returned when using Azure SQL or AAD authentication. This approach maintains test coverage via local server test stages while avoiding flaky failures on Azure.

  • Added IsNotAzureServer conditional attribute to six test methods
  • Added explanatory comments referencing issue #3821
  • One test (CreateDatabaseWhileTryingToConnect) already had the restriction in place

benrr101
benrr101 previously approved these changes Dec 2, 2025
Copy link
Copy Markdown
Contributor

@benrr101 benrr101 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤷‍♂️

@paulmedynski paulmedynski self-assigned this Dec 2, 2025
@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 2, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.76%. Comparing base (ac757fa) to head (9e1d2bc).
⚠️ Report is 21 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (ac757fa) and HEAD (9e1d2bc). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (ac757fa) HEAD (9e1d2bc)
addons 1 0
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3823      +/-   ##
==========================================
- Coverage   76.47%   69.76%   -6.72%     
==========================================
  Files         273      268       -5     
  Lines       44180    66432   +22252     
==========================================
+ Hits        33787    46346   +12559     
- Misses      10393    20086    +9693     
Flag Coverage Δ
addons ?
netcore 69.76% <ø> (-6.74%) ⬇️
netfx 69.15% <ø> (-7.16%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mdaigle mdaigle closed this Dec 2, 2025
@github-project-automation github-project-automation Bot moved this from In review to Done in SqlClient Board Dec 2, 2025
@mdaigle mdaigle reopened this Dec 2, 2025
@github-project-automation github-project-automation Bot moved this from Done to Backlog in SqlClient Board Dec 2, 2025
cheenamalhotra
cheenamalhotra previously approved these changes Dec 2, 2025
@cheenamalhotra cheenamalhotra moved this from Backlog to In review in SqlClient Board Dec 2, 2025
@cheenamalhotra
Copy link
Copy Markdown
Member

Looks like the SqlCommandReliabilityTest.RetryExecuteAsyncCancel test is still running in Ubuntu22_Azure_Sql configuration where it failed in both pipelines.

@mdaigle mdaigle changed the title Disable SqlConnectionReliabilityTests on Azure Disable Sql*ReliabilityTests on Azure Dec 3, 2025
@mdaigle mdaigle requested a review from benrr101 December 3, 2025 21:32
@mdaigle mdaigle requested a review from paulmedynski December 3, 2025 21:32
@mdaigle mdaigle merged commit 2b4ab13 into main Dec 3, 2025
250 checks passed
@mdaigle mdaigle deleted the dev/mdaigle/disable-flaky-sqlconnectionreliabilitytest branch December 3, 2025 22:40
@github-project-automation github-project-automation Bot moved this from In review to Done in SqlClient Board Dec 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area\Tests Issues that are targeted to tests or test projects

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants