Skip to content

PYTHON-5879 Skip coverage on macOS/Win64 encryption tests #2882

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aclark4life merged 7 commits into from
Jun 18, 2026
Merged

PYTHON-5879 Skip coverage on macOS/Win64 encryption tests #2882

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
611ded6
PYTHON-5879 Skip coverage on macOS encryption tests
aclark4life Jun 17, 2026
a3566f3
Address PR feedback: clarify comment, PR title updated to mention Win64
aclark4life Jun 17, 2026
e91a765
PYTHON-5879 Generate non-coverage companion tasks for macOS/Win64 enc…
aclark4life Jun 17, 2026
06af4e4
PYTHON-5879 Use distinct tag for non-coverage companion tasks to avoi…
aclark4life Jun 17, 2026
32f7bb5
PYTHON-5879 Only run sharded_cluster non-standard tasks on PRs
aclark4life Jun 18, 2026
9ee6d4a
Merge branch 'master' into PYTHON-5879
aclark4life Jun 18, 2026
aa49c31
PYTHON-5879 Fix inaccurate comment about PyPy toolchain support
aclark4life Jun 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions .evergreen/generated_configs/tasks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4827,6 +4827,30 @@ tasks:
- noauth
- free-threaded
- pr
- cov
- name: test-non-standard-latest-python3.14t-noauth-ssl-replica-set
commands:
- func: run server
vars:
AUTH: noauth
SSL: ssl
TOPOLOGY: replica_set
VERSION: latest
- func: run tests
vars:
AUTH: noauth
SSL: ssl
TOPOLOGY: replica_set
VERSION: latest
TOOLCHAIN_VERSION: 3.14t
tags:
- test-non-standard-no-cov
- server-latest
- python-3.14t
- replica_set-noauth-ssl
- noauth
- free-threaded
- pr
- name: test-non-standard-latest-pypy3.11-noauth-ssl-replica-set
commands:
- func: run server
Expand Down Expand Up @@ -4873,6 +4897,29 @@ tasks:
- sharded_cluster-auth-ssl
- auth
- pr
- cov
- name: test-non-standard-latest-python3.14-auth-ssl-sharded-cluster
commands:
- func: run server
vars:
AUTH: auth
SSL: ssl
TOPOLOGY: sharded_cluster
VERSION: latest
- func: run tests
vars:
AUTH: auth
SSL: ssl
TOPOLOGY: sharded_cluster
VERSION: latest
TOOLCHAIN_VERSION: "3.14"
tags:
- test-non-standard-no-cov
- server-latest
- python-3.14
- sharded_cluster-auth-ssl
- auth
- pr
- name: test-non-standard-latest-python3.13-noauth-nossl-standalone-cov
commands:
- func: run server
Expand All @@ -4897,6 +4944,29 @@ tasks:
- standalone-noauth-nossl
- noauth
- pr
- cov
- name: test-non-standard-latest-python3.13-noauth-nossl-standalone
commands:
- func: run server
vars:
AUTH: noauth
SSL: nossl
TOPOLOGY: standalone
VERSION: latest
- func: run tests
vars:
AUTH: noauth
SSL: nossl
TOPOLOGY: standalone
VERSION: latest
TOOLCHAIN_VERSION: "3.13"
tags:
- test-non-standard-no-cov
- server-latest
- python-3.13
- standalone-noauth-nossl
- noauth
- pr
- name: test-non-standard-rapid-python3.11-noauth-ssl-replica-set
commands:
- func: run server
Expand Down
12 changes: 8 additions & 4 deletions .evergreen/generated_configs/variants.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,8 @@ buildvariants:
tags: [encryption_tag]
- name: encryption-macos
tasks:
- name: .test-non-standard !.pypy
- name: .test-non-standard !.pypy !.cov
- name: .test-non-standard-no-cov !.pypy
display_name: Encryption macOS
run_on:
- macos-14
Expand All @@ -203,7 +204,8 @@ buildvariants:
tags: [encryption_tag]
- name: encryption-win64
tasks:
- name: .test-non-standard !.pypy
- name: .test-non-standard !.pypy !.cov
- name: .test-non-standard-no-cov !.pypy
display_name: Encryption Win64
run_on:
- windows-2022-latest-small
Expand All @@ -224,7 +226,8 @@ buildvariants:
tags: [encryption_tag]
- name: encryption-crypt_shared-macos
tasks:
- name: .test-non-standard !.pypy
- name: .test-non-standard !.pypy !.cov
- name: .test-non-standard-no-cov !.pypy
display_name: Encryption crypt_shared macOS
run_on:
- macos-14
Expand All @@ -235,7 +238,8 @@ buildvariants:
tags: [encryption_tag]
- name: encryption-crypt_shared-win64
tasks:
- name: .test-non-standard !.pypy
- name: .test-non-standard !.pypy !.cov
- name: .test-non-standard-no-cov !.pypy
display_name: Encryption crypt_shared Win64
run_on:
- windows-2022-latest-small
Expand Down
27 changes: 26 additions & 1 deletion .evergreen/scripts/generate_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,11 @@ def get_encryption_expansions(encryption):
display_name = get_variant_name(encryption, host, **expansions)
tasks = [".test-non-standard"]
if host != "rhel8":
tasks = [".test-non-standard !.pypy"]
# Exclude PyPy (no Evergreen toolchain on macOS/win64) and coverage tasks

@blink1073 blink1073 Jun 18, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment isn't accurate, we have PyPy in our toolchain on all hosts.

# (encryption suites exceed the 60-min timeout with coverage overhead on macOS/win64).
# Also include the non-coverage companion tasks (test-non-standard-no-cov) which
# carry the "latest" server tasks without COVERAGE=1.
tasks = [".test-non-standard !.pypy !.cov", ".test-non-standard-no-cov !.pypy"]
variant = create_variant(
tasks,
display_name,
Expand Down Expand Up @@ -670,12 +674,33 @@ def create_test_non_standard_tasks():
expansions["TEST_MIN_DEPS"] = "1"
elif pr:
expansions["COVERAGE"] = "1"
tags.append("cov")
name = get_task_name("test-non-standard", python=python, **expansions)
server_func = FunctionCall(func="run server", vars=expansions)
test_vars = expansions.copy()
test_vars["TOOLCHAIN_VERSION"] = python
test_func = FunctionCall(func="run tests", vars=test_vars)
tasks.append(EvgTask(name=name, tags=tags, commands=[server_func, test_func]))
# For each coverage task, also emit a non-coverage companion so that
# macOS/Win64 encryption variants (which filter out .cov due to timeout
# constraints) still have a "latest" task to activate in patch builds.
if pr and "cov" in tags:
nc_expansions = {k: v for k, v in expansions.items() if k != "COVERAGE"}
Comment thread
aclark4life marked this conversation as resolved.
# Use a distinct primary tag so companions are not selected by existing
# ".test-non-standard" selectors (e.g. load-balancer, PyOpenSSL variants).
nc_tags = [
"test-non-standard-no-cov" if t == "test-non-standard" else t
for t in tags
if t != "cov"
]
nc_name = get_task_name("test-non-standard", python=python, **nc_expansions)
nc_server_func = FunctionCall(func="run server", vars=nc_expansions)
nc_test_vars = nc_expansions.copy()
nc_test_vars["TOOLCHAIN_VERSION"] = python
nc_test_func = FunctionCall(func="run tests", vars=nc_test_vars)
tasks.append(
EvgTask(name=nc_name, tags=nc_tags, commands=[nc_server_func, nc_test_func])
)
return tasks


Expand Down
Loading