CLOUD-929 - Initial pytest test migration#2058
Open
jvpasinatto wants to merge 28 commits into
Open
Conversation
|
|
||
| # TODO: consider using Python for filtering instead of yq | ||
| yq_filter = f""" | ||
| del(.metadata.ownerReferences[].apiVersion) | |
Contributor
There was a problem hiding this comment.
This would be nice in a separated file perhaps
|
|
||
|
|
||
| @pytest.fixture(scope="class") | ||
| def deploy_chaos_mesh(namespace): |
Contributor
There was a problem hiding this comment.
I think we could have separate file for helm installed dependencies and use a generic function with parameters to install a helm chart since subprocess is going to be used fot it
pooknull
reviewed
Jan 23, 2026
Comment on lines
+26
to
+36
| def kubectl_bin(*args, check: bool = True, input_data: str = "") -> str: | ||
| """Execute kubectl command""" | ||
| cmd = ["kubectl"] + list(args) | ||
| logger.debug(" ".join(map(str, cmd))) | ||
| result = subprocess.run(cmd, check=check, capture_output=True, text=True, input=input_data) | ||
|
|
||
| if result.stderr: | ||
| logger.warning(f"kubectl error: {result.stderr}") | ||
| return result.stderr | ||
|
|
||
| return result.stdout |
Contributor
There was a problem hiding this comment.
I think it would be nice to add a -n <current-namespace> option and stop changing the kubectl context. This would match the approach used in kuttl tests and would let us run multiple tests in parallel on the same cluster. I don’t think it would cause any problems.
pooknull
reviewed
Jan 23, 2026
| os.environ.setdefault("DELETE_CRD_ON_START", "1") | ||
| os.environ.setdefault("SKIP_DELETE", "0") | ||
| os.environ.setdefault("SKIP_BACKUPS_TO_AWS_GCP_AZURE", "1") | ||
| os.environ.setdefault("UPDATE_COMPARE_FILES", "0") |
Contributor
There was a problem hiding this comment.
UPDATE_COMPARE_FILES isn't implemented in this PR
pooknull
reviewed
Jan 23, 2026
|
|
||
| if result.stderr: | ||
| logger.warning(f"kubectl error: {result.stderr}") | ||
| return result.stderr |
Contributor
There was a problem hiding this comment.
Suggested change
| return result.stderr |
If we have stderr and result.returncode is 0, I think we should still return stdout
eleo007
reviewed
Jan 27, 2026
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| create_infra() { | ||
| local ns="$1" | ||
|
|
||
| echo "$ns" > /tmp/pytest_current_namespace |
Contributor
There was a problem hiding this comment.
[shfmt] reported by reviewdog 🐶
Suggested change
| echo "$ns" > /tmp/pytest_current_namespace | |
| echo "$ns" >/tmp/pytest_current_namespace |
jvpasinatto
requested review from
egegunes,
gkech,
hors,
mayankshah1607,
nmarukovich and
oksana-grishchenko
as code owners
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
This PR lays the initial foundation for migrating the existing e2e-tests suite to pytest by introducing a Python test harness (fixtures + helper library), converting several tests to pytest, and updating CI (Jenkins + GitHub Actions) to run checks and publish pytest-style reports.
Changes:
- Add Python project + locked dependencies via
uv(pyproject.toml,uv.lock) and Makefile targets to manage them. - Introduce pytest infrastructure (
e2e-tests/conftest.py,e2e-tests/lib/*) and migrateinit-deploy,finalizer, andlivenesstests to pytest. - Update Jenkins to execute tests via pytest (including a wrapper for legacy bash tests) and generate/merge HTML + JUnit reports; add a GitHub Action for ruff/mypy.
Reviewed changes
Copilot reviewed 28 out of 30 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
uv.lock |
Adds a locked Python dependency set for the pytest migration. |
pyproject.toml |
Defines the Python test project, deps, and tool configs (pytest/mypy/ruff). |
Makefile |
Adds uv installation plus Python deps/format/check targets. |
Jenkinsfile |
Runs tests via pytest, generates per-test reports, merges/publishes final report. |
e2e-tests/test_pytest_wrapper.py |
Pytest wrapper entrypoint to run legacy bash tests under pytest. |
e2e-tests/README.md |
Documents uv setup and pytest usage for e2e tests. |
e2e-tests/liveness/test_liveness.py |
Migrates liveness test to pytest. |
e2e-tests/lib/utils.py |
Adds shared utilities (retry, git helpers, rich logging theme/highlighter). |
e2e-tests/lib/secrets.py |
Adds helpers for reading/applying secrets and cloud credential handling. |
e2e-tests/lib/report_generator.py |
Adds pytest-html customizations and report extras generator. |
e2e-tests/lib/operator.py |
Adds Python operator deploy / CRD finalizer handling helpers. |
e2e-tests/lib/mongo.py |
Adds MongoDB helpers for mongosh execution and JSON comparisons. |
e2e-tests/lib/kubectl.py |
Adds kubectl execution + common wait/detect helpers. |
e2e-tests/lib/k8s_collector.py |
Adds K8s resource/log collection for failure diagnostics. |
e2e-tests/lib/config.py |
Adds cluster apply + kubectl compare (via yq filtering) helpers. |
e2e-tests/lib/bash_wrapper.py |
Runs legacy run bash scripts with live output under pytest. |
e2e-tests/lib/__init__.py |
Marks lib as a Python package for imports/plugins. |
e2e-tests/init-deploy/test_init_deploy.py |
Migrates init-deploy scenario to pytest. |
e2e-tests/init-deploy/conf/another-name-rs0.yml |
Adjusts replset termination grace period used by migrated test. |
e2e-tests/init-deploy/compare/statefulset_another-name-rs0.yml |
Updates expected manifest to match grace period change. |
e2e-tests/init-deploy/compare/statefulset_another-name-rs0-oc.yml |
Same as above for OpenShift expected manifest. |
e2e-tests/init-deploy/compare/statefulset_another-name-rs0-4-oc.yml |
Same as above for OpenShift expected manifest variant. |
e2e-tests/init-deploy/compare/find-1.json |
Adds expected JSON output for migrated init-deploy checks. |
e2e-tests/init-deploy/compare/find-2.json |
Adds expected JSON output for migrated init-deploy checks. |
e2e-tests/init-deploy/compare/find-3.json |
Adds expected JSON output for migrated init-deploy checks. |
e2e-tests/functions |
Adds namespace marker for pytest failure diagnostics (legacy bash tests). |
e2e-tests/finalizer/test_finalizer.py |
Migrates finalizer test to pytest. |
e2e-tests/conftest.py |
Adds pytest fixtures + env setup + failure report enrichment. |
.gitignore |
Ignores pytest reports and Python __pycache__. |
.github/workflows/e2e-py-check.yml |
Adds ruff + mypy checks for Python e2e code on PRs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+3
to
+7
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'e2e-tests/**/*.py' | ||
|
|
Comment on lines
+291
to
+295
| export DEBUG_TESTS=1 | ||
| export KUBECONFIG=/tmp/${CLUSTER_NAME}-${clusterSuffix} | ||
| export PATH="\$HOME/.local/bin:\$PATH" | ||
| mkdir -p e2e-tests/reports | ||
|
|
Comment on lines
1886
to
+1889
| local ns="$1" | ||
|
|
||
| echo "$ns" > /tmp/pytest_current_namespace | ||
|
|
Comment on lines
+83
to
+88
| """Get namespace from global or temp file (for bash wrapper tests).""" | ||
| if _current_namespace: | ||
| return _current_namespace | ||
| try: | ||
| with open("/tmp/pytest_current_namespace") as f: | ||
| return f.read().strip() or None |
Comment on lines
+1
to
+3
| import os | ||
| import re | ||
| from typing import Any, List, MutableSequence |
Comment on lines
+93
to
+98
| def highlight_log_levels(logs: str) -> str: | ||
| """Add basic color highlighting for common log levels""" | ||
| return LOG_LEVEL_PATTERN.sub( | ||
| lambda m: f'<span style="color: {LOG_LEVEL_COLORS[m.group(1)]};">{m.group(1)}</span>', | ||
| logs, | ||
| ) |
Comment on lines
+136
to
+139
| create_collapsible_section("Operator Pod Logs", highlight_log_levels(summary["logs"])) | ||
| ), | ||
| extras.html(create_collapsible_section("Kubernetes Resources", summary["resources"])), | ||
| extras.html( |
Comment on lines
+41
to
+45
| for i, var in enumerate(ENV_VARS_TO_REPORT): | ||
| value = os.environ.get(var, "") | ||
| escaped_value = value.replace("<", "<").replace(">", ">") | ||
| bg = "#f9f9f9" if i % 2 == 0 else "#ffffff" | ||
| rows += f'<tr style="background-color: {bg};"><td style="border: 1px solid #ddd; padding: 6px;"><code>{var}</code></td><td style="border: 1px solid #ddd; padding: 6px;"><code>{escaped_value}</code></td></tr>\n' |
Comment on lines
+30
to
+46
| replica_set = "cfg" if "cfg" in uri else "rs0" | ||
| connection_string = f"{driver}://{uri}{suffix}/admin?ssl=false&replicaSet={replica_set}" | ||
| if mongo_flag: | ||
| connection_string += f" {mongo_flag}" | ||
|
|
||
| result = kubectl_bin( | ||
| "exec", | ||
| self.client, | ||
| "--", | ||
| "timeout", | ||
| str(timeout), | ||
| "mongosh", | ||
| f"{connection_string}", | ||
| "--eval", | ||
| command, | ||
| "--quiet", | ||
| check=False, |
Collaborator
Commit: fb8cc71 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Due to the high volume of requests, we're unable to provide free service for this account. To continue using the service, please upgarde to a paid plan.
CHANGE DESCRIPTION
This PR sets up the initial groundwork for transitioning to pytest as our testing framework. It includes:
Create virtual environment and install dependencies:
uv sync --lockedRun all tests:
uv run pytestRun a specific test
uv run pytest e2e-tests/init-deployEnabling debug logging:
LOG_LEVEL=DEBUG uv run pytestCHECKLIST
Jira
Needs Doc) and QA (Needs QA)?Tests
compare/*-oc.yml)?Config/Logging/Testability