feat(analyzer): decorator ranges, pure-deletion narrowing, safe-line checks#5293
Merged
Merged
Conversation
Contributor
|
Warning Review limit reached
More reviews will be available in 39 minutes and 57 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughUpdates pytest marker analysis to better attribute symbol-level edits, treat more diff lines as safe for narrowing, detect deleted symbols in pure-deletion diffs, improve function-context parsing, and flag ChangesSymbol Analyzer and Conftest Improvements
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Caution Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional.
❌ Failed checks (2 errors)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsLinked repositories: Your configuration references 1 linked repositories, but your current plan allows 0. Analyzed ``, skipped Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
Report bugs in Issues Welcome! 🎉This pull request will be automatically processed with the following features: 🔄 Automatic Actions
📋 Available CommandsPR Status Management
Review & Approval
Testing & Validation
Container Operations
Cherry-pick Operations
Branch Management
Label Management
✅ Merge RequirementsThis PR will be automatically approved when the following conditions are met:
📊 Review ProcessApprovers and ReviewersApprovers:
Reviewers:
Available Labels
AI Features
Security Checks
💡 Tips
For more information, please refer to the project documentation or contact the maintainers. |
openshift-virtualization-qe-bot
added
tox:verify-tc-requirement-polarion:running
|
D/S test |
openshift-virtualization-qe-bot
added
the
tox:verify-tc-requirement-polarion:failed
rnetser
force-pushed
the
feat/analyzer-symbol-narrowing
branch
from
cc36652 to
6a00f8e
Compare
|
Clean rebase detected — no code changes compared to previous head ( |
openshift-virtualization-qe-bot
added
tox:verify-tc-requirement-polarion:running
|
D/S test |
Collaborator
Author
|
@coderabbitai |
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
gh pr view 5293 --repo RedHatQE/openshift-virtualization-tests --json files,additions,deletions,title,body
gh pr diff 5293 --repo RedHatQE/openshift-virtualization-testsLength of output: 26809
MEDIUM — Import-continuation regex over-matches module-level assignmentsFile: elif re.match(pattern=r"^[A-Za-z_]\w*(?:\s+as\s+\w+)?\s*,\s*$", string=line_content):
# Import continuation line (e.g. "TIMEOUT_2MIN," inside
# a multi-line "from ... import (...)" block).
has_unattributed = TrueThe pattern matches any bare identifier followed by a trailing comma, not just import continuations. A module-level constant like this is indistinguishable from an import: REGISTRY = (
HandlerA, # matches the regex — treated as safe
HandlerB, # matches the regex — treated as safe
)Adding LOW —
|
openshift-virtualization-qe-bot
added
tox:verify-tc-requirement-polarion:running
|
D/S test |
|
D/S test |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/tests_analyzer/README.md`:
- Around line 78-80: The safe-line heuristic docs in README are out of sync with
`_extract_modified_symbols()`: the implementation only treats closing
parentheses as safe, while the docs also claim closing brackets are recognized.
Update the docs to match the current behavior, or extend
`_extract_modified_symbols()` (and its tests) to handle `]` as well so the
documented safe patterns align with what the analyzer actually supports.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 813b65c8-d8cc-419b-866c-2492bf5ba68c
📒 Files selected for processing (3)
scripts/tests_analyzer/README.mdscripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (2)
**
⚙️ CodeRabbit configuration file
**: # AI Review and Development StandardsAssisted-by: Claude noreply@anthropic.com
Coding standards, conventions, and review guidelines for openshift-virtualization-tests.
These rules apply to ALL contributors and review tools — human and AI alike.
Strict Rules (MANDATORY)
Linter Suppressions PROHIBITED
- ❌ NEVER add
# noqa,# type: ignore,# pylint: disable- ❌ NEVER disable linter/mypy rules to work around issues
- ✅ FIX THE CODE - If linter complains, the code is wrong
- If you think a rule is wrong: ASK the user for explicit approval
Code Reuse (Search-First Development)
Before writing ANY new code:
- SEARCH codebase for existing implementations
- CHECK
utilities/for shared functions- CHECK
libs/for shared libraries- CHECK
tests/for shared fixtures and helper functions- CHECK
pyproject.tomldependencies — project packages (e.g.,pyhelper-utils,ocp-resources,openshift-python-wrapper) may already provide the functionality- VERIFY no similar logic exists elsewhere
- NEVER duplicate logic - extract to shared module
- REUSE existing code and patterns — only write new when nothing exists
External package examples:
- Shell commands — use
pyhelper_utils.shell.run_command, NEVER usesubprocess.rundirectly in test/utility code- OpenShift resources — use
ocp-resourcesclasses, NEVER construct raw YAML dictsPython Requirements
- Type hints MANDATORY - mypy strict mode in
libs/, all new public functions under utilities MUST be typed- Use
TYPE_CHECKINGfor type-only imports - wrap imports needed solely for type hints inif TYPE_CHECKING:to avoid runtime overhead and circular imports- Google-format docstrings REQUIRED - for all public functions with non-obvious return values OR side effects
- No defensive programming - fail-fast, don't hide bugs with fake defaults (see exceptions below)
- ALWAYS use
uv run-...
Files:
scripts/tests_analyzer/README.mdscripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
⚙️ CodeRabbit configuration file
**: ## PR Template Validation
Check the PR description for required sections from.github/pull_request_template.md.
Required sections (must be present, even if empty):
##### What this PR does / why we need it:— MUST be present AND have meaningful content.
Flag as HIGH if the section is missing, empty, whitespace-only, contains only HTML comments,
or contains only placeholder tokens such asTBD,TBA,N/A,-,—,none, or..##### Which issue(s) this PR fixes:— must be present (may be empty)##### Special notes for reviewer:— must be present (may be empty)##### jira-ticket:— must be present (may be empty)
If any required section is absent, orWhat this PR does / why we need it:has no content,
flag it as HIGH severity and ask the author to restore the missing template section(s).Approval Policy
You may approve the PR when ALL of the following are true:
- All your review comments have been addressed with either:
- a code/doc change that fixes the issue, or
- a substantive author response that justifies no code change.
Thread "resolved" state alone is not sufficient.
OR you had no review comments.- If you posted a test execution plan comment requesting tests, and the PR author replied
with a comment explaining why the requested tests are not needed or were already covered,
treat that as an acceptable response — do not block approval on the test plan alone.- The author's explanation must be reasonable and specific (not just "N/A" or "not needed").
Accept explanations like: "these tests were already run in CI", "this change is docs-only",
"the affected tests are quarantined", or "verified manually on cluster X".
Files:
scripts/tests_analyzer/README.mdscripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Never add linter suppressions like# noqa,# type: ignore, or# pylint: disable. Fix the code instead. If you believe a rule is wrong, ask the user for explicit approval.
Search the codebase for existing implementations before writing new code. Checkutilities/,libs/,tests/, andpyproject.tomldependencies. Never duplicate logic—extract to shared modules. Usepyhelper_utils.shell.run_commandfor shell commands instead ofsubprocess.run, and useocp-resourcesclasses instead of raw YAML dicts.
Type hints are MANDATORY. Use mypy strict mode inlibs/and all new public functions under utilities. UseTYPE_CHECKINGfor type-only imports to avoid runtime overhead and circular imports.
Write Google-format docstrings for all public functions with non-obvious return values or side effects.
Always useuv runto execute commands. Never executepython,pip,pytest,tox, orpre-commitdirectly. Useuv run python,uv run pytest,uv run tox,uv run pre-commit, anduv addfor package installation.
Always use absolute imports. Never use relative imports.
Prefer specific imports usingfrom module import funcfor functions and constants. Usefrom package import module(thenmodule.Name) when retaining the module name meaningfully improves readability. Never use bareimport modulewithout afromclause.
Always use named arguments for function calls with more than one argument.
Never use single-letter variable names. Always use descriptive, meaningful names.
No dead code. Every function, variable, and fixture must be used or removed. Code marked with# skip-unused-codeis excluded from dead code analysis (enforced via custom ruff plugin).
Prefer direct attribute access usingfoo.attr. Save to variables only when reusing the same attribute multiple times improves readability or extracting clarifies intent.
Imports must always be at the top of the module. Do not import inside functions.
No defensive programming. Fail...
Files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
🧠 Learnings (37)
📚 Learning: 2025-12-22T16:27:40.244Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3196
File: tests/network/upgrade/test_upgrade_network.py:4-4
Timestamp: 2025-12-22T16:27:40.244Z
Learning: For PRs that remove tests, rely on pytest --collect-only to verify the test discovery results (which tests are selected/deselected) and ensure the removal is clean and the test module remains functional. Full test execution is not required for test deletion PRs. This guideline applies to test files anywhere under the tests/ directory (e.g., tests/network/upgrade/test_upgrade_network.py) and should be used for similar test-deletion scenarios across the repository.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-07T09:52:12.342Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3358
File: tests/network/sriov/test_sriov.py:21-21
Timestamp: 2026-01-07T09:52:12.342Z
Learning: When a PR only removes or modifies pytest markers in tests (e.g., removing pytest.mark.post_upgrade) and the test logic remains unchanged, prefer verifying with pytest --collect-only instead of running the full test suite. This validates that marker usage and test selection behavior are preserved. If the test logic changes, or markers affect behavior beyond collection, run the full test suite to confirm.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-18T13:18:48.808Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 3273
File: tests/network/connectivity/test_ovs_linux_bridge.py:5-9
Timestamp: 2026-01-18T13:18:48.808Z
Learning: In tests/network/connectivity/test_ovs_linux_bridge.py and similar test files, prefer importing ipaddress as a module and using qualified calls like ipaddress.ip_interface(...) rather than from ipaddress import ip_interface. This preserves module context for readability, especially when chaining properties (e.g., ipaddress.ip_interface(...).ip). This is an intentional exception to the general rule favoring specific imports, and should apply to test files under the tests directory where module context aids understanding.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-18T14:51:50.846Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3495
File: tests/network/third_part_ip_request/test_third_party_ip_request.py:4-12
Timestamp: 2026-01-18T14:51:50.846Z
Learning: In the openshift-virtualization-tests repository, tests consistently import pytest as a module (import pytest) and avoid from pytest import ...; this is the established pattern across 398+ test files. Do not flag or refactor imports to use specific pytest names in tests under tests/**. If a file already follows this pattern, leave it as is; this guideline applies broadly to Python test files under the tests directory.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-29T05:30:13.982Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 3649
File: tests/network/user_defined_network/ip_specification/libipspec.py:1-4
Timestamp: 2026-01-29T05:30:13.982Z
Learning: In the openshift-virtualization-tests repository, Python imports should use module import style for the standard library 'json' (import json) rather than 'from json import ...'. This improves readability by making calls like json.loads and json.dumps explicit, and aligns with patterns used for modules like logging, ipaddress, cloudinit, and pytest. Apply this to all Python test files under tests/ (and similar test directories).
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-02-02T17:41:12.759Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3690
File: tests/after_cluster_deploy_sanity/test_after_cluster_deploy_sanity.py:65-65
Timestamp: 2026-02-02T17:41:12.759Z
Learning: In test files, keep test_* functions with simple one-line docstrings. For helper functions, utilities, and library code with non-obvious return values or side effects, use Google-style docstrings with Args, Returns, and Side effects sections. Do not require Google-style docstrings for pytest test functions themselves.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-02-03T07:34:34.184Z
Learnt from: RoniKishner
Repo: RedHatQE/openshift-virtualization-tests PR: 3697
File: tests/infrastructure/instance_types/test_common_vm_instancetype.py:53-98
Timestamp: 2026-02-03T07:34:34.184Z
Learning: In test files (Python, pytest), prefer using tier3 markers for categorization since tier2, tier1, and tier4 are not used in this repository. Do not rely on non-official markers; formalize and document the allowed markers in pytest.ini (or equivalent) to ensure consistent usage. When reviewing new tests, ensure markers align with this convention (only tier3, unless a project-wide decision defines additional tiers) and remove any unnecessary or undocumented markers.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-02-10T15:04:14.799Z
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3577
File: tests/virt/conftest.py:251-267
Timestamp: 2026-02-10T15:04:14.799Z
Learning: In Python tests, remove all bare time.sleep() calls. Replace with a waiting mechanism such as TimeoutSampler from the timeout_sampler package or a function decorated with retry (with appropriate timeout/conditions) to ensure determinism and avoid flakiness. This applies to all Python tests under the tests directory (not just this file) to maintain consistent waiting behavior across the suite.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-02-25T11:00:02.013Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3873
File: tests/network/localnet/test_non_udn_localnet.py:19-27
Timestamp: 2026-02-25T11:00:02.013Z
Learning: In the test codebase, do not import from conftest.py files. This avoids import ambiguity in pytest. Do not import constants or helpers defined in conftest.py into tests. If a value is needed in both conftest.py and test files, duplicate it in both places or place it in a separate utility module that is importable by tests.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-03-29T13:51:25.599Z
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 4267
File: tests/storage/cross_cluster_live_migration/test_cclm.py:96-106
Timestamp: 2026-03-29T13:51:25.599Z
Learning: In this repository, follow the existing pytest convention for `pytest.mark.parametrize` argument names: use a single comma-separated string for `argnames` (e.g., `"dv_wait_timeout, vms_for_cclm"`), not a tuple (e.g., `("dv_wait_timeout", "vms_for_cclm")`). Do not flag or suggest changing `argnames` to a tuple. Also note that PT006 is not enforced by Ruff in this repo, so reviewers should not treat PT006 as a reason to alter the `argnames` format.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-12T11:24:13.825Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:50-52
Timestamp: 2026-01-12T11:24:13.825Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when catching exceptions in Python, use LOGGER.error before re-raising and do not replace it with LOGGER.exception in except blocks. This follows the established pattern across the codebase.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-01-12T14:25:05.723Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3366
File: tests/storage/cdi_clone/test_clone.py:5-9
Timestamp: 2026-01-12T14:25:05.723Z
Learning: In Python tests and utility code across the repository, bitmath.parse_string_unsafe correctly parses Kubernetes quantities (e.g., '4Gi', '512Mi', PVC storage requests) without supplying system=bitmath.NIST. There are 30+ usages indicating this is the standard behavior. Reviewers should verify that code that builds or compares quantity strings does not pass the NIST parameter, and if a new test relies on quantity parsing, assume no NIST parameter is required unless explicitly documented.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-01-20T01:03:13.139Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:1-8
Timestamp: 2026-01-20T01:03:13.139Z
Learning: In the openshift-virtualization-tests repository, Python imports should consistently use module-level imports for the logging module (i.e., import logging) rather than from logging import ... The established pattern spans 270+ files and should not be flagged for refactoring. Apply this guideline to Python files across the repo (e.g., tests/network/provider_migration/libprovider.py).
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-01-21T21:26:41.805Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 3559
File: utilities/infra.py:251-254
Timestamp: 2026-01-21T21:26:41.805Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when reviewing Python code, recognize that with Python 3.14 the syntax 'except ValueError, TypeError:' is valid if there is no 'as' clause, and should not be flagged as Python 2 syntax. If you use an 'as' binding (e.g., 'except (ValueError, TypeError) as e:'), parentheses are required. Ensure this pattern is version-consistent and not flagged as Python 2 syntax when 'as' is absent.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-01-25T13:18:21.675Z
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 3571
File: tests/storage/storage_migration/utils.py:158-167
Timestamp: 2026-01-25T13:18:21.675Z
Learning: In reviews of the openshift-virtualization-tests repo (and similar Python code), avoid suggesting minor stylistic changes that require extra verification (e.g., removing dict.keys() checks for membership) unless the change has clear correctness or maintainability impact. Focus on fixes with observable behavior, security, performance, or maintainability benefits; defer low-impact style tweaks that are costly to verify.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-02-18T06:35:39.536Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3847
File: utilities/virt.py:2449-2453
Timestamp: 2026-02-18T06:35:39.536Z
Learning: In Python code, a function named clearly and self-descriptively can be deemed not to require a docstring. However, treat this as a context-specific guideline and not a universal rule. For public APIs or functions with side effects, prefer concise docstrings explaining behavior, inputs, outputs, and side effects. This guidance is based on the example in utilities/virt.py from RedHatQE/openshift-virtualization-tests where validate_libvirt_persistent_domain(vm, admin_client) was considered self-documenting.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-02-23T16:33:22.070Z
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3883
File: utilities/pytest_utils.py:441-463
Timestamp: 2026-02-23T16:33:22.070Z
Learning: In Python code reviews, the guideline to always use named arguments for multi-argument calls does not apply to built-ins or methods that have positional-only parameters (those defined with a / in their signature). Do not flag or require named arguments for calls like dict.get(key, default=None, /), list.pop(), str.split(sep, maxsplit) and similar built-ins that cannot accept keyword arguments. Apply the named-argument rule only to functions/methods that explicitly accept keyword arguments.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-03-17T01:32:02.617Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:02.617Z
Learning: In RedHatQE/openshift-virtualization-tests, when reviewing Python files, post targeted inline comments on the Files changed tab at the exact location (file and line) of the issue rather than opening a single discussion thread for multiple issues. This should be done for each applicable location to improve traceability and clarity. If multiple issues exist in the same file, address them with separate inline comments pointing to the specific lines.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-03-17T01:32:02.617Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:02.617Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, CodeRabbit should post targeted inline comments at each applicable location in the Files Changed tab, rather than aggregating multiple issues into a single PR discussion thread reply. This guideline applies to all Python files (any file ending in .py) changed in a PR; for non-Python files, follow the same inline-comment-at-location principle if relevant.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-04T13:45:29.122Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4725
File: utilities/console.py:54-59
Timestamp: 2026-05-04T13:45:29.122Z
Learning: During review of RedHatQE/openshift-virtualization-tests “lint-cleanup” PRs (e.g., changes targeting lint issues like stale noqa/utf-8 headers), do not flag existing `# type: ignore` directives that were already present before the PR and were not introduced or modified by the PR. Only raise findings for `# type: ignore` suppressions that the PR itself adds, changes, or otherwise makes newly effective (i.e., they appear in the diff as additions/edits).
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-04T13:45:33.892Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4725
File: tests/virt/cluster/common_templates/centos/test_centos_os_support.py:78-83
Timestamp: 2026-05-04T13:45:33.892Z
Learning: When reviewing lint-cleanup or formatting-only pull requests in this repo (e.g., changes like removing/updating `# noqa` comments or UTF-8 headers), do not raise findings for code patterns that already existed before the PR. Specifically, if a problematic construct such as `.is_connective(tcp_timeout=120)` was present in the base branch, suppress that finding and only raise issues when the PR itself introduces or modifies that construct (i.e., the diff adds/changes the call or its arguments). Apply this rule across all Python files (`**/*.py`).
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-05T17:01:15.294Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4739
File: tests/virt/node/descheduler/conftest.py:2-2
Timestamp: 2026-05-05T17:01:15.294Z
Learning: In this repo’s Python code, it’s acceptable (and preferred by convention) to build `run_command` inputs using `shlex.split(f"<command> {arg}")` rather than converting to direct list literals like `['oc', 'adm', 'uncordon', name]`. During code review, generally don’t flag `shlex.split(...)` usage for `run_command` calls and don’t suggest replacing it with list literals; the string-form pattern is used to keep commands readable and consistent with how they’re typed in a terminal.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-08T12:49:20.694Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 4788
File: utilities/os_utils.py:257-262
Timestamp: 2026-05-08T12:49:20.694Z
Learning: In RedHatQE/openshift-virtualization-tests, the Ruff flake8-boolean-trap rules FBT001/FBT002 are intentionally not enabled (pyproject.toml does not select the FBT rules; confirmed via `ruff check --show-settings`). Therefore, do not flag boolean positional parameters as FBT001/FBT002 violations in this repository. If Ruff configuration changes and starts selecting FBT rules, this exception should be reconsidered.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-12T05:10:24.601Z
Learnt from: acinko-rh
Repo: RedHatQE/openshift-virtualization-tests PR: 4780
File: tests/storage/utils.py:568-572
Timestamp: 2026-05-12T05:10:24.601Z
Learning: In this repository, Ruff rule UP043 ("unnecessary default type arguments") is enforced. When annotating `collections.abc.Generator` return types, prefer the single-parameter form `Generator[YieldType]` rather than `Generator[YieldType, None, None]`. Explicit `None, None` for the SendType and ReturnType are unnecessary defaults (per PEP 696) and will trigger UP043. Apply this consistently across all Python files.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-13T19:23:09.603Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4833
File: tests/network/localnet/migration_stuntime/libstuntime.py:25-25
Timestamp: 2026-05-13T19:23:09.603Z
Learning: In this repository, do not recommend adding `from __future__ import annotations` to fix forward-reference type annotation issues (e.g., Ruff UP037). Follow the established convention: use quoted string type annotations for forward references when the referenced class/type is defined later in the same file (e.g., `"ContinuousPing"`), and prefer `typing.Self` for self-referential return types.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-18T06:30:56.781Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_bitwarden.py:207-207
Timestamp: 2026-05-18T06:30:56.781Z
Learning: During Ruff/lint rule-enablement PRs in this repository (e.g., when introducing a new rule like PLC0415), it’s acceptable to keep CI green by adding per-line, targeted suppressions for pre-existing violations: add only `# noqa: <single-ruff-rule-id>` at the end of the specific violating line. In this PR context, reviewers should NOT flag these targeted `# noqa: PLC0415` comments as policy violations, assuming the suppression is for a pre-existing issue and is documented in the PR description as a candidate for follow-up cleanup. Do not allow blanket `# noqa` (without a specific rule) or `per-file-ignores`; those remain disallowed.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-18T06:31:12.015Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_pytest_utils.py:270-270
Timestamp: 2026-05-18T06:31:12.015Z
Learning: In RedHatQE/openshift-virtualization-tests, if a PR is a Ruff rule-enforcement PR and its “Special notes for reviewer” documents that pre-existing Ruff violations are being temporarily handled via per-line suppressions (e.g., `# noqa: PLC0415`) to keep CI green, reviewers should treat those specific `# noqa: <rule>` comments as an agreed, temporary mechanism. Do not flag them as code-quality issues and do not recommend removing, consolidating, or refactoring those suppressions within the same PR; cleanup/remediation is expected to happen in dedicated follow-up PRs instead.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-18T06:31:15.083Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_data_collector.py:304-304
Timestamp: 2026-05-18T06:31:15.083Z
Learning: When reviewing Python code in this repository for Ruff/linter rule rollouts, do not treat temporary suppression comments as violations in the specific migration scenario where a PR enables a new Ruff rule (e.g., PLC0415) and the PR description explicitly documents that all *pre-existing* violations are being annotated with `# noqa: <RULE>` as a short-lived measure. In that case, only flag `# noqa: <RULE>` suppressions that are newly introduced on code that did not previously violate the rule—i.e., verify via the PR diff against the prior state (and/or prior Ruff findings) that the suppressed line was already violating before the rule was enabled. Ignore suppressions that are covering violations that existed before the new rule rollout and were intentionally bulk-added for cleanup in follow-up PRs.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-18T06:31:20.848Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_hco.py:501-501
Timestamp: 2026-05-18T06:31:20.848Z
Learning: When reviewing Python code in RedHatQE/openshift-virtualization-tests, avoid flagging Ruff `# noqa: <RULE>` suppressions as issues if they were intentionally added as a temporary measure to keep CI green after a PR enables a new Ruff/lint rule (e.g., PLC0415) and the PR description documents this under "Special notes for reviewer". Treat these suppressions as deferred technical debt. Only flag `# noqa: PLC0415` (and similar rule-specific suppressions) when they are newly introduced without an accompanying documented intent in the PR (and thus appear to be masking a new violation rather than a pre-existing one).
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-18T09:09:09.479Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4878
File: utilities/unittests/test_pytest_utils.py:2194-2197
Timestamp: 2026-05-18T09:09:09.479Z
Learning: In this repository (RedHatQE/openshift-virtualization-tests), do not flag missing return type annotations or missing argument type annotations as Ruff “ANN” rule violations (e.g., ANN001/ANN002/ANN201/ANN202). The repo’s Ruff configuration does not enable ANN rules and only uses `extend-select = ["PLC0415"]`, so missing type annotations should not be treated as ANN lint failures during code review.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-19T07:48:17.119Z
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 4784
File: libs/vm/affinity.py:104-104
Timestamp: 2026-05-19T07:48:17.119Z
Learning: When using Kubernetes API models like `NodeSelectorRequirement` or `LabelSelectorRequirement` with operators `Exists` or `DoesNotExist`, the `values` field must not be non-empty. It is valid for `values` to be omitted / left as `None` (Python) / passed as `null`—Kubernetes rejects non-empty `values` for these operators, but does not require the field to be present or explicitly set to an empty list. In code reviews, do not treat missing `values=[]` for `Exists`/`DoesNotExist` as a validation issue; only flag cases where `values` is provided with actual elements.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-19T07:48:17.119Z
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 4784
File: libs/vm/affinity.py:104-104
Timestamp: 2026-05-19T07:48:17.119Z
Learning: When constructing Kubernetes `NodeSelectorRequirement` (or `LabelSelectorRequirement`) objects in code, do not treat `values` being omitted, `None`, or an empty list as an API-validation problem when the requirement’s operator is `Exists` or `DoesNotExist`. Per the Kubernetes API spec, these operators only require that the `values` array is not non-empty (i.e., it must be empty); they do not require the field to be explicitly present as `[]`. Therefore, reviewers should not flag `values=None`/missing `values` for `Exists`/`DoesNotExist`.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-26T15:52:31.613Z
Learnt from: rlobillo
Repo: RedHatQE/openshift-virtualization-tests PR: 4983
File: utilities/hco.py:376-378
Timestamp: 2026-05-26T15:52:31.613Z
Learning: For Python files in this repo, don’t raise review findings for missing type hints or missing/Google-style docstrings on an existing function when the PR’s only functional change is adding one or more new parameters to that function and the PR does not otherwise refactor or substantially rewrite its body/signature. Treat type-annotation/docstring improvements as out of scope for focused parameter-add PRs and defer them to a follow-up. Only raise missing type-hint or docstring issues when the PR introduces an entirely new function or substantially rewrites an existing one.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-06-21T20:28:07.727Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 5283
File: tests/network/libs/service.py:14-16
Timestamp: 2026-06-21T20:28:07.727Z
Learning: When reviewing Python code in this repository targeting Python 3.14 with PEP 649 (deferred annotation evaluation using descriptors) enabled by default, do not flag annotations that reference types imported only under `if TYPE_CHECKING:` (e.g., `client: DynamicClient | None = None`) as potential runtime `NameError` problems. With PEP 649 enabled, annotations are not evaluated at function definition time, so these patterns are valid without `from __future__ import annotations`.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-05-18T10:38:33.820Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4884
File: .coderabbit.yaml:79-80
Timestamp: 2026-05-18T10:38:33.820Z
Learning: In this repo, pytest tests are expected to live only in files named `test_*.py`. When applying CodeRabbit STP case B (newly added `def test_*` inside an existing test file), scope the check to existing files that match `test_*.py` and do not broaden it to newly added `def test_*` found in other non-`test_*.py` Python files.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-06-04T13:37:28.366Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 5128
File: scripts/polarion_sync/scanner.py:42-54
Timestamp: 2026-06-04T13:37:28.366Z
Learning: In this repository, the AGENTS.md guideline requiring use of `pyhelper_utils.shell.run_command` (instead of `subprocess.run`) is intended for test and supporting code under `tests/`, `utilities/`, and `libs/`. For Python files under `scripts/` (e.g., `scripts/polarion_sync/`, `scripts/tests_analyzer/`), do not treat `subprocess.run(...)` usage as a violation of that AGENTS.md guideline; it may be used directly.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
📚 Learning: 2026-06-04T13:37:29.897Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 5128
File: scripts/polarion_sync/push_gate.py:32-349
Timestamp: 2026-06-04T13:37:29.897Z
Learning: When reviewing Python code under this repo’s `scripts/` directory (e.g., `scripts/**.py`), do not flag `subprocess.run` usage as a violation of the `AGENTS.md` guideline. The `pyhelper_utils.shell.run_command` vs `subprocess.run` rule should only be applied to `tests/` and `utilities/`; treat `scripts/` as explicitly exempt and follow the existing `subprocess.run` usage pattern within `scripts/` packages (e.g., `scripts/tests_analyzer/`).
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.pyscripts/tests_analyzer/pytest_marker_analyzer.py
🪛 markdownlint-cli2 (0.22.1)
scripts/tests_analyzer/README.md
[warning] 69-69: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 74-74: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 78-78: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
🔇 Additional comments (3)
scripts/tests_analyzer/pytest_marker_analyzer.py (2)
1537-1540: Handle deleted assignment symbols in pure-deletion diffs.A pure deletion like
-pytest_plugins = [...]still yieldsmodified_symbols=set(), so_extract_modified_items_from_conftest()can miss structural plugin removals and narrow away affected tests.
1777-1780: Don’t return an empty classification for deleted decorator-only diffs.A pure deletion such as
-@pytest.mark.s390xstill falls through tomodified_symbols=set(), which can narrow away the changed test instead of forcing the conservative fallback this analyzer needs for marker/quarantine removals.scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py (1)
2947-2950: Make this regression hit the)allowlist branch.The added line is
+ join,, so the test exercises the trailing-comma regex, not the closing-paren path. If thestartswith(")")branch regresses, this test still passes.
Collaborator
Author
|
/rerun-smoke |
1 similar comment
Collaborator
Author
|
/rerun-smoke |
Collaborator
Author
|
@coderabbitai review |
Contributor
✅ Action performedReview finished.
|
CI validates polarion test case IDs — fake IDs like CNV-1234 in test data strings cause the verify-tc-requirement-polarion check to fail. Replace with generic marker names. Signed-off-by: rnetser <rnetser@redhat.com>
Collaborator
Author
Smoke Analyzer Validation ReportDate range: last 30 days | PRs scanned: 400 | PRs with CR recommendation: 202 Results (16 PRs successfully compared)
Note: 186/202 PRs returned Changes in this commit
|
Collaborator
Author
|
/approve |
Collaborator
Author
|
/verified |
Contributor
Test execution plan request detailsCRITICAL: You MUST post an inline review comment on the first changed line of the first file. As an expert software testing engineer, analyze all modified files in this PR and create a targeted test execution plan. Analysis Requirements:
Output rules: Your deliverable: Test Execution Plan
Use these formats:
Real test commands (MANDATORY when changes affect session/runtime code): When the affected code runs at session/collection time (conftest fixtures, pytest plugins,
Example output for a session-startup fix: Guidelines:
Hardware-Related Checks (SR-IOV, GPU, DPDK): When PR modifies fixtures for hardware-specific resources:
CRITICAL WORKFLOW COMPLETION RULES: When responding to this test execution plan request, you MUST follow these rules EXACTLY:
ABSOLUTE PROHIBITIONS (violating these creates empty/meaningless reviews):
Remember: The pull request review is visible to users. Additional comments are redundant noise. CRITICAL — Verification rules for smoke and gating decisions: |
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== PR changed files =="
gh pr view 5293 --repo RedHatQE/openshift-virtualization-tests --json files,headRefOid,baseRefName,headRefName \
--jq '{headRefOid, baseRefName, headRefName, files: [.files[].path]}'
echo
echo "== Name-status diff order =="
git diff --name-status HEAD~1..HEAD
echo
echo "== First file unified=0 diff =="
git diff --unified=0 HEAD~1..HEAD -- scripts/tests_analyzer/pytest_marker_analyzer.py | sed -n '1,220p'
echo
echo "== AST outline: scripts/tests_analyzer/pytest_marker_analyzer.py =="
ast-grep outline scripts/tests_analyzer/pytest_marker_analyzer.py --view expanded | sed -n '1,260p'
echo
echo "== AST outline: scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py =="
ast-grep outline scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py --view expanded | sed -n '1,260p'
echo
echo "== Relevant diff snippets: analyzer =="
git diff --unified=3 HEAD~1..HEAD -- scripts/tests_analyzer/pytest_marker_analyzer.py | sed -n '1,260p'
echo
echo "== Relevant diff snippets: tests =="
git diff --unified=3 HEAD~1..HEAD -- scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py | sed -n '1,260p'
echo
echo "== Relevant diff snippets: README =="
git diff --unified=3 HEAD~1..HEAD -- scripts/tests_analyzer/README.md | sed -n '1,220p'
echo
echo "== Smoke-marked tests =="
rg -n --glob 'tests/**' '`@pytest`\.mark\.smoke' . | sed -n '1,120p' || true
echo
echo "== Gating-marked tests =="
rg -n --glob 'tests/**' '`@pytest`\.mark\.gating' . | sed -n '1,120p' || true
echo
echo "== Markers in analyzer unit tests path =="
rg -n '`@pytest`\.mark\.(smoke|gating)' scripts/tests_analyzer/tests || true
echo
echo "== Test file symbols mentioning changed analyzer functions =="
rg -n "_extract_deleted_symbols_from_diff|_extract_modified_symbols|_parse_diff_for_functions|_symbol_start_line|_extract_modified_items_from_conftest|_check_test_impact|pytest_plugins" \
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py | sed -n '1,240p'Length of output: 35549 🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="RedHatQE/openshift-virtualization-tests"
pr=5293
echo "== PR files with patch excerpts =="
gh api "repos/${repo}/pulls/${pr}/files?per_page=100" > /tmp/pr_files_5293.json
jq -r '.[] | "FILE: \(.filename)\nSTATUS: \(.status)\nPATCH:\n\(.patch // "<no patch>")\n---"' /tmp/pr_files_5293.json | sed -n '1,320p'
echo
echo "== Compute first changed line in first PR file =="
python - <<'PY'
import json, re
from pathlib import Path
data = json.load(open('/tmp/pr_files_5293.json'))
first = data[0]
patch = first.get("patch") or ""
print(first["filename"])
print("status:", first["status"])
print("patch snippet:")
print(patch[:800])
new_line = None
for line in patch.splitlines():
if line.startswith('@@'):
m = re.search(r'\+(\d+)(?:,(\d+))?', line)
if m:
new_start = int(m.group(1))
new_line = new_start
continue
elif line.startswith('+') and not line.startswith('+++'):
print("first added line:", new_line, line)
break
elif line.startswith(' ') and new_line is not None:
new_line += 1
elif line.startswith('-') and not line.startswith('---'):
pass
PY
echo
echo "== Search analyzer unit tests for changed coverage classes =="
rg -n '^class Test(ExtractModifiedItemsFromConftestDiffFailure|ExtractModifiedSymbolsUnattributed|RunGithubModeCheckoutGithubPrInfo|CheckConftestPathwayOpaqueNarrowing|ExtractDeletedSymbolsFromDiff|PureDeletionSymbolNarrowing|TypeCheckingLineSafety|DecoratorLineRange|ImportContinuationSafety|ParseDiffForFunctionsContextReset|PytestPluginsDetection)\b|^\s+def test_' \
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py | sed -n '930,3090p'
echo
echo "== Post single inline review comment with test execution plan =="
cat > /tmp/test_plan_body.md <<'EOF'
**Test Execution Plan**
**Run smoke tests: False** — no verified dependency path from any smoke-marked test to the changed files. This PR only changes `scripts/tests_analyzer/pytest_marker_analyzer.py`, its unit tests, and `scripts/tests_analyzer/README.md`; it does not touch `utilities/`, `libs/`, or any `conftest.py`, so there is no traced smoke path through fixtures or shared runtime code.
**Run gating tests: False** — no verified dependency path from any gating-marked test to the changed files for the same reason. The affected logic is isolated to the analyzer and its dedicated unit tests.
**Affected tests to run**
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractModifiedSymbolsUnattributed` — covers safe unmapped lines and conservative fallback behavior
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractModifiedItemsFromConftestDiffFailure` — covers conftest narrowing behavior after symbol classification changes
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestRunGithubModeCheckoutGithubPrInfo` — covers GitHub/checkout path threading used by remote analysis
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestCheckConftestPathwayOpaqueNarrowing` — covers opaque conftest narrowing with modified symbol classification
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractDeletedSymbolsFromDiff` — validates pure-deletion symbol extraction
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestPureDeletionSymbolNarrowing` — validates narrowing on deletion-only diffs
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestTypeCheckingLineSafety` — validates `TYPE_CHECKING` safe-line handling
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestDecoratorLineRange` — validates decorator-inclusive symbol ranges and decorator-only diff attribution
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestImportContinuationSafety` — validates safe handling of import continuation and closing-paren lines
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestParseDiffForFunctionsContextReset` — validates decorator/context handling in diff-to-function attribution
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestPytestPluginsDetection` — validates structural detection of `pytest_plugins`
**Suggested command**
`pytest scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py -k 'ExtractModifiedSymbolsUnattributed or ExtractModifiedItemsFromConftestDiffFailure or RunGithubModeCheckoutGithubPrInfo or CheckConftestPathwayOpaqueNarrowing or ExtractDeletedSymbolsFromDiff or PureDeletionSymbolNarrowing or TypeCheckingLineSafety or DecoratorLineRange or ImportContinuationSafety or ParseDiffForFunctionsContextReset or PytestPluginsDetection'`
EOF
# Create a non-blocking PR review with one inline comment on the first changed line of the first file.
gh api "repos/${repo}/pulls/${pr}/reviews" \
-f event='COMMENT' \
-f comments[][path]='scripts/tests_analyzer/README.md' \
-F comments[][line]=127 \
-f comments[][side]='RIGHT' \
-f comments[][body@]=/tmp/test_plan_body.mdLength of output: 23703 🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="RedHatQE/openshift-virtualization-tests"
pr=5293
cat > /tmp/test_plan_body.md <<'EOF'
**Test Execution Plan**
**Run smoke tests: False** — no verified dependency path from any smoke-marked test to the changed files. This PR only changes `scripts/tests_analyzer/pytest_marker_analyzer.py`, its unit tests, and `scripts/tests_analyzer/README.md`; it does not touch `utilities/`, `libs/`, or any `conftest.py`, so there is no traced smoke path through fixtures or shared runtime code.
**Run gating tests: False** — no verified dependency path from any gating-marked test to the changed files for the same reason. The affected logic is isolated to the analyzer and its dedicated unit tests.
**Affected tests to run**
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractModifiedSymbolsUnattributed` — covers safe unmapped lines and conservative fallback behavior
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractModifiedItemsFromConftestDiffFailure` — covers conftest narrowing behavior after symbol classification changes
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestRunGithubModeCheckoutGithubPrInfo` — covers GitHub/checkout path threading used by remote analysis
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestCheckConftestPathwayOpaqueNarrowing` — covers opaque conftest narrowing with modified symbol classification
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractDeletedSymbolsFromDiff` — validates pure-deletion symbol extraction
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestPureDeletionSymbolNarrowing` — validates narrowing on deletion-only diffs
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestTypeCheckingLineSafety` — validates `TYPE_CHECKING` safe-line handling
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestDecoratorLineRange` — validates decorator-inclusive symbol ranges and decorator-only diff attribution
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestImportContinuationSafety` — validates safe handling of import continuation and closing-paren lines
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestParseDiffForFunctionsContextReset` — validates decorator/context handling in diff-to-function attribution
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestPytestPluginsDetection` — validates structural detection of `pytest_plugins`
**Suggested command**
`pytest scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py -k 'ExtractModifiedSymbolsUnattributed or ExtractModifiedItemsFromConftestDiffFailure or RunGithubModeCheckoutGithubPrInfo or CheckConftestPathwayOpaqueNarrowing or ExtractDeletedSymbolsFromDiff or PureDeletionSymbolNarrowing or TypeCheckingLineSafety or DecoratorLineRange or ImportContinuationSafety or ParseDiffForFunctionsContextReset or PytestPluginsDetection'`
EOF
python - <<'PY'
import json
body = open('/tmp/test_plan_body.md').read()
payload = {
"event": "COMMENT",
"comments": [
{
"path": "scripts/tests_analyzer/README.md",
"line": 55,
"side": "RIGHT",
"body": body,
}
],
}
with open('/tmp/review_payload.json', 'w') as f:
json.dump(payload, f)
PY
# Retry inline review using a proper JSON payload.
if gh api "repos/${repo}/pulls/${pr}/reviews" --input /tmp/review_payload.json; then
exit 0
fi
# Fallback allowed by the workflow: post as a regular PR comment if inline fails after retrying.
gh pr comment "${pr}" --repo "${repo}" --body-file /tmp/test_plan_body.mdLength of output: 5563 |
Collaborator
Author
|
@coderabbitai review |
Contributor
✅ Action performedReview finished.
|
Collaborator
Author
|
@coderabbitai full review |
Contributor
✅ Action performedFull review finished. |
Signed-off-by: rnetser <rnetser@redhat.com>
|
Clean rebase detected — no code changes compared to previous head ( |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/tests_analyzer/pytest_marker_analyzer.py`:
- Around line 2192-2232: The opaque-import shortcut in pytest_marker_analyzer’s
conftest resolution is too permissive for assignment symbols, so update this
branch to only treat known function/class symbols as safely skippable. In the
classification handling around fixture matching, make sure module-level
assignments in modified_symbols or new_symbols do not get resolved as safe just
because fixture.function_calls does not reference them; instead, detect
assignment-like symbols and fall back to the conservative opaque-import path.
Use the existing classification, fixtures_dict, and fixture.function_calls
checks to narrow the optimization without changing the behavior for true
function/class symbols.
In `@scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py`:
- Around line 1397-1440: The test no longer covers the fallback described
because `_build_line_to_symbol_map()` now maps decorator lines directly to
`test_new_feature`. Update `test_decorator_lines_outside_symbol_range_are_safe`
so it either matches the new direct-attribution behavior or uses a fixture where
the added decorator line still falls outside the symbol range and exercises
`_extract_modified_symbols`’ safe/unmapped path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: b66a9b2b-6eef-48a0-9f6b-c7167b785d6a
📒 Files selected for processing (3)
scripts/tests_analyzer/README.mdscripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📜 Review details
⏰ Context from checks skipped due to timeout. (23)
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
- GitHub Check: build-container
- GitHub Check: verify-bugs-are-open
- GitHub Check: tox
- GitHub Check: security-committer-identity
- GitHub Check: security-suspicious-paths
🧰 Additional context used
📓 Path-based instructions (2)
**
⚙️ CodeRabbit configuration file
**: # AI Review and Development StandardsAssisted-by: Claude noreply@anthropic.com
Coding standards, conventions, and review guidelines for openshift-virtualization-tests.
These rules apply to ALL contributors and review tools — human and AI alike.
Strict Rules (MANDATORY)
Linter Suppressions PROHIBITED
- ❌ NEVER add
# noqa,# type: ignore,# pylint: disable- ❌ NEVER disable linter/mypy rules to work around issues
- ✅ FIX THE CODE - If linter complains, the code is wrong
- If you think a rule is wrong: ASK the user for explicit approval
Code Reuse (Search-First Development)
Before writing ANY new code:
- SEARCH codebase for existing implementations
- CHECK
utilities/for shared functions- CHECK
libs/for shared libraries- CHECK
tests/for shared fixtures and helper functions- CHECK
pyproject.tomldependencies — project packages (e.g.,pyhelper-utils,ocp-resources,openshift-python-wrapper) may already provide the functionality- VERIFY no similar logic exists elsewhere
- NEVER duplicate logic - extract to shared module
- REUSE existing code and patterns — only write new when nothing exists
External package examples:
- Shell commands — use
pyhelper_utils.shell.run_command, NEVER usesubprocess.rundirectly in test/utility code- OpenShift resources — use
ocp-resourcesclasses, NEVER construct raw YAML dictsPython Requirements
- Type hints MANDATORY - mypy strict mode in
libs/, all new public functions under utilities MUST be typed- Use
TYPE_CHECKINGfor type-only imports - wrap imports needed solely for type hints inif TYPE_CHECKING:to avoid runtime overhead and circular imports- Google-format docstrings REQUIRED - for all public functions with non-obvious return values OR side effects
- No defensive programming - fail-fast, don't hide bugs with fake defaults (see exceptions below)
- ALWAYS use
uv run-...
Files:
scripts/tests_analyzer/README.mdscripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
⚙️ CodeRabbit configuration file
**: ## PR Template Validation
Check the PR description for required sections from.github/pull_request_template.md.
Required sections (must be present, even if empty):
##### What this PR does / why we need it:— MUST be present AND have meaningful content.
Flag as HIGH if the section is missing, empty, whitespace-only, contains only HTML comments,
or contains only placeholder tokens such asTBD,TBA,N/A,-,—,none, or..##### Which issue(s) this PR fixes:— must be present (may be empty)##### Special notes for reviewer:— must be present (may be empty)##### jira-ticket:— must be present (may be empty)
If any required section is absent, orWhat this PR does / why we need it:has no content,
flag it as HIGH severity and ask the author to restore the missing template section(s).Approval Policy
You may approve the PR when ALL of the following are true:
- All your review comments have been addressed with either:
- a code/doc change that fixes the issue, or
- a substantive author response that justifies no code change.
Thread "resolved" state alone is not sufficient.
OR you had no review comments.- If you posted a test execution plan comment requesting tests, and the PR author replied
with a comment explaining why the requested tests are not needed or were already covered,
treat that as an acceptable response — do not block approval on the test plan alone.- The author's explanation must be reasonable and specific (not just "N/A" or "not needed").
Accept explanations like: "these tests were already run in CI", "this change is docs-only",
"the affected tests are quarantined", or "verified manually on cluster X".
Files:
scripts/tests_analyzer/README.mdscripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Never add linter suppressions like# noqa,# type: ignore, or# pylint: disable. Fix the code instead. If you believe a rule is wrong, ask the user for explicit approval.
Search the codebase for existing implementations before writing new code. Checkutilities/,libs/,tests/, andpyproject.tomldependencies. Never duplicate logic—extract to shared modules. Usepyhelper_utils.shell.run_commandfor shell commands instead ofsubprocess.run, and useocp-resourcesclasses instead of raw YAML dicts.
Type hints are MANDATORY. Use mypy strict mode inlibs/and all new public functions under utilities. UseTYPE_CHECKINGfor type-only imports to avoid runtime overhead and circular imports.
Write Google-format docstrings for all public functions with non-obvious return values or side effects.
Always useuv runto execute commands. Never executepython,pip,pytest,tox, orpre-commitdirectly. Useuv run python,uv run pytest,uv run tox,uv run pre-commit, anduv addfor package installation.
Always use absolute imports. Never use relative imports.
Prefer specific imports usingfrom module import funcfor functions and constants. Usefrom package import module(thenmodule.Name) when retaining the module name meaningfully improves readability. Never use bareimport modulewithout afromclause.
Always use named arguments for function calls with more than one argument.
Never use single-letter variable names. Always use descriptive, meaningful names.
No dead code. Every function, variable, and fixture must be used or removed. Code marked with# skip-unused-codeis excluded from dead code analysis (enforced via custom ruff plugin).
Prefer direct attribute access usingfoo.attr. Save to variables only when reusing the same attribute multiple times improves readability or extracting clarifies intent.
Imports must always be at the top of the module. Do not import inside functions.
No defensive programming. Fail...
Files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
🧠 Learnings (37)
📚 Learning: 2026-01-12T11:24:13.825Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:50-52
Timestamp: 2026-01-12T11:24:13.825Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when catching exceptions in Python, use LOGGER.error before re-raising and do not replace it with LOGGER.exception in except blocks. This follows the established pattern across the codebase.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-12T14:25:05.723Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3366
File: tests/storage/cdi_clone/test_clone.py:5-9
Timestamp: 2026-01-12T14:25:05.723Z
Learning: In Python tests and utility code across the repository, bitmath.parse_string_unsafe correctly parses Kubernetes quantities (e.g., '4Gi', '512Mi', PVC storage requests) without supplying system=bitmath.NIST. There are 30+ usages indicating this is the standard behavior. Reviewers should verify that code that builds or compares quantity strings does not pass the NIST parameter, and if a new test relies on quantity parsing, assume no NIST parameter is required unless explicitly documented.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-20T01:03:13.139Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:1-8
Timestamp: 2026-01-20T01:03:13.139Z
Learning: In the openshift-virtualization-tests repository, Python imports should consistently use module-level imports for the logging module (i.e., import logging) rather than from logging import ... The established pattern spans 270+ files and should not be flagged for refactoring. Apply this guideline to Python files across the repo (e.g., tests/network/provider_migration/libprovider.py).
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-21T21:26:41.805Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 3559
File: utilities/infra.py:251-254
Timestamp: 2026-01-21T21:26:41.805Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when reviewing Python code, recognize that with Python 3.14 the syntax 'except ValueError, TypeError:' is valid if there is no 'as' clause, and should not be flagged as Python 2 syntax. If you use an 'as' binding (e.g., 'except (ValueError, TypeError) as e:'), parentheses are required. Ensure this pattern is version-consistent and not flagged as Python 2 syntax when 'as' is absent.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-25T13:18:21.675Z
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 3571
File: tests/storage/storage_migration/utils.py:158-167
Timestamp: 2026-01-25T13:18:21.675Z
Learning: In reviews of the openshift-virtualization-tests repo (and similar Python code), avoid suggesting minor stylistic changes that require extra verification (e.g., removing dict.keys() checks for membership) unless the change has clear correctness or maintainability impact. Focus on fixes with observable behavior, security, performance, or maintainability benefits; defer low-impact style tweaks that are costly to verify.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-02-18T06:35:39.536Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3847
File: utilities/virt.py:2449-2453
Timestamp: 2026-02-18T06:35:39.536Z
Learning: In Python code, a function named clearly and self-descriptively can be deemed not to require a docstring. However, treat this as a context-specific guideline and not a universal rule. For public APIs or functions with side effects, prefer concise docstrings explaining behavior, inputs, outputs, and side effects. This guidance is based on the example in utilities/virt.py from RedHatQE/openshift-virtualization-tests where validate_libvirt_persistent_domain(vm, admin_client) was considered self-documenting.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-02-23T16:33:22.070Z
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3883
File: utilities/pytest_utils.py:441-463
Timestamp: 2026-02-23T16:33:22.070Z
Learning: In Python code reviews, the guideline to always use named arguments for multi-argument calls does not apply to built-ins or methods that have positional-only parameters (those defined with a / in their signature). Do not flag or require named arguments for calls like dict.get(key, default=None, /), list.pop(), str.split(sep, maxsplit) and similar built-ins that cannot accept keyword arguments. Apply the named-argument rule only to functions/methods that explicitly accept keyword arguments.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-03-17T01:32:02.617Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:02.617Z
Learning: In RedHatQE/openshift-virtualization-tests, when reviewing Python files, post targeted inline comments on the Files changed tab at the exact location (file and line) of the issue rather than opening a single discussion thread for multiple issues. This should be done for each applicable location to improve traceability and clarity. If multiple issues exist in the same file, address them with separate inline comments pointing to the specific lines.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-03-17T01:32:02.617Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:02.617Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, CodeRabbit should post targeted inline comments at each applicable location in the Files Changed tab, rather than aggregating multiple issues into a single PR discussion thread reply. This guideline applies to all Python files (any file ending in .py) changed in a PR; for non-Python files, follow the same inline-comment-at-location principle if relevant.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-04T13:45:29.122Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4725
File: utilities/console.py:54-59
Timestamp: 2026-05-04T13:45:29.122Z
Learning: During review of RedHatQE/openshift-virtualization-tests “lint-cleanup” PRs (e.g., changes targeting lint issues like stale noqa/utf-8 headers), do not flag existing `# type: ignore` directives that were already present before the PR and were not introduced or modified by the PR. Only raise findings for `# type: ignore` suppressions that the PR itself adds, changes, or otherwise makes newly effective (i.e., they appear in the diff as additions/edits).
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-04T13:45:33.892Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4725
File: tests/virt/cluster/common_templates/centos/test_centos_os_support.py:78-83
Timestamp: 2026-05-04T13:45:33.892Z
Learning: When reviewing lint-cleanup or formatting-only pull requests in this repo (e.g., changes like removing/updating `# noqa` comments or UTF-8 headers), do not raise findings for code patterns that already existed before the PR. Specifically, if a problematic construct such as `.is_connective(tcp_timeout=120)` was present in the base branch, suppress that finding and only raise issues when the PR itself introduces or modifies that construct (i.e., the diff adds/changes the call or its arguments). Apply this rule across all Python files (`**/*.py`).
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-05T17:01:15.294Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4739
File: tests/virt/node/descheduler/conftest.py:2-2
Timestamp: 2026-05-05T17:01:15.294Z
Learning: In this repo’s Python code, it’s acceptable (and preferred by convention) to build `run_command` inputs using `shlex.split(f"<command> {arg}")` rather than converting to direct list literals like `['oc', 'adm', 'uncordon', name]`. During code review, generally don’t flag `shlex.split(...)` usage for `run_command` calls and don’t suggest replacing it with list literals; the string-form pattern is used to keep commands readable and consistent with how they’re typed in a terminal.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-08T12:49:20.694Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 4788
File: utilities/os_utils.py:257-262
Timestamp: 2026-05-08T12:49:20.694Z
Learning: In RedHatQE/openshift-virtualization-tests, the Ruff flake8-boolean-trap rules FBT001/FBT002 are intentionally not enabled (pyproject.toml does not select the FBT rules; confirmed via `ruff check --show-settings`). Therefore, do not flag boolean positional parameters as FBT001/FBT002 violations in this repository. If Ruff configuration changes and starts selecting FBT rules, this exception should be reconsidered.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-12T05:10:24.601Z
Learnt from: acinko-rh
Repo: RedHatQE/openshift-virtualization-tests PR: 4780
File: tests/storage/utils.py:568-572
Timestamp: 2026-05-12T05:10:24.601Z
Learning: In this repository, Ruff rule UP043 ("unnecessary default type arguments") is enforced. When annotating `collections.abc.Generator` return types, prefer the single-parameter form `Generator[YieldType]` rather than `Generator[YieldType, None, None]`. Explicit `None, None` for the SendType and ReturnType are unnecessary defaults (per PEP 696) and will trigger UP043. Apply this consistently across all Python files.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-13T19:23:09.603Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4833
File: tests/network/localnet/migration_stuntime/libstuntime.py:25-25
Timestamp: 2026-05-13T19:23:09.603Z
Learning: In this repository, do not recommend adding `from __future__ import annotations` to fix forward-reference type annotation issues (e.g., Ruff UP037). Follow the established convention: use quoted string type annotations for forward references when the referenced class/type is defined later in the same file (e.g., `"ContinuousPing"`), and prefer `typing.Self` for self-referential return types.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-18T06:30:56.781Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_bitwarden.py:207-207
Timestamp: 2026-05-18T06:30:56.781Z
Learning: During Ruff/lint rule-enablement PRs in this repository (e.g., when introducing a new rule like PLC0415), it’s acceptable to keep CI green by adding per-line, targeted suppressions for pre-existing violations: add only `# noqa: <single-ruff-rule-id>` at the end of the specific violating line. In this PR context, reviewers should NOT flag these targeted `# noqa: PLC0415` comments as policy violations, assuming the suppression is for a pre-existing issue and is documented in the PR description as a candidate for follow-up cleanup. Do not allow blanket `# noqa` (without a specific rule) or `per-file-ignores`; those remain disallowed.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-18T06:31:12.015Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_pytest_utils.py:270-270
Timestamp: 2026-05-18T06:31:12.015Z
Learning: In RedHatQE/openshift-virtualization-tests, if a PR is a Ruff rule-enforcement PR and its “Special notes for reviewer” documents that pre-existing Ruff violations are being temporarily handled via per-line suppressions (e.g., `# noqa: PLC0415`) to keep CI green, reviewers should treat those specific `# noqa: <rule>` comments as an agreed, temporary mechanism. Do not flag them as code-quality issues and do not recommend removing, consolidating, or refactoring those suppressions within the same PR; cleanup/remediation is expected to happen in dedicated follow-up PRs instead.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-18T06:31:15.083Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_data_collector.py:304-304
Timestamp: 2026-05-18T06:31:15.083Z
Learning: When reviewing Python code in this repository for Ruff/linter rule rollouts, do not treat temporary suppression comments as violations in the specific migration scenario where a PR enables a new Ruff rule (e.g., PLC0415) and the PR description explicitly documents that all *pre-existing* violations are being annotated with `# noqa: <RULE>` as a short-lived measure. In that case, only flag `# noqa: <RULE>` suppressions that are newly introduced on code that did not previously violate the rule—i.e., verify via the PR diff against the prior state (and/or prior Ruff findings) that the suppressed line was already violating before the rule was enabled. Ignore suppressions that are covering violations that existed before the new rule rollout and were intentionally bulk-added for cleanup in follow-up PRs.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-18T06:31:20.848Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_hco.py:501-501
Timestamp: 2026-05-18T06:31:20.848Z
Learning: When reviewing Python code in RedHatQE/openshift-virtualization-tests, avoid flagging Ruff `# noqa: <RULE>` suppressions as issues if they were intentionally added as a temporary measure to keep CI green after a PR enables a new Ruff/lint rule (e.g., PLC0415) and the PR description documents this under "Special notes for reviewer". Treat these suppressions as deferred technical debt. Only flag `# noqa: PLC0415` (and similar rule-specific suppressions) when they are newly introduced without an accompanying documented intent in the PR (and thus appear to be masking a new violation rather than a pre-existing one).
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-18T09:09:09.479Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4878
File: utilities/unittests/test_pytest_utils.py:2194-2197
Timestamp: 2026-05-18T09:09:09.479Z
Learning: In this repository (RedHatQE/openshift-virtualization-tests), do not flag missing return type annotations or missing argument type annotations as Ruff “ANN” rule violations (e.g., ANN001/ANN002/ANN201/ANN202). The repo’s Ruff configuration does not enable ANN rules and only uses `extend-select = ["PLC0415"]`, so missing type annotations should not be treated as ANN lint failures during code review.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-19T07:48:17.119Z
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 4784
File: libs/vm/affinity.py:104-104
Timestamp: 2026-05-19T07:48:17.119Z
Learning: When using Kubernetes API models like `NodeSelectorRequirement` or `LabelSelectorRequirement` with operators `Exists` or `DoesNotExist`, the `values` field must not be non-empty. It is valid for `values` to be omitted / left as `None` (Python) / passed as `null`—Kubernetes rejects non-empty `values` for these operators, but does not require the field to be present or explicitly set to an empty list. In code reviews, do not treat missing `values=[]` for `Exists`/`DoesNotExist` as a validation issue; only flag cases where `values` is provided with actual elements.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-19T07:48:17.119Z
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 4784
File: libs/vm/affinity.py:104-104
Timestamp: 2026-05-19T07:48:17.119Z
Learning: When constructing Kubernetes `NodeSelectorRequirement` (or `LabelSelectorRequirement`) objects in code, do not treat `values` being omitted, `None`, or an empty list as an API-validation problem when the requirement’s operator is `Exists` or `DoesNotExist`. Per the Kubernetes API spec, these operators only require that the `values` array is not non-empty (i.e., it must be empty); they do not require the field to be explicitly present as `[]`. Therefore, reviewers should not flag `values=None`/missing `values` for `Exists`/`DoesNotExist`.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-26T15:52:31.613Z
Learnt from: rlobillo
Repo: RedHatQE/openshift-virtualization-tests PR: 4983
File: utilities/hco.py:376-378
Timestamp: 2026-05-26T15:52:31.613Z
Learning: For Python files in this repo, don’t raise review findings for missing type hints or missing/Google-style docstrings on an existing function when the PR’s only functional change is adding one or more new parameters to that function and the PR does not otherwise refactor or substantially rewrite its body/signature. Treat type-annotation/docstring improvements as out of scope for focused parameter-add PRs and defer them to a follow-up. Only raise missing type-hint or docstring issues when the PR introduces an entirely new function or substantially rewrites an existing one.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-06-21T20:28:07.727Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 5283
File: tests/network/libs/service.py:14-16
Timestamp: 2026-06-21T20:28:07.727Z
Learning: When reviewing Python code in this repository targeting Python 3.14 with PEP 649 (deferred annotation evaluation using descriptors) enabled by default, do not flag annotations that reference types imported only under `if TYPE_CHECKING:` (e.g., `client: DynamicClient | None = None`) as potential runtime `NameError` problems. With PEP 649 enabled, annotations are not evaluated at function definition time, so these patterns are valid without `from __future__ import annotations`.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-06-04T13:37:28.366Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 5128
File: scripts/polarion_sync/scanner.py:42-54
Timestamp: 2026-06-04T13:37:28.366Z
Learning: In this repository, the AGENTS.md guideline requiring use of `pyhelper_utils.shell.run_command` (instead of `subprocess.run`) is intended for test and supporting code under `tests/`, `utilities/`, and `libs/`. For Python files under `scripts/` (e.g., `scripts/polarion_sync/`, `scripts/tests_analyzer/`), do not treat `subprocess.run(...)` usage as a violation of that AGENTS.md guideline; it may be used directly.
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-06-04T13:37:29.897Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 5128
File: scripts/polarion_sync/push_gate.py:32-349
Timestamp: 2026-06-04T13:37:29.897Z
Learning: When reviewing Python code under this repo’s `scripts/` directory (e.g., `scripts/**.py`), do not flag `subprocess.run` usage as a violation of the `AGENTS.md` guideline. The `pyhelper_utils.shell.run_command` vs `subprocess.run` rule should only be applied to `tests/` and `utilities/`; treat `scripts/` as explicitly exempt and follow the existing `subprocess.run` usage pattern within `scripts/` packages (e.g., `scripts/tests_analyzer/`).
Applied to files:
scripts/tests_analyzer/pytest_marker_analyzer.pyscripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2025-12-22T16:27:40.244Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3196
File: tests/network/upgrade/test_upgrade_network.py:4-4
Timestamp: 2025-12-22T16:27:40.244Z
Learning: For PRs that remove tests, rely on pytest --collect-only to verify the test discovery results (which tests are selected/deselected) and ensure the removal is clean and the test module remains functional. Full test execution is not required for test deletion PRs. This guideline applies to test files anywhere under the tests/ directory (e.g., tests/network/upgrade/test_upgrade_network.py) and should be used for similar test-deletion scenarios across the repository.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-07T09:52:12.342Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3358
File: tests/network/sriov/test_sriov.py:21-21
Timestamp: 2026-01-07T09:52:12.342Z
Learning: When a PR only removes or modifies pytest markers in tests (e.g., removing pytest.mark.post_upgrade) and the test logic remains unchanged, prefer verifying with pytest --collect-only instead of running the full test suite. This validates that marker usage and test selection behavior are preserved. If the test logic changes, or markers affect behavior beyond collection, run the full test suite to confirm.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-18T13:18:48.808Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 3273
File: tests/network/connectivity/test_ovs_linux_bridge.py:5-9
Timestamp: 2026-01-18T13:18:48.808Z
Learning: In tests/network/connectivity/test_ovs_linux_bridge.py and similar test files, prefer importing ipaddress as a module and using qualified calls like ipaddress.ip_interface(...) rather than from ipaddress import ip_interface. This preserves module context for readability, especially when chaining properties (e.g., ipaddress.ip_interface(...).ip). This is an intentional exception to the general rule favoring specific imports, and should apply to test files under the tests directory where module context aids understanding.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-18T14:51:50.846Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3495
File: tests/network/third_part_ip_request/test_third_party_ip_request.py:4-12
Timestamp: 2026-01-18T14:51:50.846Z
Learning: In the openshift-virtualization-tests repository, tests consistently import pytest as a module (import pytest) and avoid from pytest import ...; this is the established pattern across 398+ test files. Do not flag or refactor imports to use specific pytest names in tests under tests/**. If a file already follows this pattern, leave it as is; this guideline applies broadly to Python test files under the tests directory.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-01-29T05:30:13.982Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 3649
File: tests/network/user_defined_network/ip_specification/libipspec.py:1-4
Timestamp: 2026-01-29T05:30:13.982Z
Learning: In the openshift-virtualization-tests repository, Python imports should use module import style for the standard library 'json' (import json) rather than 'from json import ...'. This improves readability by making calls like json.loads and json.dumps explicit, and aligns with patterns used for modules like logging, ipaddress, cloudinit, and pytest. Apply this to all Python test files under tests/ (and similar test directories).
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-02-02T17:41:12.759Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3690
File: tests/after_cluster_deploy_sanity/test_after_cluster_deploy_sanity.py:65-65
Timestamp: 2026-02-02T17:41:12.759Z
Learning: In test files, keep test_* functions with simple one-line docstrings. For helper functions, utilities, and library code with non-obvious return values or side effects, use Google-style docstrings with Args, Returns, and Side effects sections. Do not require Google-style docstrings for pytest test functions themselves.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-02-03T07:34:34.184Z
Learnt from: RoniKishner
Repo: RedHatQE/openshift-virtualization-tests PR: 3697
File: tests/infrastructure/instance_types/test_common_vm_instancetype.py:53-98
Timestamp: 2026-02-03T07:34:34.184Z
Learning: In test files (Python, pytest), prefer using tier3 markers for categorization since tier2, tier1, and tier4 are not used in this repository. Do not rely on non-official markers; formalize and document the allowed markers in pytest.ini (or equivalent) to ensure consistent usage. When reviewing new tests, ensure markers align with this convention (only tier3, unless a project-wide decision defines additional tiers) and remove any unnecessary or undocumented markers.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-02-10T15:04:14.799Z
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3577
File: tests/virt/conftest.py:251-267
Timestamp: 2026-02-10T15:04:14.799Z
Learning: In Python tests, remove all bare time.sleep() calls. Replace with a waiting mechanism such as TimeoutSampler from the timeout_sampler package or a function decorated with retry (with appropriate timeout/conditions) to ensure determinism and avoid flakiness. This applies to all Python tests under the tests directory (not just this file) to maintain consistent waiting behavior across the suite.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-02-25T11:00:02.013Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3873
File: tests/network/localnet/test_non_udn_localnet.py:19-27
Timestamp: 2026-02-25T11:00:02.013Z
Learning: In the test codebase, do not import from conftest.py files. This avoids import ambiguity in pytest. Do not import constants or helpers defined in conftest.py into tests. If a value is needed in both conftest.py and test files, duplicate it in both places or place it in a separate utility module that is importable by tests.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-03-29T13:51:25.599Z
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 4267
File: tests/storage/cross_cluster_live_migration/test_cclm.py:96-106
Timestamp: 2026-03-29T13:51:25.599Z
Learning: In this repository, follow the existing pytest convention for `pytest.mark.parametrize` argument names: use a single comma-separated string for `argnames` (e.g., `"dv_wait_timeout, vms_for_cclm"`), not a tuple (e.g., `("dv_wait_timeout", "vms_for_cclm")`). Do not flag or suggest changing `argnames` to a tuple. Also note that PT006 is not enforced by Ruff in this repo, so reviewers should not treat PT006 as a reason to alter the `argnames` format.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
📚 Learning: 2026-05-18T10:38:33.820Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4884
File: .coderabbit.yaml:79-80
Timestamp: 2026-05-18T10:38:33.820Z
Learning: In this repo, pytest tests are expected to live only in files named `test_*.py`. When applying CodeRabbit STP case B (newly added `def test_*` inside an existing test file), scope the check to existing files that match `test_*.py` and do not broaden it to newly added `def test_*` found in other non-`test_*.py` Python files.
Applied to files:
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py
🪛 GitHub Check: can-be-merged
scripts/tests_analyzer/README.md
[error] 55-55: CI failed due to missing required LGTM approvals. Minimum 2 required, (0 given). Unresolved review conversation at scripts/tests_analyzer/README.md:55 (reviewers: vsibirsk, RoniKishner, dshchedr, myakove).
🪛 markdownlint-cli2 (0.22.1)
scripts/tests_analyzer/README.md
[warning] 69-69: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 74-74: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 78-78: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
🔇 Additional comments (1)
scripts/tests_analyzer/pytest_marker_analyzer.py (1)
1507-1541: HIGH: Pure deletions still miss top-level assignments likepytest_plugins.
_build_line_to_symbol_map()treats module-level assignments as symbols, but this helper only extracts deleteddef/classlines. A diff like-pytest_plugins = [...]therefore produces an empty deletion set, so the structural conftest path never sees the change and can skip affected tests.
Collaborator
Author
|
@coderabbitai review |
Contributor
✅ Action performedReview finished.
|
The test was incorrectly described as testing unmapped decorator lines, but _symbol_start_line() already includes decorators in symbol ranges. Rename to test_new_decorated_function_classified_as_new_symbol and add assertions verifying the function is in new_symbols, not modified_symbols. Also remove fake polarion marker references that fail CI validation. Signed-off-by: rnetser <rnetser@redhat.com>
Collaborator
Author
|
@coderabbitai approve |
Contributor
✅ Action performedComments resolved and changes approved. |
Collaborator
Author
|
/approve /verified (compared with CR nalysis) |
Contributor
Test execution plan request detailsCRITICAL: You MUST post an inline review comment on the first changed line of the first file. As an expert software testing engineer, analyze all modified files in this PR and create a targeted test execution plan. Analysis Requirements:
Output rules: Your deliverable: Test Execution Plan
Use these formats:
Real test commands (MANDATORY when changes affect session/runtime code): When the affected code runs at session/collection time (conftest fixtures, pytest plugins,
Example output for a session-startup fix: Guidelines:
Hardware-Related Checks (SR-IOV, GPU, DPDK): When PR modifies fixtures for hardware-specific resources:
CRITICAL WORKFLOW COMPLETION RULES: When responding to this test execution plan request, you MUST follow these rules EXACTLY:
ABSOLUTE PROHIBITIONS (violating these creates empty/meaningless reviews):
Remember: The pull request review is visible to users. Additional comments are redundant noise. CRITICAL — Verification rules for smoke and gating decisions: |
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="RedHatQE/openshift-virtualization-tests"
pr="5293"
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
echo "== Fetch PR files =="
gh api "repos/${repo}/pulls/${pr}/files?per_page=100" > "${tmpdir}/pr_files.json"
jq -r '.[].filename' "${tmpdir}/pr_files.json"
echo
echo "== Compute first changed file + first changed line =="
python - <<'PY' > "${tmpdir}/first_changed.json"
import json, re, sys
data = json.load(open(sys.argv[1]))
for f in data:
patch = f.get("patch") or ""
if not patch:
continue
cur = None
first_line = None
for line in patch.splitlines():
if line.startswith('@@'):
m = re.search(r'\+(\d+)(?:,(\d+))?', line)
if m:
cur = int(m.group(1))
continue
if cur is None:
continue
if line.startswith('+') and not line.startswith('+++'):
first_line = cur
break
if line.startswith(' '):
cur += 1
elif line.startswith('-') and not line.startswith('---'):
pass
if first_line is not None:
print(json.dumps({
"path": f["filename"],
"line": first_line,
"status": f["status"],
}))
sys.exit(0)
raise SystemExit("No changed line found in PR files")
PY
python - "${tmpdir}/pr_files.json" "${tmpdir}/first_changed.json" <<'PY'
import json, sys
files = json.load(open(sys.argv[1]))
first = json.load(open(sys.argv[2]))
print(f"first changed file: {first['path']}")
print(f"first changed line: {first['line']}")
for f in files:
if f["filename"] == first["path"]:
print("patch snippet:")
print((f.get("patch") or "")[:1200])
break
PY
echo
echo "== Diff summary for changed files =="
git diff --name-status HEAD~1..HEAD
echo
for f in scripts/tests_analyzer/pytest_marker_analyzer.py \
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py \
scripts/tests_analyzer/README.md; do
if [ -f "$f" ]; then
echo "--- $f"
git diff --unified=0 HEAD~1..HEAD -- "$f" | sed -n '1,220p'
echo
fi
done
echo "== AST outline: scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py =="
ast-grep outline scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py --view expanded | sed -n '1,260p' || true
echo
echo "== Relevant analyzer unit-test classes =="
rg -n '^class Test(ExtractModifiedSymbolsUnattributed|ExtractModifiedItemsFromConftestDiffFailure|RunGithubModeCheckoutGithubPrInfo|CheckConftestPathwayOpaqueNarrowing|ExtractDeletedSymbolsFromDiff|PureDeletionSymbolNarrowing|TypeCheckingLineSafety|DecoratorLineRange|ImportContinuationSafety|ParseDiffForFunctionsContextReset|PytestPluginsDetection)\b' \
scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py || true
echo
echo "== Smoke-marked tests under tests/ =="
rg -n --glob 'tests/**' '`@pytest`\.mark\.smoke' tests/ | sed -n '1,120p' || true
echo
echo "== Gating-marked tests under tests/ =="
rg -n --glob 'tests/**' '`@pytest`\.mark\.gating' tests/ | sed -n '1,120p' || true
echo
echo "== Existing PR review comments matching Test Execution Plan =="
gh api "repos/${repo}/pulls/${pr}/comments?per_page=100" > "${tmpdir}/review_comments.json"
python - "${tmpdir}/review_comments.json" "${tmpdir}/first_changed.json" <<'PY'
import json, sys
comments = json.load(open(sys.argv[1]))
first = json.load(open(sys.argv[2]))
matches = []
for c in comments:
body = c.get("body") or ""
if "**Test Execution Plan**" not in body:
continue
matches.append({
"id": c["id"],
"user": c["user"]["login"],
"path": c.get("path"),
"line": c.get("line"),
"side": c.get("side"),
"url": c.get("html_url"),
})
print(json.dumps(matches, indent=2))
already = any(
m["path"] == first["path"] and m["line"] == first["line"]
for m in matches
)
open(sys.argv[2] + ".exists", "w").write("yes\n" if already else "no\n")
PY
first_path="$(python -c 'import json; import sys; d=json.load(open(sys.argv[1])); print(d["path"])' "${tmpdir}/first_changed.json")"
first_line="$(python -c 'import json; import sys; d=json.load(open(sys.argv[1])); print(d["line"])' "${tmpdir}/first_changed.json")"
exists="$(cat "${tmpdir}/first_changed.json.exists")"
cat > "${tmpdir}/test_plan_body.md" <<'EOF'
**Test Execution Plan**
**Run smoke tests: False** — no verified dependency path from any smoke-marked test to the changed files. The PR only changes `scripts/tests_analyzer/pytest_marker_analyzer.py`, `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py`, and `scripts/tests_analyzer/README.md`; it does not modify `utilities/`, `libs/`, or any `conftest.py`, so there is no traced smoke path through fixtures or shared runtime code.
**Run gating tests: False** — no verified dependency path from any gating-marked test to the changed files for the same reason. The impacted behavior is isolated to the pytest marker analyzer and its dedicated unit tests.
**Affected tests to run**
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractModifiedSymbolsUnattributed`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractModifiedItemsFromConftestDiffFailure`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestRunGithubModeCheckoutGithubPrInfo`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestCheckConftestPathwayOpaqueNarrowing`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractDeletedSymbolsFromDiff`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestPureDeletionSymbolNarrowing`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestTypeCheckingLineSafety`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestDecoratorLineRange`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestImportContinuationSafety`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestParseDiffForFunctionsContextReset`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestPytestPluginsDetection`
**Real test commands**
- `pytest scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py -k 'ExtractModifiedSymbolsUnattributed or ExtractModifiedItemsFromConftestDiffFailure or RunGithubModeCheckoutGithubPrInfo or CheckConftestPathwayOpaqueNarrowing or ExtractDeletedSymbolsFromDiff or PureDeletionSymbolNarrowing or TypeCheckingLineSafety or DecoratorLineRange or ImportContinuationSafety or ParseDiffForFunctionsContextReset or PytestPluginsDetection'`
- `pytest scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py`
EOF
if [ "$exists" = "yes" ]; then
echo
echo "A matching Test Execution Plan comment already exists at ${first_path}:${first_line}; skipping duplicate post."
exit 0
fi
echo
echo "== Post inline review comment =="
python - "${tmpdir}/test_plan_body.md" "${first_path}" "${first_line}" > "${tmpdir}/review_payload.json" <<'PY'
import json, sys
body = open(sys.argv[1]).read()
path = sys.argv[2]
line = int(sys.argv[3])
payload = {
"event": "COMMENT",
"comments": [
{
"path": path,
"line": line,
"side": "RIGHT",
"body": body,
}
],
}
print(json.dumps(payload))
PY
gh api "repos/${repo}/pulls/${pr}/reviews" --input "${tmpdir}/review_payload.json"Length of output: 8054 🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="RedHatQE/openshift-virtualization-tests"
pr="5293"
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
echo "== Fetch PR files =="
gh api "repos/${repo}/pulls/${pr}/files?per_page=100" > "${tmpdir}/pr_files.json"
jq -r '.[].filename' "${tmpdir}/pr_files.json"
echo
echo "== Compute first changed file + first changed line =="
python - "${tmpdir}/pr_files.json" <<'PY' > "${tmpdir}/first_changed.json"
import json, re, sys
data = json.load(open(sys.argv[1]))
for f in data:
patch = f.get("patch") or ""
if not patch:
continue
cur = None
for line in patch.splitlines():
if line.startswith("@@"):
m = re.search(r"\+(\d+)(?:,(\d+))?", line)
if m:
cur = int(m.group(1))
continue
if cur is None:
continue
if line.startswith("+") and not line.startswith("+++"):
print(json.dumps({
"path": f["filename"],
"line": cur,
"status": f["status"],
}))
raise SystemExit(0)
if line.startswith(" "):
cur += 1
elif line.startswith("-") and not line.startswith("---"):
pass
raise SystemExit("No changed line found in PR files")
PY
python - "${tmpdir}/pr_files.json" "${tmpdir}/first_changed.json" <<'PY'
import json, sys
files = json.load(open(sys.argv[1]))
first = json.load(open(sys.argv[2]))
print(f"first changed file: {first['path']}")
print(f"first changed line: {first['line']}")
for f in files:
if f["filename"] == first["path"]:
print("patch snippet:")
print((f.get("patch") or "")[:1000])
break
PY
echo
echo "== Existing PR review comments matching Test Execution Plan =="
gh api "repos/${repo}/pulls/${pr}/comments?per_page=100" > "${tmpdir}/review_comments.json"
python - "${tmpdir}/review_comments.json" "${tmpdir}/first_changed.json" <<'PY'
import json, sys
comments = json.load(open(sys.argv[1]))
first = json.load(open(sys.argv[2]))
matches = []
already = False
for c in comments:
body = c.get("body") or ""
if "**Test Execution Plan**" not in body:
continue
entry = {
"id": c["id"],
"user": c["user"]["login"],
"path": c.get("path"),
"line": c.get("line"),
"side": c.get("side"),
"url": c.get("html_url"),
}
matches.append(entry)
if entry["path"] == first["path"] and entry["line"] == first["line"]:
already = True
print(json.dumps(matches, indent=2))
with open(sys.argv[2] + ".exists", "w") as fh:
fh.write("yes\n" if already else "no\n")
PY
first_path="$(python - "${tmpdir}/first_changed.json" <<'PY'
import json, sys
print(json.load(open(sys.argv[1]))["path"])
PY
)"
first_line="$(python - "${tmpdir}/first_changed.json" <<'PY'
import json, sys
print(json.load(open(sys.argv[1]))["line"])
PY
)"
exists="$(cat "${tmpdir}/first_changed.json.exists")"
cat > "${tmpdir}/test_plan_body.md" <<'EOF'
**Test Execution Plan**
**Run smoke tests: False** — no verified dependency path from any smoke-marked test to the changed files. This PR only changes `scripts/tests_analyzer/pytest_marker_analyzer.py`, `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py`, and `scripts/tests_analyzer/README.md`; it does not touch `utilities/`, `libs/`, or any `conftest.py`, so there is no traced smoke path through fixtures or shared runtime code.
**Run gating tests: False** — no verified dependency path from any gating-marked test to the changed files for the same reason. The affected behavior is isolated to the pytest marker analyzer and its dedicated unit tests.
**Affected tests to run**
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractModifiedSymbolsUnattributed`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractModifiedItemsFromConftestDiffFailure`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestRunGithubModeCheckoutGithubPrInfo`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestCheckConftestPathwayOpaqueNarrowing`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestExtractDeletedSymbolsFromDiff`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestPureDeletionSymbolNarrowing`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestTypeCheckingLineSafety`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestDecoratorLineRange`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestImportContinuationSafety`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestParseDiffForFunctionsContextReset`
- `scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py::TestPytestPluginsDetection`
**Real test commands**
- `pytest scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py -k 'ExtractModifiedSymbolsUnattributed or ExtractModifiedItemsFromConftestDiffFailure or RunGithubModeCheckoutGithubPrInfo or CheckConftestPathwayOpaqueNarrowing or ExtractDeletedSymbolsFromDiff or PureDeletionSymbolNarrowing or TypeCheckingLineSafety or DecoratorLineRange or ImportContinuationSafety or ParseDiffForFunctionsContextReset or PytestPluginsDetection'`
- `pytest scripts/tests_analyzer/tests/test_pytest_marker_analyzer.py`
EOF
if [ "$exists" = "yes" ]; then
echo
echo "A matching Test Execution Plan comment already exists at ${first_path}:${first_line}; skipping duplicate post."
exit 0
fi
echo
echo "== Post inline review comment =="
python - "${tmpdir}/test_plan_body.md" "${first_path}" "${first_line}" <<'PY' > "${tmpdir}/review_payload.json"
import json, sys
body = open(sys.argv[1]).read()
path = sys.argv[2]
line = int(sys.argv[3])
print(json.dumps({
"event": "COMMENT",
"comments": [{
"path": path,
"line": line,
"side": "RIGHT",
"body": body,
}],
}))
PY
if gh api "repos/${repo}/pulls/${pr}/reviews" --input "${tmpdir}/review_payload.json"; then
exit 0
fi
echo
echo "== Inline review post failed; fallback to regular PR comment =="
gh pr comment "${pr}" --repo "${repo}" --body-file "${tmpdir}/test_plan_body.md"Length of output: 8058 |
|
New container for quay.io/openshift-cnv/openshift-virtualization-tests:latest published |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Improves the pytest marker analyzer's symbol-level diff attribution:
Decorator line ranges:
_symbol_start_line()includes decorator lines in symbol ranges, so marker-only changes (e.g. adding@pytest.mark.s390x) map to the decorated symbol instead of triggering file-level fallback.Pure-deletion symbol narrowing: Diffs that only delete code now extract deleted
def/classnames via_extract_deleted_symbols_from_diff()for symbol-level narrowing instead of returningNone(conservative file-level fallback). This is critical for quarantine marker removal workflows.Safe unmapped line checks:
TYPE_CHECKINGguards, closing parens),__all__, and import continuation lines are now treated as safe unmapped lines that don't trigger conservative fallback.Which issue(s) this PR fixes:
Special notes for reviewer:
All changes are in the analyzer script and its unit tests. 119 tests pass. No changes to test infrastructure or production code.
jira-ticket:
Summary by CodeRabbit
def/classdefinitions are treated as meaningful modifications for narrowing.conftest.pyimpact analysis, including recognizingpytest_pluginschanges as structural (triggering dependent tests).conftest.pynarrowing behavior.pytest_pluginsbehavior.