Switch to path filtering instead of conformance markers to run conformance tests#45
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates how GenAI semantic-convention conformance tests are selected/executed so non-conformance test environments don’t import conformance-only dependencies, addressing CI breakage referenced in #10.
Changes:
- Switch conformance selection from a
pytest.mark.conformancemarker to runningtests/test_conformance.pydirectly in*-conformancetox envs, and--ignore-ing it in regular envs. - Make OTLP/gRPC exporter imports lazy in the shared conformance runner so the module can be imported without conformance-only deps installed.
- Remove the now-unused
conformancepytest marker registration and module-level marker usage; update AGENTS.md accordingly.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| util/opentelemetry-test-util-genai/src/opentelemetry/test_util_genai/conformance.py | Lazily imports OTLP/gRPC exporters and updates docs to reflect path-based conformance execution. |
| tox.ini | Runs conformance tests by targeting tests/test_conformance.py directly and ignores it in non-conformance envs. |
| pytest.ini | Removes the unused conformance marker registration. |
| instrumentation/opentelemetry-instrumentation-openai-v2/tests/test_conformance.py | Drops module-level conformance marker (no longer needed with path-based selection). |
| instrumentation/opentelemetry-instrumentation-langchain/tests/test_conformance.py | Drops module-level conformance marker (no longer needed with path-based selection). |
| instrumentation/opentelemetry-instrumentation-anthropic/tests/test_conformance.py | Drops module-level conformance marker (no longer needed with path-based selection). |
| AGENTS.md | Updates contributor guidance to match the new tox/path-based conformance execution model. |
DylanRussell
approved these changes
May 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
And fix CI broken in #10