Weaver live check test util

[lmolkova]

Description

Adds WeaverLiveCheck helper that allows to validate instrumentation output against otel semconv.

Examples:

1. you can call report = check.end() and it won't raise. You can explore report (which contains a list of violations and also full weaver report as dict from json) - party on

        with WeaverLiveCheck() as weaver:
            # ... configure provider, emit telemetry ...
            provider.force_flush()
            report = weaver.end()

        seen_metrics = report["statistics"]["seen_registry_metrics"]
        self.assertEqual(seen_metrics.get("test.metric"), 1)

2. you can call report = check.end_and_check() and it'd raise LiveCheckError (extends AssertionError) if any violations are found. You still get full report as a property on the error if you want to assert on more

        with WeaverLiveCheck() as weaver:
            # ... configure provider, emit telemetry ...
            provider.force_flush()

            with self.assertRaises(LiveCheckError) as cm:
                weaver.end_and_check()

        seen_metrics = cm.exception.report["statistics"]["seen_registry_metrics"]
        self.assertEqual(seen_metrics.get("test.metric"), 1)

Checklist:

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[xrmx]

Love this, thanks a lot!

[herin049]

Now that we've dropped 3.9, you could do:

def _collect(obj: Any) -> list[dict]:
    match obj:
        case {"live_check_result": {"all_advice": advices}, **_rest}:
            violations = [a for a in advices if a.get("level") == "violation"]
            return violations + list(chain.from_iterable(_collect(v) for v in obj.values()))
        case dict():
            return list(chain.from_iterable(_collect(v) for v in obj.values()))
        case list():
            return list(chain.from_iterable(_collect(item) for item in obj))
        case _:
            return []

raw = _collect(report)

[lmolkova]

nice!

[lmolkova]

it segfaults on pypy 310 on ubuntu, reverted some parts

Fatal Python error: Segmentation fault

Stack (most recent call first, approximate line numbers):
  File "/home/runner/work/opentelemetry-python/opentelemetry-python/tests/opentelemetry-test-utils/src/opentelemetry/test/weaver_live_check.py", line 54 in _collect
  File "/home/runner/work/opentelemetry-python/opentelemetry-python/tests/opentelemetry-test-utils/src/opentelemetry/test/weaver_live_check.py", line 65 in <genexpr>
  File "/home/runner/work/opentelemetry-python/opentelemetry-python/tests/opentelemetry-test-utils/src/opentelemetry/test/weaver_live_check.py", line 54 in _collect
  File "/home/runner/work/opentelemetry-python/opentelemetry-python/tests/opentelemetry-test-utils/src/opentelemetry/test/weaver_live_check.py", line 69 in <genexpr>
  File "/home/runner/work/opentelemetry-python/opentelemetry-python/tests/opentelemetry-test-utils/src/opentelemetry/test/weaver_live_check.py", line 54 in _collect
  File "/home/runner/work/opentelemetry-python/opentelemetry-python/tests/opentelemetry-test-utils/src/opentelemetry/test/weaver_live_check.py", line 65 in <genexpr>
  File "/home/runner/work/opentelemetry-python/opentelemetry-python/tests/opentelemetry-test-utils/src/opentelemetry/test/weaver_live_check.py", line 54 in _collect
  File "/home/runner/work/opentelemetry-python/opentelemetry-python/tests/opentelemetry-test-utils/src/opentelemetry/test/weaver_live_check.py", line 41 in _extract_violations
  File "/home/runner/work/opentelemetry-python/opentelemetry-python/tests/opentelemetry-test-utils/src/opentelemetry/test/weaver_live_check.py", line 175 in violations
  File "/opt/hostedtoolcache/PyPy/3.10.16/x64/lib/pypy3.10/functools.py", line 973 in __get__
  File "/home/runner/work/opentelemetry-python/opentelemetry-python/tests/opentelemetry-test-utils/tests/test_weaver_live_check.py", line 64 in test_end_and_check_no_violations

https://github.com/open-telemetry/opentelemetry-python/actions/runs/24545978826/job/71761592153?pr=5088

[lmolkova]

for whatever reason tox does not like -e along with platform marker, but we don't really need those as editable