Merge branch 'main' into weaver-live-check

Author: lmolkova

AGENTS.md

@@ -0,0 +1,76 @@
+# OpenTelemetry Python
+
+This file is here to steer AI assisted PRs towards being high quality and valuable contributions
+that do not create excessive maintainer burden.
+
+Monorepo with the core OpenTelemetry Python API, SDK, and related packages.
+
+## General Rules and Guidelines
+
+The OpenTelemetry community has broader guidance on GenAI contributions at
+https://github.com/open-telemetry/community/blob/main/policies/genai.md — please read it before
+contributing.
+
+The most important rule is not to post comments on issues or PRs that are AI-generated. Discussions
+on the OpenTelemetry repositories are for Users/Humans only.
+
+Follow the PR scoping guidance in [CONTRIBUTING.md](CONTRIBUTING.md). Keep AI-assisted PRs tightly
+isolated to the requested change and never include unrelated cleanup or opportunistic improvements
+unless they are strictly necessary for correctness.
+
+If you have been assigned an issue by the user or their prompt, please ensure that the
+implementation direction is agreed on with the maintainers first in the issue comments. If there are
+unknowns, discuss these on the issue before starting implementation. Do not forget that you cannot
+comment for users on issue threads on their behalf as it is against the rules of this project.
+
+## Structure
+
+- `opentelemetry-api/` - the OpenTelemetry API package
+- `opentelemetry-sdk/` - the OpenTelemetry SDK package
+- `opentelemetry-semantic-conventions/` - semantic conventions
+- `opentelemetry-proto/` / `opentelemetry-proto-json/` - protobuf definitions and generated code
+- `exporter/` - exporters (OTLP, Prometheus, Zipkin, etc.)
+- `propagator/` - context propagators (B3, Jaeger)
+- `shim/` - compatibility shims (OpenTracing, OpenCensus)
+
+Each package lives under its own directory with a `pyproject.toml` and `tests/`.
+
+## Commands
+
+```sh
+# Install all packages and dev tools
+uv sync --frozen --all-packages
+
+# Lint (runs ruff via pre-commit)
+uv run tox -e precommit
+
+# Test a specific package
+uv run tox -e py312-test-opentelemetry-sdk
+
+# Lint (pylint) a specific package
+uv run tox -e lint-opentelemetry-sdk
+
+# Type check
+uv run tox -e typecheck
+```
+
+## Guidelines
+
+- Each package has its own `pyproject.toml` with version, dependencies, and entry points.
+- The monorepo uses `uv` workspaces.
+- `tox.ini` defines the test matrix - check it for available test environments.
+- Do not add `type: ignore` comments. If a type error arises, solve it properly or write a follow-up plan to address it in another PR.
+- Whenever applicable, all code changes should have tests that actually validate the changes.
+
+## Commit formatting
+
+We appreciate it if users disclose the use of AI tools when the significant part of a commit is
+taken from a tool without changes. When making a commit this should be disclosed through an
+`Assisted-by:` commit message trailer.
+
+Examples:
+
+```
+Assisted-by: ChatGPT 5.2
+Assisted-by: Claude Opus 4.6
+```

CHANGELOG.md

@@ -12,6 +12,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+- `opentelemetry-sdk`: fix YAML structure injection via environment variable substitution in declarative file configuration; values containing newlines are now emitted as quoted YAML scalars per spec requirement
+  ([#5091](https://github.com/open-telemetry/opentelemetry-python/pull/5091))
 - `opentelemetry-sdk`: Add `create_logger_provider`/`configure_logger_provider` to declarative file configuration, enabling LoggerProvider instantiation from config files without reading env vars
   ([#4990](https://github.com/open-telemetry/opentelemetry-python/pull/4990))
 - `opentelemetry-sdk`: Add `service` resource detector support to declarative file configuration via `detection_development.detectors[].service`
@@ -20,6 +22,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   ([#4907](https://github.com/open-telemetry/opentelemetry-python/issues/4907))
 - Drop Python 3.9 support
   ([#5076](https://github.com/open-telemetry/opentelemetry-python/pull/5076))
+- `opentelemetry-semantic-conventions`: use `X | Y` union annotation
+  ([#5096](https://github.com/open-telemetry/opentelemetry-python/pull/5096))
 - Add WeaverLiveCheck test util
   ([#5088](https://github.com/open-telemetry/opentelemetry-python/pull/5088))
 

CLAUDE.md

@@ -0,0 +1 @@
+See [AGENTS.md](AGENTS.md).

opentelemetry-sdk/src/opentelemetry/sdk/_configuration/file/_env_substitution.py

@@ -81,6 +81,23 @@ def replace_var(match) -> str:
                 f"Environment variable '{var_name}' not found and no default provided"
             )
 
+        # Per spec: "It MUST NOT be possible to inject YAML structures by
+        # environment variables." Newlines are the primary injection vector —
+        # a value like "legit\nmalicious_key: val" would create extra YAML
+        # keys if substituted verbatim. Wrap such values in a YAML
+        # double-quoted scalar so the newline is treated as literal text.
+        # Simple values (no newlines) are returned as-is so that YAML type
+        # coercion still applies per spec ("Node types MUST be interpreted
+        # after environment variable substitution takes place").
+        if "\n" in value or "\r" in value:
+            escaped = (
+                value.replace("\\", "\\\\")
+                .replace('"', '\\"')
+                .replace("\n", "\\n")
+                .replace("\r", "\\r")
+                .replace("\t", "\\t")
+            )
+            return f'"{escaped}"'
         return value
 
     return re.sub(pattern, replace_var, text)

opentelemetry-sdk/tests/_configuration/file/test_env_substitution.py

@@ -16,6 +16,8 @@
 import unittest
 from unittest.mock import patch
 
+import yaml
+
 from opentelemetry.sdk._configuration.file import (
     EnvSubstitutionError,
     substitute_env_vars,
@@ -115,3 +117,45 @@ def test_only_dollar_signs(self):
         """Test string with only escaped dollar signs."""
         result = substitute_env_vars("$$$$")
         self.assertEqual(result, "$$")
+
+    def test_newline_in_value_prevents_yaml_injection(self):
+        """Values containing newlines must not inject YAML structure.
+
+        Per spec: "It MUST NOT be possible to inject YAML structures by
+        environment variables." A value like "legit\\nmalicious_key: val"
+        must be emitted as a quoted scalar, not raw YAML.
+        """
+        with patch.dict(
+            os.environ,
+            {"SERVICE_NAME": "legit-service\nmalicious_key: injected_value"},
+        ):
+            result = substitute_env_vars(
+                "file_format: '0.1'\nservice_name: ${SERVICE_NAME}"
+            )
+        parsed = yaml.safe_load(result)
+        self.assertNotIn("malicious_key", parsed)
+        self.assertIn("legit-service", parsed["service_name"])
+
+    def test_newline_in_value_preserved_as_literal(self):
+        """Newline within a value is preserved as a literal newline character."""
+        with patch.dict(os.environ, {"MULTI": "line1\nline2"}):
+            result = substitute_env_vars("key: ${MULTI}")
+        parsed = yaml.safe_load(result)
+        self.assertEqual(parsed["key"], "line1\nline2")
+
+    def test_carriage_return_in_value_is_escaped(self):
+        """Carriage return in value is escaped, not injected."""
+        with patch.dict(os.environ, {"VAL": "text\r\nmore"}):
+            result = substitute_env_vars("key: ${VAL}")
+        parsed = yaml.safe_load(result)
+        self.assertIsInstance(parsed["key"], str)
+
+    def test_type_coercion_preserved_for_simple_values(self):
+        """Simple values without newlines still undergo YAML type coercion per spec."""
+        with patch.dict(os.environ, {"BOOL_VAL": "true", "INT_VAL": "42"}):
+            bool_result = yaml.safe_load(
+                substitute_env_vars("key: ${BOOL_VAL}")
+            )
+            int_result = yaml.safe_load(substitute_env_vars("key: ${INT_VAL}"))
+        self.assertIs(bool_result["key"], True)
+        self.assertEqual(int_result["key"], 42)

opentelemetry-sdk/tests/trace/test_span_processor.py

@@ -461,7 +461,7 @@ def delayed_flush(_):
         for mock_processor in mocks:
             multi_processor.add_span_processor(mock_processor)
 
-        flushed = multi_processor.force_flush(timeout_millis=10)
+        flushed = multi_processor.force_flush(timeout_millis=25)
         # let the thread executing the late_mock continue
         wait_event.set()
 

opentelemetry-semantic-conventions/src/opentelemetry/semconv/_incubating/metrics/container_metrics.py

@@ -13,15 +13,8 @@
 # limitations under the License.
 
 
-from typing import (
-    Callable,
-    Final,
-    Generator,
-    Iterable,
-    Optional,
-    Sequence,
-    Union,
-)
+from collections.abc import Callable, Generator, Iterable, Sequence
+from typing import Final
 
 from opentelemetry.metrics import (
     CallbackOptions,
@@ -33,10 +26,10 @@
 )
 
 # pylint: disable=invalid-name
-CallbackT = Union[
-    Callable[[CallbackOptions], Iterable[Observation]],
-    Generator[Iterable[Observation], CallbackOptions, None],
-]
+CallbackT = (
+    Callable[[CallbackOptions], Iterable[Observation]]
+    | Generator[Iterable[Observation], CallbackOptions, None]
+)
 
 CONTAINER_CPU_TIME: Final = "container.cpu.time"
 """
@@ -66,7 +59,7 @@ def create_container_cpu_time(meter: Meter) -> Counter:
 
 
 def create_container_cpu_usage(
-    meter: Meter, callbacks: Optional[Sequence[CallbackT]]
+    meter: Meter, callbacks: Sequence[CallbackT] | None
 ) -> ObservableGauge:
     """Container's CPU usage, measured in cpus. Range from 0 to the number of allocatable CPUs"""
     return meter.create_observable_gauge(
@@ -284,7 +277,7 @@ def create_container_network_io(meter: Meter) -> Counter:
 
 
 def create_container_uptime(
-    meter: Meter, callbacks: Optional[Sequence[CallbackT]]
+    meter: Meter, callbacks: Sequence[CallbackT] | None
 ) -> ObservableGauge:
     """The time the container has been running"""
     return meter.create_observable_gauge(

opentelemetry-semantic-conventions/src/opentelemetry/semconv/_incubating/metrics/cpu_metrics.py

@@ -13,15 +13,8 @@
 # limitations under the License.
 
 
-from typing import (
-    Callable,
-    Final,
-    Generator,
-    Iterable,
-    Optional,
-    Sequence,
-    Union,
-)
+from collections.abc import Callable, Generator, Iterable, Sequence
+from typing import Final
 
 from opentelemetry.metrics import (
     CallbackOptions,
@@ -32,10 +25,10 @@
 )
 
 # pylint: disable=invalid-name
-CallbackT = Union[
-    Callable[[CallbackOptions], Iterable[Observation]],
-    Generator[Iterable[Observation], CallbackOptions, None],
-]
+CallbackT = (
+    Callable[[CallbackOptions], Iterable[Observation]]
+    | Generator[Iterable[Observation], CallbackOptions, None]
+)
 
 CPU_FREQUENCY: Final = "cpu.frequency"
 """
@@ -44,7 +37,7 @@
 
 
 def create_cpu_frequency(
-    meter: Meter, callbacks: Optional[Sequence[CallbackT]]
+    meter: Meter, callbacks: Sequence[CallbackT] | None
 ) -> ObservableGauge:
     """Deprecated. Use `system.cpu.frequency` instead"""
     return meter.create_observable_gauge(
@@ -77,7 +70,7 @@ def create_cpu_time(meter: Meter) -> Counter:
 
 
 def create_cpu_utilization(
-    meter: Meter, callbacks: Optional[Sequence[CallbackT]]
+    meter: Meter, callbacks: Sequence[CallbackT] | None
 ) -> ObservableGauge:
     """Deprecated. Use `system.cpu.utilization` instead"""
     return meter.create_observable_gauge(