feat(otel): Zero-code auto-instrumentation via opentelemetry-distro (#2349)

Author: schloerke

CHANGELOG.md

@@ -31,6 +31,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Improvements
 
+* The `shiny[otel]` optional dependency group now includes `opentelemetry-distro[otlp]`, so OpenTelemetry zero-code auto-instrumentation works out of the box: `opentelemetry-instrument shiny run app.py`. This is now the documented standard way to enable OpenTelemetry — the docs and the `examples/open-telemetry/` example no longer configure providers inside the app (in-code `set_tracer_provider()` setup is silently ignored when a provider is already installed, e.g. under `opentelemetry-instrument`). The OTLP exporters are also included, making it easy to switch between gRPC and HTTP export via standard `OTEL_*` environment variables. Note that `opentelemetry-distro` pins `opentelemetry-sdk` to a matching minor version, so if you combine `shiny[otel]` with other packages that pin the OpenTelemetry SDK (e.g. `logfire`), the resolver may need matching versions. (#2349)
+
 * `@add_example()` (internal docs decorator) gained an `example_name=` parameter that looks up the example in the nearest `api-examples/` directory, and all `ex_dir=` call sites that pointed inside an `api-examples/` tree were migrated to it. `ex_dir=` remains only for examples outside the nearest `api-examples/` tree. This also fixed nine call sites that passed the example name positionally (where it was silently treated as `app_file=`), so their examples were missing from the generated docs. (#2328)
 
 * Playwright's `OutputDataFrame.set_filter()` controller now supports multi-column filters (#2093)

examples/open-telemetry/README.md

@@ -154,10 +154,28 @@ pip install -r requirements.txt
 
 ### Run the App
 
+Run the app under OpenTelemetry zero-code auto-instrumentation
+(`opentelemetry-instrument` ships with `shiny[otel]`), printing spans and log
+events to the console:
+
+```bash
+opentelemetry-instrument --traces_exporter console --logs_exporter console \
+    --metrics_exporter none shiny run app.py
+```
+
+To export to an observability backend instead, drop the exporter flags and use
+standard `OTEL_*` environment variables (OTLP is the default exporter):
+
 ```bash
-python app.py
+OTEL_SERVICE_NAME=my-shiny-app opentelemetry-instrument shiny run app.py
 ```
 
+Note: the app itself intentionally contains **no** OpenTelemetry setup code.
+Instrumentation is an operational concern applied at launch. Configuring providers
+inside the app (`trace.set_tracer_provider(...)`) is discouraged: providers can only
+be installed once per process, so in-code setup conflicts with — and is silently
+ignored under — external instrumentation.
+
 The app will open in your browser. As you interact with the buttons:
 
 - **"Compute (Normal Telemetry)"**: Creates Shiny spans for reactive execution
@@ -170,8 +188,8 @@ Watch the console output to see which spans are created.
 You can set the default collection level via environment variable:
 
 ```bash
-export SHINY_OTEL_COLLECT=session
-python app.py
+SHINY_OTEL_COLLECT=session opentelemetry-instrument --traces_exporter console \
+    --logs_exporter console --metrics_exporter none shiny run app.py
 ```
 
 The `otel.suppress` decorator and context manager will override this default.
@@ -180,10 +198,15 @@ The `otel.suppress` decorator and context manager will override this default.
 
 ### Default Collection Level
 
-The default collection level is set by:
+The default collection level is set by the `SHINY_OTEL_COLLECT` environment variable
+(defaults to `all`).
 
-1. The `SHINY_OTEL_COLLECT` environment variable (defaults to `all`)
-2. Can be overridden programmatically with `otel.suppress` / `otel.suppress()`
+`otel.suppress` and `otel.collect` are *absolute* per-object overrides that take
+precedence over the global level: `suppress` forces telemetry off even when the
+global level is `all`, and `collect` forces it on even when the global level is
+lower — running this app with `SHINY_OTEL_COLLECT=session` still produces spans for
+the `otel.collect` card. Infrastructure spans (`session_start`, `session_end`,
+`reactive_update`) follow only the environment variable.
 
 ### Span Hierarchy
 

examples/open-telemetry/app.py

@@ -6,53 +6,43 @@
 2. Automatic logging of reactive value updates
 3. How `otel.suppress` suppresses both Shiny's internal spans and value logs
 
-Run with:
-    SHINY_OTEL_COLLECT=all python app.py
+Run under OpenTelemetry zero-code auto-instrumentation (installed with
+`pip install "shiny[otel]"`), printing spans and log events to the console:
 
-Or with shinylive:
-    shinylive export . site
-    python -m http.server --directory site 8008
+    opentelemetry-instrument --traces_exporter console --logs_exporter console \\
+        --metrics_exporter none shiny run app.py
+
+To see how the global collect level interacts with `otel.suppress` / `otel.collect`,
+lower it with `SHINY_OTEL_COLLECT` (the app displays the active level):
+
+    SHINY_OTEL_COLLECT=session opentelemetry-instrument --traces_exporter console \\
+        --logs_exporter console --metrics_exporter none shiny run app.py
+
+At `session` level the "Normal" card stops producing spans, but the
+`otel.collect` card still does -- see the comment on `with otel.collect():`
+below for why.
 
 Watch the console for spans and log events as you interact with the app.
 """
 
-# Set up OpenTelemetry logging to the console
-from opentelemetry import trace
-from opentelemetry._logs import set_logger_provider
-from opentelemetry.sdk._logs import LoggerProvider
-from opentelemetry.sdk._logs.export import (
-    ConsoleLogRecordExporter,
-    SimpleLogRecordProcessor,
-)
-from opentelemetry.sdk.trace import TracerProvider
-from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
-
 from shiny import App, otel, reactive, render, ui
 
-# Set up debug OpenTelemetry tracing/logging to the console
-# NOT recommended for production use
-trace_provider = TracerProvider()
-trace_provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
-trace.set_tracer_provider(trace_provider)
-log_provider = LoggerProvider()
-log_provider.add_log_record_processor(
-    SimpleLogRecordProcessor(ConsoleLogRecordExporter())
-)
-set_logger_provider(log_provider)
-
-# # Typical setup behavior:
-# import logfire
-# logfire.configure(
-#     token="fake-token-for-demo",
-#     service_name="shiny-open-telemetry-demo",
-#     environment="development",
-# )
+# This app intentionally contains NO OpenTelemetry setup code. Instrumentation
+# is an operational concern: apply it at launch with `opentelemetry-instrument`
+# (see the module docstring above) and configure exporters via its CLI flags or
+# standard `OTEL_*` environment variables. Configuring providers inside the app
+# (`trace.set_tracer_provider(...)`) is discouraged -- providers can only be
+# installed once per process, so code-based setup conflicts with (and is ignored
+# under) external instrumentation.
 
 app_ui = ui.page_fluid(
     ui.h2("OpenTelemetry: Collection Control & Value Logging"),
-    ui.markdown("""
+    ui.markdown(f"""
         This demo shows how `otel.suppress` controls **Shiny's internal spans and value logs**.
 
+        Global collect level: **`{otel.get_level().name.lower()}`**
+        (set via `SHINY_OTEL_COLLECT`; defaults to `all`)
+
         Watch the console to see:
         - **Spans**: Session lifecycle, reactive execution
         - **Logs**: Reactive value updates with source references
@@ -88,7 +78,8 @@
             ui.markdown("""
                 **Telemetry:** ✅ Shiny spans + value logs
                 Uses `@otel.collect` to re-enable Shiny telemetry inside
-                a broad `otel.suppress()` block.
+                a broad `otel.suppress()` block — even when the global
+                `SHINY_OTEL_COLLECT` level is lower.
                 """),
         ),
     ),
@@ -147,8 +138,17 @@ def private_counter_display():
             counter_val = private_counter()
             return f"Slider: {slider_val}\nCounter: {counter_val}\n\n❌ No telemetry"
 
+        # Demonstrates re-enabling telemetry within a suppressed context.
+        #
+        # Note: `otel.collect` is an ABSOLUTE per-object setting, not a relative
+        # one. Reactive objects created inside this block capture level ALL at
+        # creation time, which takes precedence over the global level -- so
+        # `output collect_counter_display` still produces spans even when
+        # SHINY_OTEL_COLLECT is set lower (e.g. `session`). `otel.suppress` is
+        # symmetric: it forces telemetry off even when the global level is
+        # `all`. Use `collect` to keep a few critical outputs observable while
+        # the rest of the app runs quiet.
         with otel.collect():
-            # Demonstrates re-enabling telemetry within a suppressed context
             collect_counter = reactive.value(0)
 
             @reactive.effect

pyproject.toml

@@ -80,7 +80,14 @@ dependencies = [
 
 [project.optional-dependencies]
 theme = ["libsass>=0.23.0", "brand_yml>=0.2.0"]
-otel = ["opentelemetry-sdk>=1.24.0"]
+otel = [
+    # 1.24.0+ required: same sanitized-stack-trace floor as opentelemetry-api above
+    "opentelemetry-sdk>=1.24.0",
+    # Enables zero-code auto-instrumentation via `opentelemetry-instrument shiny run
+    # app.py`. The [otlp] extra provides the default `otlp` exporter so the bare
+    # command works without extra flags (and allows switching between gRPC/HTTP).
+    "opentelemetry-distro[otlp]>=0.45b0",
+]
 test = [
     "pytest>=6.2.4",
     "pytest-asyncio>=0.17.2",