feat: auto-forward OTEL_* env vars with one-shot token protection for headers (#3180)

* Initial plan

* feat: add OTEL env passthrough and one-shot token protection

* feat: add OTEL env passthrough and one-shot token protection

* style: fix markdown table separators in OTEL docs section

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* ci: recompile smoke-codex workflow

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Author: 3 people

.github/workflows/smoke-codex.lock.yml

@@ -192,6 +192,57 @@ When enabled, the library logs:
 
 **Note:** Debug output goes to stderr and does not interfere with command stdout. See `containers/agent/one-shot-token/README.md` for complete documentation.
 
+## OpenTelemetry (OTEL) Environment Variables
+
+AWF automatically forwards all `OTEL_*` environment variables and `COPILOT_OTEL_FILE_EXPORTER_PATH` into the agent container — no `--env-all` or explicit `--env` flags are required. This covers the full set of [OpenTelemetry SDK environment variables](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/).
+
+### Automatic forwarding
+
+Any variable present in the host environment with the `OTEL_` prefix is passed through:
+
+```bash
+export OTEL_SERVICE_NAME=my-agent
+export OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.example.com
+export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $MY_OTEL_TOKEN"
+sudo -E awf --allow-domains otel.example.com -- agent-command
+```
+
+### Security: one-shot token protection for OTEL credentials
+
+The following OTEL variables often carry bearer tokens or other credentials and are included in the one-shot token protection list (`AWF_ONE_SHOT_TOKENS`). They are cached on first access and removed from `/proc/self/environ`, preventing exfiltration by compromised subprocesses:
+
+| Variable | Content |
+|----------|---------|
+| `OTEL_EXPORTER_OTLP_HEADERS` | Global auth headers (e.g. `Authorization=Bearer <token>`) |
+| `OTEL_EXPORTER_OTLP_TRACES_HEADERS` | Per-signal auth headers |
+| `OTEL_EXPORTER_OTLP_METRICS_HEADERS` | Per-signal auth headers |
+| `OTEL_EXPORTER_OTLP_LOGS_HEADERS` | Per-signal auth headers |
+
+### Network requirements
+
+- **OTLP/HTTP (`http/protobuf`, default):** Traffic goes through the Squid proxy on ports 80/443. Add the OTLP collector domain to `--allow-domains`:
+
+  ```bash
+  awf --allow-domains otel.example.com -- agent-command
+  ```
+
+- **OTLP/gRPC (port 4317):** gRPC clients typically do not respect `HTTP_PROXY` env vars, and port 4317 is not covered by AWF's iptables DNAT rules (only 80/443). Traffic to port 4317 hits the default DROP rule and is blocked. Use `http/protobuf` protocol instead:
+
+  ```bash
+  export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
+  ```
+
+- **File-based export (`COPILOT_OTEL_FILE_EXPORTER_PATH`):** Writes spans to a local file — no network access needed. AWF forwards this variable automatically. gh-aw uploads the file as an Actions artifact.
+
+### Key OTEL variables by category
+
+| Category | Variables |
+|----------|-----------|
+| **Sensitive (one-shot protected)** | `OTEL_EXPORTER_OTLP_HEADERS`, `OTEL_EXPORTER_OTLP_TRACES_HEADERS`, `OTEL_EXPORTER_OTLP_METRICS_HEADERS`, `OTEL_EXPORTER_OTLP_LOGS_HEADERS` |
+| **Network-affecting** | `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`, `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT`, `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`, `OTEL_EXPORTER_OTLP_PROTOCOL` |
+| **Safe / local config** | `OTEL_SERVICE_NAME`, `OTEL_RESOURCE_ATTRIBUTES`, `OTEL_SDK_DISABLED`, `OTEL_LOG_LEVEL`, `OTEL_PROPAGATORS`, `OTEL_TRACES_SAMPLER`, `OTEL_TRACES_EXPORTER`, `OTEL_METRICS_EXPORTER`, `OTEL_LOGS_EXPORTER`, `OTEL_EXPORTER_OTLP_TIMEOUT`, `OTEL_EXPORTER_OTLP_COMPRESSION` |
+| **Copilot-specific** | `COPILOT_OTEL_FILE_EXPORTER_PATH` |
+
 ## Workflow-Scope Docker-in-Docker (`DOCKER_HOST`)
 
 When a GitHub Actions workflow enables Docker-in-Docker (DinD) at the **workflow scope** — for example by starting a `docker:dind` service container and setting `DOCKER_HOST: tcp://localhost:2375` in the runner's environment — AWF handles the conflict automatically.

docs/environment.md

@@ -304,4 +304,98 @@ describe('agent environment: credentials', () => {
       else delete process.env.GITHUB_TOKEN;
     }
   });
+
+  describe('OTEL environment variable forwarding', () => {
+    const otelVars: Record<string, string> = {
+      OTEL_SERVICE_NAME: 'my-service',
+      OTEL_EXPORTER_OTLP_ENDPOINT: 'https://otel.example.com:4318',
+      OTEL_EXPORTER_OTLP_HEADERS: 'Authorization=Bearer secret-token',
+      OTEL_EXPORTER_OTLP_TRACES_HEADERS: 'Authorization=Bearer traces-token',
+      OTEL_EXPORTER_OTLP_METRICS_HEADERS: 'Authorization=Bearer metrics-token',
+      OTEL_EXPORTER_OTLP_LOGS_HEADERS: 'Authorization=Bearer logs-token',
+      OTEL_RESOURCE_ATTRIBUTES: 'host.name=runner01',
+      OTEL_SDK_DISABLED: 'false',
+    };
+
+    let origVals: Record<string, string | undefined>;
+
+    beforeEach(() => {
+      origVals = {};
+      for (const key of Object.keys(otelVars)) {
+        origVals[key] = process.env[key];
+        process.env[key] = otelVars[key];
+      }
+    });
+
+    afterEach(() => {
+      for (const key of Object.keys(otelVars)) {
+        if (origVals[key] !== undefined) process.env[key] = origVals[key];
+        else delete process.env[key];
+      }
+    });
+
+    it('should auto-forward OTEL_* variables in default (non-env-all) mode', () => {
+      const result = generateDockerCompose(mockConfig, mockNetworkConfig);
+      const env = result.services.agent.environment as Record<string, string>;
+
+      expect(env.OTEL_SERVICE_NAME).toBe('my-service');
+      expect(env.OTEL_EXPORTER_OTLP_ENDPOINT).toBe('https://otel.example.com:4318');
+      expect(env.OTEL_EXPORTER_OTLP_HEADERS).toBe('Authorization=Bearer secret-token');
+      expect(env.OTEL_EXPORTER_OTLP_TRACES_HEADERS).toBe('Authorization=Bearer traces-token');
+      expect(env.OTEL_EXPORTER_OTLP_METRICS_HEADERS).toBe('Authorization=Bearer metrics-token');
+      expect(env.OTEL_EXPORTER_OTLP_LOGS_HEADERS).toBe('Authorization=Bearer logs-token');
+      expect(env.OTEL_RESOURCE_ATTRIBUTES).toBe('host.name=runner01');
+      expect(env.OTEL_SDK_DISABLED).toBe('false');
+    });
+
+    it('should not forward OTEL_* variables when not set in host environment', () => {
+      for (const key of Object.keys(otelVars)) {
+        delete process.env[key];
+      }
+      const result = generateDockerCompose(mockConfig, mockNetworkConfig);
+      const env = result.services.agent.environment as Record<string, string>;
+
+      for (const key of Object.keys(otelVars)) {
+        expect(env[key]).toBeUndefined();
+      }
+    });
+
+    it('should include OTEL header vars in AWF_ONE_SHOT_TOKENS', () => {
+      const result = generateDockerCompose(mockConfig, mockNetworkConfig);
+      const env = result.services.agent.environment as Record<string, string>;
+      const oneShot = env.AWF_ONE_SHOT_TOKENS ?? '';
+
+      expect(oneShot).toContain('OTEL_EXPORTER_OTLP_HEADERS');
+      expect(oneShot).toContain('OTEL_EXPORTER_OTLP_TRACES_HEADERS');
+      expect(oneShot).toContain('OTEL_EXPORTER_OTLP_METRICS_HEADERS');
+      expect(oneShot).toContain('OTEL_EXPORTER_OTLP_LOGS_HEADERS');
+    });
+  });
+
+  describe('COPILOT_OTEL_FILE_EXPORTER_PATH forwarding', () => {
+    it('should forward COPILOT_OTEL_FILE_EXPORTER_PATH when set', () => {
+      const original = process.env.COPILOT_OTEL_FILE_EXPORTER_PATH;
+      process.env.COPILOT_OTEL_FILE_EXPORTER_PATH = '/tmp/otel-spans.json';
+      try {
+        const result = generateDockerCompose(mockConfig, mockNetworkConfig);
+        const env = result.services.agent.environment as Record<string, string>;
+        expect(env.COPILOT_OTEL_FILE_EXPORTER_PATH).toBe('/tmp/otel-spans.json');
+      } finally {
+        if (original !== undefined) process.env.COPILOT_OTEL_FILE_EXPORTER_PATH = original;
+        else delete process.env.COPILOT_OTEL_FILE_EXPORTER_PATH;
+      }
+    });
+
+    it('should not set COPILOT_OTEL_FILE_EXPORTER_PATH when not in host environment', () => {
+      const original = process.env.COPILOT_OTEL_FILE_EXPORTER_PATH;
+      delete process.env.COPILOT_OTEL_FILE_EXPORTER_PATH;
+      try {
+        const result = generateDockerCompose(mockConfig, mockNetworkConfig);
+        const env = result.services.agent.environment as Record<string, string>;
+        expect(env.COPILOT_OTEL_FILE_EXPORTER_PATH).toBeUndefined();
+      } finally {
+        if (original !== undefined) process.env.COPILOT_OTEL_FILE_EXPORTER_PATH = original;
+      }
+    });
+  });
 });

src/services/agent-environment-credentials.test.ts

@@ -130,7 +130,7 @@ export function buildAgentEnvironment(params: AgentEnvironmentParams): Record<st
     }),
     // Configure one-shot-token library with sensitive tokens to protect
     // These tokens are cached on first access and unset from /proc/self/environ
-    AWF_ONE_SHOT_TOKENS: 'COPILOT_GITHUB_TOKEN,GITHUB_TOKEN,GH_TOKEN,GITHUB_API_TOKEN,GITHUB_PAT,GH_ACCESS_TOKEN,OPENAI_API_KEY,OPENAI_KEY,ANTHROPIC_API_KEY,CLAUDE_API_KEY,CODEX_API_KEY,COPILOT_API_KEY,COPILOT_PROVIDER_API_KEY',
+    AWF_ONE_SHOT_TOKENS: 'COPILOT_GITHUB_TOKEN,GITHUB_TOKEN,GH_TOKEN,GITHUB_API_TOKEN,GITHUB_PAT,GH_ACCESS_TOKEN,OPENAI_API_KEY,OPENAI_KEY,ANTHROPIC_API_KEY,CLAUDE_API_KEY,CODEX_API_KEY,COPILOT_API_KEY,COPILOT_PROVIDER_API_KEY,OTEL_EXPORTER_OTLP_HEADERS,OTEL_EXPORTER_OTLP_TRACES_HEADERS,OTEL_EXPORTER_OTLP_METRICS_HEADERS,OTEL_EXPORTER_OTLP_LOGS_HEADERS',
   };
 
   // Copilot CLI requires Node.js. Ask the agent entrypoint to fail fast with a
@@ -284,11 +284,33 @@ export function buildAgentEnvironment(params: AgentEnvironmentParams): Record<st
       'DOCKER_CONFIG',
       'DOCKER_API_VERSION',
       'DOCKER_DEFAULT_PLATFORM',
+      // Copilot OTEL file exporter path — written to a local file, no network needed.
+      // gh-aw uploads the resulting file as an Actions artifact.
+      'COPILOT_OTEL_FILE_EXPORTER_PATH',
     ] as const;
     for (const v of alwaysForwardVars) {
       if (process.env[v]) environment[v] = process.env[v]!;
     }
 
+    // Forward all OTEL_* environment variables — standardized prefix per the OpenTelemetry spec.
+    // This covers the full set of ~50+ OTEL_ variables (safe, network-affecting, and sensitive)
+    // without requiring users to pass --env-all or list each variable explicitly.
+    // Sensitive header vars (OTEL_EXPORTER_OTLP_HEADERS and per-signal variants) are also
+    // included in AWF_ONE_SHOT_TOKENS above, so they are cached on first access and removed
+    // from /proc/self/environ to prevent exfiltration by compromised subprocesses.
+    // EXCLUDED_ENV_VARS guards against leaking proxy/Actions/AWF internal vars (e.g., if a
+    // future OTEL_ variable overlaps); the hasOwnProperty check prevents --env-all or earlier
+    // alwaysForwardVars entries from being silently overwritten.
+    // Note: process.env values are typed as string | undefined, so the value check is a
+    // required TypeScript type guard (Object.entries won't include missing keys at runtime).
+    for (const [key, value] of Object.entries(process.env)) {
+      if (key.startsWith('OTEL_') && value !== undefined
+          && !EXCLUDED_ENV_VARS.has(key)
+          && !Object.prototype.hasOwnProperty.call(environment, key)) {
+        environment[key] = value;
+      }
+    }
+
     // When DinD is exposed via a Unix socket override, keep the agent's default docker
     // client target aligned with the socket mounted into /host. This intentionally
     // overrides any inherited DOCKER_HOST so --enable-dind + --docker-host cannot leave