docs: OpenTelemetry (#7103)

khvn26 · web-flow · commit 7beff52a1b75 · 2026-04-06T15:43:05.000+01:00
diff --git a/docs/docs/deployment-self-hosting/core-configuration/environment-variables.md b/docs/docs/deployment-self-hosting/core-configuration/environment-variables.md
@@ -1,10 +1,14 @@
 ---
-title: "Environment Variables"
-description: "A reference page listing all environment variables for the API and Frontend."
+title: 'Environment Variables'
+description: 'A reference page listing all environment variables for the API and Frontend.'
 sidebar_position: 20
 ---
 
-This page provides a comprehensive reference for all the environment variables you can use to configure your self-hosted Flagsmith instance. You'll find variables for both the API and the frontend, along with a brief description of what each one does. Use this as a handy guide when setting up or tweaking your deployment, whether you're running locally, in the cloud, or on Kubernetes. If you're not sure what a particular variable does, or whether you need to set it, check the relevant section below for more details.
+This page provides a comprehensive reference for all the environment variables you can use to configure your self-hosted
+Flagsmith instance. You'll find variables for both the API and the frontend, along with a brief description of what each
+one does. Use this as a handy guide when setting up or tweaking your deployment, whether you're running locally, in the
+cloud, or on Kubernetes. If you're not sure what a particular variable does, or whether you need to set it, check the
+relevant section below for more details.
 
 ## API Environment Variables
 
@@ -13,41 +17,66 @@ This page provides a comprehensive reference for all the environment variables y
 - `REDIS_URL`: The URL of your Redis instance.
 - `ENV`: The environment the application is running in, e.g. "prod".
 - `SENTRY_DSN`: If you want to send errors to Sentry, specify the DSN here.
-- `SENTRY_TRACE_SAMPLE_RATE`: The percentage of transactions to trace in Sentry. See [Sentry's documentation](https://docs.sentry.io/platforms/python/performance/instrumentation/django/#configure) for more info.
+- `SENTRY_TRACE_SAMPLE_RATE`: The percentage of transactions to trace in Sentry. See
+  [Sentry's documentation](https://docs.sentry.io/platforms/python/performance/instrumentation/django/#configure) for
+  more info.
 - `LOG_LEVEL`: The log level to output at. One of `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`.
 - `LOG_FORMAT`: The format to use for logging. One of `generic` or `json`.
 - `DJANGO_SECRET_KEY`: A long, random and unique string used for cryptographic signing.
-- `ACCESS_LOG_LOCATION`: The location to store web logs generated by Gunicorn if running as a Docker container. If not set, no logs will be stored. If set to `-`, the logs will be sent to `stdout`.
+- `ACCESS_LOG_LOCATION`: The location to store web logs generated by Gunicorn if running as a Docker container. If not
+  set, no logs will be stored. If set to `-`, the logs will be sent to `stdout`.
 - `DJANGO_SETTINGS_MODULE`: Python path to settings file for the given environment, e.g. "app.settings.develop"
-- `ALLOW_ADMIN_INITIATION_VIA_CLI`: Enables the `bootstrap` management command which creates default admin user, organisation, and project.
+- `ALLOW_ADMIN_INITIATION_VIA_CLI`: Enables the `bootstrap` management command which creates default admin user,
+  organisation, and project.
 - `ADMIN_EMAIL`: Email to use for the default superuser creation.
 - `ORGANISATION_NAME`: Organisation name to use for the default organisation.
 - `PROJECT_NAME`: Project name to use for the default project.
 - `ENABLE_GZIP_COMPRESSION`: If Django should gzip compress HTTP responses. Defaults to `False`.
 - `GOOGLE_ANALYTICS_KEY`: If Google Analytics is required, add your tracking code.
-- `GOOGLE_SERVICE_ACCOUNT`: Service account JSON for accessing the Google API, used for getting usage of an organisation - needs access to analytics.readonly scope.
+- `GOOGLE_SERVICE_ACCOUNT`: Service account JSON for accessing the Google API, used for getting usage of an
+  organisation - needs access to analytics.readonly scope.
 - `INFLUXDB_TOKEN`: If you want to send API events to InfluxDB, specify this write token.
 - `INFLUXDB_URL`: The URL for your InfluxDB database.
 - `INFLUXDB_ORG`: The organisation string for your InfluxDB API call.
 - `GA_TABLE_ID`: GA table ID (view) to query when looking for organisation usage.
-- `USER_CREATE_PERMISSIONS`: Set the permissions for creating new users, using a comma-separated list of djoser or rest_framework permissions. Use this to turn off public user creation for self-hosting. e.g. `'djoser.permissions.CurrentUserOrAdmin'`. Defaults to `'rest_framework.permissions.AllowAny'`.
-- `ALLOW_REGISTRATION_WITHOUT_INVITE`: Determines whether users can register without an invite. Defaults to True. Set to False or 0 to disable. Note that if disabled, new users must be invited via email.
+- `USER_CREATE_PERMISSIONS`: Set the permissions for creating new users, using a comma-separated list of djoser or
+  rest_framework permissions. Use this to turn off public user creation for self-hosting. e.g.
+  `'djoser.permissions.CurrentUserOrAdmin'`. Defaults to `'rest_framework.permissions.AllowAny'`.
+- `ALLOW_REGISTRATION_WITHOUT_INVITE`: Determines whether users can register without an invite. Defaults to True. Set to
+  False or 0 to disable. Note that if disabled, new users must be invited via email.
 - `PREVENT_SIGNUP`: Determines whether to prevent new signups.
 - `ENABLE_EMAIL_ACTIVATION`: New user registration will go via email activation flow, default False.
+- `OTEL_EXPORTER_OTLP_ENDPOINT`: Base OTLP/HTTP endpoint (e.g. `http://collector:4318`). If unset, OTel is disabled. See
+  [OpenTelemetry](/deployment-self-hosting/scaling-and-performance/opentelemetry).
+- `OTEL_SERVICE_NAME`: The `service.name` resource attribute attached to all OTel telemetry. Defaults to
+  `flagsmith-api`.
+- `OTEL_TRACING_EXCLUDED_URL_PATHS`: Comma-separated URL paths to exclude from OTel tracing (e.g.
+  `health/liveness,health/readiness`).
 
 ## Frontend Environment Variables
 
 - `FLAGSMITH_API_URL`: The API to hit for requests. E.g. `https://edge.api.flagsmith.com/api/v1/`
-- `FLAGSMITH_ON_FLAGSMITH_API_KEY`: The Flagsmith environment key we use to manage features - [Flagsmith runs on Flagsmith](/deployment-self-hosting/core-configuration/running-flagsmith-on-flagsmith).
-- `FLAGSMITH_ON_FLAGSMITH_API_URL`: The API URL which the Flagsmith client should communicate with. Flagsmith runs on Flagsmith. E.g. `https://edge.api.flagsmith.com/api/v1/`. If you are self-hosting and using your own Flagsmith instance to manage its own features, you would generally point this to the same domain name as your own Flagsmith instance.
-- `DISABLE_ANALYTICS_FEATURES`: Disables any in-app analytics-related features: API Usage charts, flag analytics. E.g. `DISABLE_ANALYTICS_FEATURES=1`.
-- `ENABLE_FLAG_EVALUATION_ANALYTICS`: Determines if the Flagsmith SDK should send usage analytics. If you want to enable Flag Analytics, set this. E.g. `ENABLE_FLAG_EVALUATION_ANALYTICS=1`.
-- `PROXY_API_URL`: Proxies the API via this application. Set this to the hostname of the API being proxied. Proxies `/api/v1/` through to `PROXY_API_URL`. If you are using this, any setting to `FLAGSMITH_API_URL` will be ignored and the browser will use the frontend node server to send API requests. Do not prepend `api/v1/` - it will be added automatically.
+- `FLAGSMITH_ON_FLAGSMITH_API_KEY`: The Flagsmith environment key we use to manage features -
+  [Flagsmith runs on Flagsmith](/deployment-self-hosting/core-configuration/running-flagsmith-on-flagsmith).
+- `FLAGSMITH_ON_FLAGSMITH_API_URL`: The API URL which the Flagsmith client should communicate with. Flagsmith runs on
+  Flagsmith. E.g. `https://edge.api.flagsmith.com/api/v1/`. If you are self-hosting and using your own Flagsmith
+  instance to manage its own features, you would generally point this to the same domain name as your own Flagsmith
+  instance.
+- `DISABLE_ANALYTICS_FEATURES`: Disables any in-app analytics-related features: API Usage charts, flag analytics. E.g.
+  `DISABLE_ANALYTICS_FEATURES=1`.
+- `ENABLE_FLAG_EVALUATION_ANALYTICS`: Determines if the Flagsmith SDK should send usage analytics. If you want to enable
+  Flag Analytics, set this. E.g. `ENABLE_FLAG_EVALUATION_ANALYTICS=1`.
+- `PROXY_API_URL`: Proxies the API via this application. Set this to the hostname of the API being proxied. Proxies
+  `/api/v1/` through to `PROXY_API_URL`. If you are using this, any setting to `FLAGSMITH_API_URL` will be ignored and
+  the browser will use the frontend node server to send API requests. Do not prepend `api/v1/` - it will be added
+  automatically.
 - `GOOGLE_ANALYTICS_API_KEY`: Google Analytics key to track API usage.
 - `CRISP_WEBSITE_ID`: Crisp Chat widget Website key.
 - `FIRST_PROMOTER_ID`: First Promoter ID for checkout affiliates.
-- `ALLOW_SIGNUPS`: **DEPRECATED** in favour of `PREVENT_SIGNUP` in the API. Determines whether to prevent manual signups without invites. Set it to any value to allow signups.
-- `PREVENT_FORGOT_PASSWORD`: Determines whether to prevent forgot password functionality, useful for LDAP/SAML. Set it to any value to prevent forgot password functionality.
+- `ALLOW_SIGNUPS`: **DEPRECATED** in favour of `PREVENT_SIGNUP` in the API. Determines whether to prevent manual signups
+  without invites. Set it to any value to allow signups.
+- `PREVENT_FORGOT_PASSWORD`: Determines whether to prevent forgot password functionality, useful for LDAP/SAML. Set it
+  to any value to prevent forgot password functionality.
 - `PREVENT_EMAIL_PASSWORD`: Disables email address signup, login and change email functionality.
 - `ENABLE_MAINTENANCE_MODE`: Puts the site into maintenance mode. Set it to any value to enable maintenance.
 - `AMPLITUDE_API_KEY`: The Amplitude key to use for behaviour tracking.
@@ -56,15 +85,23 @@ This page provides a comprehensive reference for all the environment variables y
 - `SENTRY_API_KEY`: Sentry key for error reporting.
 - `ALBACROSS_CLIENT_ID`: Albacross client ID key for behaviour tracking.
 - `BASE_URL`: Used for specifying a base URL path that's ignored during routing if serving from a subdirectory.
-- `USE_SECURE_COOKIES`: Enable/disable the use of secure cookies. If deploying the frontend in a private network without a domain/SSL cert, disable secure cookies to ensure that session token is persisted. Default: true.
-- `COOKIE_SAME_SITE`: Define the value of the SameSite attribute for the session token cookie set by the frontend. Further reading on this value is available [here](https://web.dev/articles/samesite-cookies-explained). Default: 'none'.
+- `USE_SECURE_COOKIES`: Enable/disable the use of secure cookies. If deploying the frontend in a private network without
+  a domain/SSL cert, disable secure cookies to ensure that session token is persisted. Default: true.
+- `COOKIE_SAME_SITE`: Define the value of the SameSite attribute for the session token cookie set by the frontend.
+  Further reading on this value is available [here](https://web.dev/articles/samesite-cookies-explained). Default:
+  'none'.
 
 ## OAuth configuration
 
 ### Google OAuth {#oauth-google}
 
-To configure Google OAuth, ensure you have set the relevant OAuth client credentials in your deployment platform, and created the **Flagsmith on Flagsmith** flag as described in the [OAuth guide](/administration-and-security/access-control/oauth). This flag controls whether Google SSO is available in your instance.
+To configure Google OAuth, ensure you have set the relevant OAuth client credentials in your deployment platform, and
+created the **Flagsmith on Flagsmith** flag as described in the
+[OAuth guide](/administration-and-security/access-control/oauth). This flag controls whether Google SSO is available in
+your instance.
 
 ### GitHub OAuth {#oauth-github}
 
-To configure GitHub OAuth, ensure you have set `GITHUB_CLIENT_ID` and `GITHUB_CLIENT_SECRET` in your environment, and created the **Flagsmith on Flagsmith** flag as described in the [OAuth guide](/administration-and-security/access-control/oauth). This enables the GitHub SSO option in your instance.
+To configure GitHub OAuth, ensure you have set `GITHUB_CLIENT_ID` and `GITHUB_CLIENT_SECRET` in your environment, and
+created the **Flagsmith on Flagsmith** flag as described in the
+[OAuth guide](/administration-and-security/access-control/oauth). This enables the GitHub SSO option in your instance.
diff --git a/docs/docs/deployment-self-hosting/scaling-and-performance/opentelemetry.md b/docs/docs/deployment-self-hosting/scaling-and-performance/opentelemetry.md
@@ -0,0 +1,104 @@
+---
+title: OpenTelemetry
+description: Exporting traces and structured logs from Flagsmith using OpenTelemetry.
+---
+
+Flagsmith supports exporting distributed traces and structured logs over
+[OTLP](https://opentelemetry.io/docs/specs/otel/protocol/), the OpenTelemetry Protocol. This lets you send observability
+data to any OTLP-compatible backend (e.g. SigNoz, Grafana Tempo, Jaeger, Datadog) without vendor lock-in.
+
+OTel instrumentation is opt-in and has no runtime overhead when disabled.
+
+## Configuration
+
+Set `OTEL_EXPORTER_OTLP_ENDPOINT` to the base OTLP/HTTP URL of your collector (e.g. `http://collector:4318`) to enable
+OTel export. When unset, no OTel code is loaded and there is no runtime overhead.
+
+Standard `OTEL_*` environment variables are also respected by the underlying OTel SDK. For example:
+
+- `OTEL_SERVICE_NAME` — overrides the `service.name` resource attribute (defaults to `flagsmith-api` or
+  `flagsmith-task-processor`).
+- `OTEL_RESOURCE_ATTRIBUTES` — adds custom resource attributes.
+- `OTEL_EXPORTER_OTLP_HEADERS` — sets authentication headers for the OTLP exporter.
+
+### Example: Docker Compose
+
+```yaml
+api:
+ environment:
+  OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-collector:4318
+```
+
+## Traces
+
+When enabled, Flagsmith produces distributed traces for every HTTP request.
+
+### What gets instrumented
+
+| Layer            | Instrumentation               | Span example                               |
+| ---------------- | ----------------------------- | ------------------------------------------ |
+| HTTP requests    | Django auto-instrumentation   | `GET /api/v1/organisations/{pk}/`          |
+| Database queries | psycopg2 auto-instrumentation | `SELECT` (with full SQL in `db.statement`) |
+| Redis commands   | Redis auto-instrumentation    | `GET` / `SET` / `EVAL`                     |
+
+Each HTTP request creates a root server span with child client spans for every database query and Redis command
+executed during the request.
+
+### Span naming
+
+HTTP spans are named `{METHOD} {route_template}`, where the route template matches the path format used in the OpenAPI
+spec:
+
+- `POST /api/v1/environments/{environment_api_key}/featurestates/`
+- `GET /api/v1/projects/{project_pk}/features/`
+
+The `http.route` attribute is set to the same normalised template.
+
+### Trace context propagation
+
+Flagsmith propagates [W3C TraceContext](https://www.w3.org/TR/trace-context/) and
+[W3C Baggage](https://www.w3.org/TR/baggage/) headers. If an upstream service sends a `traceparent` header, Flagsmith's
+spans join the existing trace. Baggage entries are forwarded into structlog attributes (see below).
+
+### SQL commenter
+
+The psycopg2 instrumentor has [SQL commenter](https://google.github.io/sqlcommenter/) enabled. This appends trace
+context as a SQL comment to every query sent to PostgreSQL:
+
+```sql
+SELECT "organisations_organisation"."id" FROM "organisations_organisation"
+/*traceparent='00-abc123...-def456...-01'*/
+```
+
+This allows correlating slow queries in PostgreSQL logs or `pg_stat_statements` back to the originating trace, without
+any changes to PostgreSQL configuration.
+
+### Excluding paths
+
+Health check endpoints and other high-frequency, low-value paths can be excluded from tracing:
+
+```
+OTEL_TRACING_EXCLUDED_URL_PATHS=health/liveness,health/readiness
+```
+
+Excluded paths produce no spans at all.
+
+## Structured logs
+
+Flagsmith uses [structlog](https://www.structlog.org/) for application logging. When OTel is enabled, structlog events
+are exported as OTLP log records. The `flagsmith.event` attribute duplicates the event name for backends that don't surface OTel's `EventName` field.
+
+### Span events
+
+When a structlog event is emitted inside an active span (e.g. during an HTTP request), it is also attached as a
+[span event](https://opentelemetry.io/docs/concepts/signals/traces/#span-events) with the same name and attributes. This
+makes application log events visible in trace detail views without requiring separate log correlation.
+
+### Console output
+
+When OTel is enabled, structlog events also include `trace_id` and `span_id` fields in their console/JSON output when an
+active span exists:
+
+```
+2025-03-31T15:34:32Z [info] list_organisations [organisations] trace_id=aabb... span_id=ccdd... user_id=1
+```
