Skip to content

Commit 2bf81c8

Browse files
majormajor
committed
docs: add Sentry error tracking integration page
Signed-off-by: Major Hayden <major@redhat.com>
1 parent 0c9ae90 commit 2bf81c8

2 files changed

Lines changed: 164 additions & 0 deletions

File tree

‎docs/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,8 @@ See the full documentation at [`../README.md`](../README.md) or browse sub-pages
4747

4848
[Providers](https://lightspeed-core.github.io/lightspeed-stack/providers.html)
4949

50+
[Sentry error tracking](https://lightspeed-core.github.io/lightspeed-stack/sentry.html)
51+
5052
[User data collection](https://lightspeed-core.github.io/lightspeed-stack/user_data_collection.html)
5153

5254
[Database structure](https://lightspeed-core.github.io/lightspeed-stack/DB/index.html)

‎docs/sentry.md‎

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
# Sentry Error Tracking Integration
2+
3+
Sentry integration is **completely optional** and **disabled by default**. Most deployments will not use it. The service starts and runs normally with no Sentry configuration present.
4+
5+
When enabled, Sentry captures unhandled exceptions and performance traces from the FastAPI application and sends them to your Sentry project for monitoring and alerting.
6+
7+
## Overview
8+
9+
Enabling Sentry gives you:
10+
11+
- **Error tracking** - unhandled exceptions are captured and reported with full stack traces
12+
- **Performance tracing** - a sample of POST requests are traced end-to-end
13+
- **Release tagging** - errors are tagged with the running service version for easier triage
14+
15+
The integration is initialized at service startup and flushes any pending events on shutdown (2-second timeout).
16+
17+
## Configuration
18+
19+
Sentry is configured entirely through environment variables. No changes to `lightspeed-stack.yaml` are required.
20+
21+
### Environment Variables
22+
23+
| Variable | Required | Default | Description |
24+
|----------|----------|---------|-------------|
25+
| `SENTRY_DSN` | Yes (to enable) | - | Data Source Name from your Sentry project. Setting this variable enables Sentry. The DSN value is never written to logs to prevent credential exposure. |
26+
| `SENTRY_ENVIRONMENT` | No | `development` | Environment tag attached to all events. Set this explicitly in production deployments. Use values like `production`, `stage`, or `dev` to distinguish clusters or deployment stages. |
27+
| `SENTRY_CA_CERTS` | No | - | Path to a CA certificate bundle file. Only needed when your Sentry instance uses a private or internal CA. If the file is missing at startup, the SDK proceeds without custom certificates and logs a warning. |
28+
29+
### Enabling Sentry
30+
31+
Set `SENTRY_DSN` to the DSN string from your Sentry project settings:
32+
33+
```bash
34+
export SENTRY_DSN="https://examplePublicKey@o0.ingest.sentry.io/0"
35+
```
36+
37+
The service logs `Sentry initialized` on startup when the DSN is present, or `Sentry DSN not configured, skipping initialization` when it is not.
38+
39+
## Behavior Details
40+
41+
### FastAPI Integration
42+
43+
The Sentry FastAPI integration is configured to capture only `POST` requests. `GET` requests (health checks, model listings, etc.) are not traced.
44+
45+
### Trace Sampling
46+
47+
Of the captured POST requests, the following routes are always excluded from tracing regardless of the sample rate:
48+
49+
- `/readiness`
50+
- `/liveness`
51+
- `/metrics`
52+
- `/` (root)
53+
54+
The remaining eligible requests are sampled at 25% for performance tracing. This keeps trace volume low and avoids noise from health check and metrics scrape traffic.
55+
56+
### Privacy
57+
58+
`send_default_pii` is set to `False`. Sentry will not attach user IP addresses, HTTP headers, or other personally identifiable information to events.
59+
60+
### Release Tagging
61+
62+
Every event is tagged with the running service version in the format `lightspeed-stack@{version}` (for example, `lightspeed-stack@0.5.0`). This makes it straightforward to correlate errors with specific releases in the Sentry UI.
63+
64+
### Shutdown Behavior
65+
66+
When the service shuts down, it flushes any buffered Sentry events before exiting. The flush has a 2-second timeout to avoid delaying shutdown.
67+
68+
## OpenShift Deployment
69+
70+
### Setting the DSN via a Secret
71+
72+
Store the Sentry DSN in an OpenShift Secret rather than hardcoding it in a Deployment manifest:
73+
74+
```bash
75+
oc create secret generic sentry-credentials \
76+
--from-literal=dsn="https://examplePublicKey@o0.ingest.sentry.io/0"
77+
```
78+
79+
Reference the Secret in your Deployment or Pod spec:
80+
81+
```yaml
82+
env:
83+
- name: SENTRY_DSN
84+
valueFrom:
85+
secretKeyRef:
86+
name: sentry-credentials
87+
key: dsn
88+
```
89+
90+
### Setting the Environment Tag
91+
92+
Use `SENTRY_ENVIRONMENT` to label events by cluster or deployment stage. This makes it easy to filter events in the Sentry UI:
93+
94+
```yaml
95+
env:
96+
- name: SENTRY_DSN
97+
valueFrom:
98+
secretKeyRef:
99+
name: sentry-credentials
100+
key: dsn
101+
- name: SENTRY_ENVIRONMENT
102+
value: "production"
103+
```
104+
105+
Set this to `stage`, `dev`, or any label that matches your deployment topology.
106+
107+
### Private CA Certificates (Enterprise Sentry Instances)
108+
109+
Most deployers will not need this. If your Sentry instance is hosted internally and uses a certificate signed by a private CA, the SDK will fail to connect without the CA bundle.
110+
111+
Mount the CA bundle into the pod using a ConfigMap or Secret, then point `SENTRY_CA_CERTS` at the mount path.
112+
113+
**Using a ConfigMap:**
114+
115+
```bash
116+
oc create configmap sentry-ca --from-file=ca-bundle.crt=/path/to/your/ca-bundle.crt
117+
```
118+
119+
```yaml
120+
volumes:
121+
- name: sentry-ca
122+
configMap:
123+
name: sentry-ca
124+
125+
containers:
126+
- name: lightspeed-stack
127+
volumeMounts:
128+
- name: sentry-ca
129+
mountPath: /etc/sentry-ca
130+
readOnly: true
131+
env:
132+
- name: SENTRY_DSN
133+
valueFrom:
134+
secretKeyRef:
135+
name: sentry-credentials
136+
key: dsn
137+
- name: SENTRY_CA_CERTS
138+
value: "/etc/sentry-ca/ca-bundle.crt"
139+
```
140+
141+
If the file is not present at the path specified by `SENTRY_CA_CERTS`, the service logs a warning and continues without custom CA certificates. It will not fail to start.
142+
143+
## Troubleshooting
144+
145+
### Events Not Appearing in Sentry
146+
147+
1. Confirm `SENTRY_DSN` is set and the value is correct. Check the service logs for `Sentry initialized` at startup.
148+
2. Verify network connectivity from the pod to the Sentry ingest endpoint. DNS resolution failures and firewall rules are common causes.
149+
3. If using a private Sentry instance, confirm `SENTRY_CA_CERTS` points to a valid CA bundle and the file is readable by the service process.
150+
4. Check that the DSN belongs to the correct Sentry project and organization.
151+
152+
### Warning: CA Cert File Not Found
153+
154+
```text
155+
CA cert file specified by SENTRY_CA_CERTS not found at /etc/sentry-ca/ca-bundle.crt; proceeding without custom CA certs
156+
```
157+
158+
The path set in `SENTRY_CA_CERTS` does not exist. Verify the ConfigMap or Secret is mounted correctly and the mount path matches the environment variable value.
159+
160+
### Events Appear in Wrong Environment
161+
162+
Check the value of `SENTRY_ENVIRONMENT`. If it is not set, events are tagged as `development` by default. Set the variable explicitly to match your deployment stage.

0 commit comments

Comments
 (0)