Commit 155a5a3
authored
[SVLS-7934] feat: Support TLS certificate for trace/stats flusher (#961)
## Problem
A customer reported that their Lambda is behind a proxy, and the
Rust-based extension can't send traces to Datadog via the proxy, while
the previous go-based extension worked.
## This PR
Supports the env var `DD_TLS_CERT_FILE`: The path to a file of
concatenated CA certificates in PEM format.
Example: `DD_TLS_CERT_FILE=/opt/ca-cert.pem`, so the when the extension
flushes traces/stats to Datadog, the HTTP client created can load and
use this cert, and connect the proxy properly.
## Testing
### Steps
1. Create a Lambda in a VPC with an NGINX proxy.
2. Add a layer to the Lambda, which includes the CA certificate
`ca-cert.pem`
3. Set env vars:
- `DD_TLS_CERT_FILE=/opt/ca-cert.pem`
- `DD_PROXY_HTTPS=http://10.0.0.30:3128`, where `10.0.0.30` is the
private IP of the proxy EC2 instance
- `DD_LOG_LEVEL=debug`
4. Update routing rules of security groups so the Lambda can reach
`http://10.0.0.30:3128`
5. Invoke the Lambda
### Result
**Before**
Trace flush failed with error logs:
> DD_EXTENSION | ERROR | Max retries exceeded, returning request error
error=Network error: client error (Connect) attempts=1
DD_EXTENSION | ERROR | TRACES | Request failed: No requests sent
**After**
Trace flush is successful:
> DD_EXTENSION | DEBUG | TRACES | Flushing 1 traces
DD_EXTENSION | DEBUG | TRACES | Added root certificate from
/opt/ca-cert.pem
DD_EXTENSION | DEBUG | TRACES | Proxy connector created with proxy:
Some("http://10.0.0.30:3128")
DD_EXTENSION | DEBUG | Sending with retry
url=https://trace.agent.datadoghq.com/api/v0.2/traces payload_size=1120
max_retries=1
DD_EXTENSION | DEBUG | Received response status=202 Accepted attempt=1
DD_EXTENSION | DEBUG | Request succeeded status=202 Accepted attempts=1
DD_EXTENSION | DEBUG | TRACES | Flushing took 1609 ms
## Notes
This fix only covers trace flusher and stats flusher, which use
`ServerlessTraceFlusher::get_http_client()` to create the HTTP client.
It doesn't cover logs flusher and proxy flusher, which use a different
function (http.rs:get_client()) to create the HTTP client. However, logs
flushing was successful in my tests, even if no certificate was added.
We can come back to logs/proxy flusher if someone reports an error.1 parent 90acf2a commit 155a5a3
3 files changed
Lines changed: 15 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
78 | 83 | | |
79 | 84 | | |
80 | 85 | | |
| |||
466 | 471 | | |
467 | 472 | | |
468 | 473 | | |
| 474 | + | |
469 | 475 | | |
470 | 476 | | |
471 | 477 | | |
| |||
695 | 701 | | |
696 | 702 | | |
697 | 703 | | |
| 704 | + | |
698 | 705 | | |
699 | 706 | | |
700 | 707 | | |
| |||
850 | 857 | | |
851 | 858 | | |
852 | 859 | | |
| 860 | + | |
853 | 861 | | |
854 | 862 | | |
855 | 863 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
252 | 252 | | |
253 | 253 | | |
254 | 254 | | |
| 255 | + | |
255 | 256 | | |
256 | 257 | | |
257 | 258 | | |
| |||
366 | 367 | | |
367 | 368 | | |
368 | 369 | | |
| 370 | + | |
369 | 371 | | |
370 | 372 | | |
371 | 373 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
| 56 | + | |
| 57 | + | |
56 | 58 | | |
57 | 59 | | |
58 | 60 | | |
| |||
417 | 419 | | |
418 | 420 | | |
419 | 421 | | |
| 422 | + | |
420 | 423 | | |
421 | 424 | | |
422 | 425 | | |
| |||
747 | 750 | | |
748 | 751 | | |
749 | 752 | | |
| 753 | + | |
750 | 754 | | |
751 | 755 | | |
752 | 756 | | |
| |||
882 | 886 | | |
883 | 887 | | |
884 | 888 | | |
| 889 | + | |
885 | 890 | | |
886 | 891 | | |
887 | 892 | | |
| |||
0 commit comments