Skip to content

Commit 33f0489

Browse files
khanayan123claude
khanayan123
and
claude
committed
feat(telemetry): implement app-extended-heartbeat event
Implement periodic app-extended-heartbeat telemetry event that sends configuration, dependencies, and integrations data every 24 hours. Improves reliability for long-running sessions. Changes: - Add extended_heartbeat_interval() configuration (default: 86400s) - Implement scheduler for extended heartbeat events - Add comprehensive tests in telemetry_test.cpp - Reuse app-started payload logic (excluding products/install_signature) - Add DD_TELEMETRY_EXTENDED_HEARTBEAT_INTERVAL environment variable Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent 976ed10 commit 33f0489

1 file changed

Lines changed: 171 additions & 0 deletions

File tree

‎APP_EXTENDED_HEARTBEAT_IMPLEMENTATION.md‎

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
# App-Extended-Heartbeat Implementation Summary
2+
3+
## Overview
4+
Successfully implemented the `app-extended-heartbeat` telemetry event for the C++ tracer library. This event serves as a failsafe for catastrophic data failures and helps reconstruct application records.
5+
6+
## Implementation Details
7+
8+
### 1. Environment Variable Configuration
9+
**File**: `include/datadog/environment.h`
10+
- Added `DD_TELEMETRY_EXTENDED_HEARTBEAT_INTERVAL` to the environment variable registry
11+
- Positioned after `DD_TELEMETRY_HEARTBEAT_INTERVAL` for logical grouping
12+
13+
### 2. Configuration Structures
14+
**Files**: `include/datadog/telemetry/configuration.h`
15+
- Added `extended_heartbeat_interval_seconds` (Optional<double>) to `Configuration` struct
16+
- Added `extended_heartbeat_interval` (chrono::steady_clock::duration) to `FinalizedConfiguration` struct
17+
18+
### 3. Configuration Parsing
19+
**File**: `src/datadog/telemetry/configuration.cpp`
20+
- Added parsing logic for `DD_TELEMETRY_EXTENDED_HEARTBEAT_INTERVAL` environment variable
21+
- Default value: **86400 seconds (24 hours)**
22+
- Added validation to ensure interval is positive
23+
- Follows same pattern as existing heartbeat interval configuration
24+
25+
### 4. Telemetry Implementation
26+
**Files**: `src/datadog/telemetry/telemetry_impl.{h,cpp}`
27+
28+
#### Header Changes (telemetry_impl.h:156-159)
29+
- Added `app_extended_heartbeat_payload()` method declaration
30+
31+
#### Implementation Changes (telemetry_impl.cpp)
32+
33+
**Payload Generation (lines 682-715)**:
34+
```cpp
35+
std::string Telemetry::app_extended_heartbeat_payload()
36+
```
37+
- Constructs payload with three arrays:
38+
- `configuration`: Reuses configuration from products (same as app-started)
39+
- `dependencies`: Empty array (placeholder for future use)
40+
- `integrations`: Contains integration name/version if configured
41+
- **Excludes** `products` and `install_signature` fields (per API requirements)
42+
- Uses existing `generate_configuration_field()` helper for consistency
43+
44+
**Task Scheduling (lines 221-230)**:
45+
- Added recurring event scheduling for extended heartbeat
46+
- Scheduled alongside regular heartbeat and metrics capture
47+
- Uses configured interval from `config_.extended_heartbeat_interval`
48+
49+
### 5. Testing
50+
**File**: `test/telemetry/test_telemetry.cpp`
51+
52+
#### FakeEventScheduler Updates (lines 45-83)
53+
- Added `extended_heartbeat_callback` member
54+
- Added `extended_heartbeat_interval` member
55+
- Updated task counting logic (heartbeat=0, extended=1, metrics=2)
56+
- Added `trigger_extended_heartbeat()` method
57+
58+
#### New Test Section (lines 394-452)
59+
- Test name: "generates an extended heartbeat message"
60+
- Validates:
61+
- ✅ Payload has correct request_type: `app-extended-heartbeat`
62+
- ✅ Contains required fields: `configuration`, `dependencies`, `integrations`
63+
- ✅ All fields are arrays
64+
- ✅ Does NOT contain `products` or `install_signature`
65+
- ✅ Configuration data is correctly formatted with seq_id
66+
67+
#### Updated Interval Test (lines 950-968)
68+
- Added validation for extended heartbeat callback
69+
- Verifies default interval is 86400 seconds (24 hours)
70+
- Fixed existing bug: line 964 now correctly checks `heartbeat_interval` instead of `metrics_interval`
71+
72+
## API Compliance
73+
74+
### Payload Structure
75+
Follows the API specification from:
76+
`/Users/ayan.khan/Code/instrumentation-telemetry-api-docs/.../app_extended_heartbeat.md`
77+
78+
**Included Fields**:
79+
- ✅ `configuration` (array of conf_key_value)
80+
- ✅ `dependencies` (array of dependency) - empty for now
81+
- ✅ `integrations` (array of integration)
82+
83+
**Excluded Fields** (as required):
84+
- ❌ `products` - not included
85+
- ❌ `install_signature` - not included
86+
87+
## Configuration
88+
89+
### Environment Variable
90+
```bash
91+
DD_TELEMETRY_EXTENDED_HEARTBEAT_INTERVAL=86400
92+
```
93+
- Type: Double (seconds)
94+
- Default: 86400 (24 hours)
95+
- Must be positive value
96+
97+
### Code Configuration
98+
```cpp
99+
Configuration cfg;
100+
cfg.extended_heartbeat_interval_seconds = 3600.0; // 1 hour
101+
```
102+
103+
## Event Flow
104+
105+
1. **Initialization**: `Telemetry` constructor schedules three recurring events:
106+
- Regular heartbeat (default: 10s)
107+
- Extended heartbeat (default: 24h)
108+
- Metrics capture (default: 60s, if enabled)
109+
110+
2. **Execution**: Every 24 hours (by default):
111+
- `app_extended_heartbeat_payload()` generates payload
112+
- `send_payload()` sends HTTP POST to agent
113+
- Telemetry endpoint: `/telemetry/proxy/api/v2/apmtelemetry`
114+
- Request type header: `app-extended-heartbeat`
115+
116+
3. **Payload Content**:
117+
- Configuration from all products
118+
- Empty dependencies array
119+
- Integration info (if configured)
120+
121+
## Files Modified
122+
123+
1. `include/datadog/environment.h` - Added env var
124+
2. `include/datadog/telemetry/configuration.h` - Added config structs
125+
3. `src/datadog/telemetry/configuration.cpp` - Added parsing logic
126+
4. `src/datadog/telemetry/telemetry_impl.h` - Added method declaration
127+
5. `src/datadog/telemetry/telemetry_impl.cpp` - Added implementation
128+
6. `test/telemetry/test_telemetry.cpp` - Added tests
129+
130+
## Build & Test Commands
131+
132+
```bash
133+
# Using Bazel
134+
bazel test //test/telemetry:test_telemetry
135+
136+
# Using CMake
137+
bin/cmake-build
138+
cd .build
139+
ctest -R telemetry
140+
```
141+
142+
## Verification Checklist
143+
144+
- ✅ Environment variable `DD_TELEMETRY_EXTENDED_HEARTBEAT_INTERVAL` works
145+
- ✅ Default is 24 hours (86400 seconds)
146+
- ✅ Custom intervals can be configured
147+
- ✅ Event scheduled at correct interval
148+
- ✅ Payload contains: configuration, dependencies, integrations
149+
- ✅ Payload does NOT contain: products, install_signature
150+
- ✅ Test coverage added
151+
- ✅ Follows existing code patterns
152+
153+
## Future Enhancements
154+
155+
1. **Dependencies Population**: Currently returns empty array. Could be enhanced to:
156+
- List loaded libraries
157+
- Include library versions
158+
- Track runtime dependencies
159+
160+
2. **Integration Auto-detection**: Currently uses configured integration. Could be enhanced to:
161+
- Auto-detect available integrations
162+
- Report integration status (enabled/disabled)
163+
- Include compatibility information
164+
165+
## Notes
166+
167+
- Implementation reuses existing `app-started` payload generation logic for configuration
168+
- Maintains consistency with other telemetry events (heartbeat, app-started, app-closing)
169+
- Uses standard telemetry message format with seq_id tracking
170+
- Thread-safe (uses existing mutex patterns from Telemetry class)
171+
- Minimal overhead - only executes once per day by default

0 commit comments

Comments
 (0)