feat(sca): runtime SCA reachability#17156
feat(sca): runtime SCA reachability#17156gh-worker-dd-mergequeue-cf854d[bot] merged 39 commits intomainfrom
Conversation
Codeowners resolved as |
Performance SLOsComparing candidate avara1986/sca-telemetry-rereporting-and-module (f4dd6de) with baseline main (a5ca84b) 📈 Performance Regressions (3 suites)📈 forktime - 4/4✅ baselineTime: ✅ 1.933ms (SLO: <3.000ms 📉 -35.6%) vs baseline: +4.9% Memory: ✅ 29.216MB (SLO: <33.000MB 📉 -11.5%) vs baseline: +4.8% ✅ configuredTime: ✅ 10.209ms (SLO: <17.000ms 📉 -39.9%) vs baseline: 📈 +11.8% Memory: ✅ 58.453MB (SLO: <60.000MB -2.6%) vs baseline: +5.5% 📈 iastaspects - 118/118✅ add_aspectTime: ✅ 103.156µs (SLO: <130.000µs 📉 -20.6%) vs baseline: +1.3% Memory: ✅ 43.970MB (SLO: <46.000MB -4.4%) vs baseline: +5.4% ✅ add_inplace_aspectTime: ✅ 100.502µs (SLO: <130.000µs 📉 -22.7%) vs baseline: -4.7% Memory: ✅ 43.884MB (SLO: <46.000MB -4.6%) vs baseline: +5.5% ✅ add_inplace_noaspectTime: ✅ 28.256µs (SLO: <40.000µs 📉 -29.4%) vs baseline: +0.3% Memory: ✅ 43.935MB (SLO: <46.000MB -4.5%) vs baseline: +5.5% ✅ add_noaspectTime: ✅ 49.422µs (SLO: <70.000µs 📉 -29.4%) vs baseline: +0.7% Memory: ✅ 43.890MB (SLO: <46.000MB -4.6%) vs baseline: +5.1% ✅ bytearray_aspectTime: ✅ 250.241µs (SLO: <400.000µs 📉 -37.4%) vs baseline: -9.0% Memory: ✅ 43.904MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ bytearray_extend_aspectTime: ✅ 646.268µs (SLO: <800.000µs 📉 -19.2%) vs baseline: -2.4% Memory: ✅ 43.906MB (SLO: <46.000MB -4.6%) vs baseline: +5.5% ✅ bytearray_extend_noaspectTime: ✅ 265.667µs (SLO: <400.000µs 📉 -33.6%) vs baseline: -0.3% Memory: ✅ 43.868MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ bytearray_noaspectTime: ✅ 139.531µs (SLO: <300.000µs 📉 -53.5%) vs baseline: -3.1% Memory: ✅ 43.894MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ bytes_aspectTime: ✅ 219.188µs (SLO: <300.000µs 📉 -26.9%) vs baseline: -6.7% Memory: ✅ 43.923MB (SLO: <46.000MB -4.5%) vs baseline: +5.1% ✅ bytes_noaspectTime: ✅ 136.229µs (SLO: <200.000µs 📉 -31.9%) vs baseline: +0.3% Memory: ✅ 43.989MB (SLO: <46.000MB -4.4%) vs baseline: +5.5% ✅ bytesio_aspectTime: ✅ 3.787ms (SLO: <5.000ms 📉 -24.3%) vs baseline: -1.6% Memory: ✅ 43.885MB (SLO: <46.000MB -4.6%) vs baseline: +5.1% ✅ bytesio_noaspectTime: ✅ 312.390µs (SLO: <420.000µs 📉 -25.6%) vs baseline: -1.4% Memory: ✅ 43.862MB (SLO: <46.000MB -4.6%) vs baseline: +5.6% ✅ capitalize_aspectTime: ✅ 88.639µs (SLO: <300.000µs 📉 -70.5%) vs baseline: -0.6% Memory: ✅ 43.912MB (SLO: <46.000MB -4.5%) vs baseline: +5.4% ✅ capitalize_noaspectTime: ✅ 248.433µs (SLO: <300.000µs 📉 -17.2%) vs baseline: -3.9% Memory: ✅ 43.851MB (SLO: <46.000MB -4.7%) vs baseline: +5.1% ✅ casefold_aspectTime: ✅ 91.736µs (SLO: <500.000µs 📉 -81.7%) vs baseline: +3.0% Memory: ✅ 43.840MB (SLO: <46.000MB -4.7%) vs baseline: +5.3% ✅ casefold_noaspectTime: ✅ 306.630µs (SLO: <500.000µs 📉 -38.7%) vs baseline: -2.1% Memory: ✅ 43.882MB (SLO: <46.000MB -4.6%) vs baseline: +5.0% ✅ decode_aspectTime: ✅ 87.324µs (SLO: <100.000µs 📉 -12.7%) vs baseline: +0.6% Memory: ✅ 44.058MB (SLO: <46.000MB -4.2%) vs baseline: +5.8% ✅ decode_noaspectTime: ✅ 155.159µs (SLO: <210.000µs 📉 -26.1%) vs baseline: +1.0% Memory: ✅ 44.010MB (SLO: <46.000MB -4.3%) vs baseline: +5.7% ✅ encode_aspectTime: ✅ 84.865µs (SLO: <200.000µs 📉 -57.6%) vs baseline: -0.4% Memory: ✅ 43.923MB (SLO: <46.000MB -4.5%) vs baseline: +5.7% ✅ encode_noaspectTime: ✅ 143.170µs (SLO: <200.000µs 📉 -28.4%) vs baseline: -0.7% Memory: ✅ 43.961MB (SLO: <46.000MB -4.4%) vs baseline: +5.6% ✅ format_aspectTime: ✅ 14.595ms (SLO: <19.200ms 📉 -24.0%) vs baseline: ~same Memory: ✅ 43.986MB (SLO: <46.000MB -4.4%) vs baseline: +5.5% ✅ format_map_aspectTime: ✅ 16.394ms (SLO: <21.500ms 📉 -23.8%) vs baseline: +0.5% Memory: ✅ 43.953MB (SLO: <46.000MB -4.4%) vs baseline: +5.2% ✅ format_map_noaspectTime: ✅ 368.862µs (SLO: <500.000µs 📉 -26.2%) vs baseline: -1.0% Memory: ✅ 43.913MB (SLO: <46.000MB -4.5%) vs baseline: +5.4% ✅ format_noaspectTime: ✅ 308.731µs (SLO: <500.000µs 📉 -38.3%) vs baseline: -1.9% Memory: ✅ 43.905MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ index_aspectTime: ✅ 122.852µs (SLO: <300.000µs 📉 -59.0%) vs baseline: -4.7% Memory: ✅ 43.904MB (SLO: <46.000MB -4.6%) vs baseline: +4.9% ✅ index_noaspectTime: ✅ 40.767µs (SLO: <300.000µs 📉 -86.4%) vs baseline: +0.9% Memory: ✅ 43.923MB (SLO: <46.000MB -4.5%) vs baseline: +5.0% ✅ join_aspectTime: ✅ 209.295µs (SLO: <300.000µs 📉 -30.2%) vs baseline: -5.3% Memory: ✅ 43.854MB (SLO: <46.000MB -4.7%) vs baseline: +5.1% ✅ join_noaspectTime: ✅ 142.726µs (SLO: <300.000µs 📉 -52.4%) vs baseline: -1.3% Memory: ✅ 43.915MB (SLO: <46.000MB -4.5%) vs baseline: +5.3% ✅ ljust_aspectTime: ✅ 498.370µs (SLO: <700.000µs 📉 -28.8%) vs baseline: -3.1% Memory: ✅ 43.880MB (SLO: <46.000MB -4.6%) vs baseline: +5.5% ✅ ljust_noaspectTime: ✅ 257.253µs (SLO: <300.000µs 📉 -14.2%) vs baseline: -2.5% Memory: ✅ 43.903MB (SLO: <46.000MB -4.6%) vs baseline: +5.4% ✅ lower_aspectTime: ✅ 293.477µs (SLO: <500.000µs 📉 -41.3%) vs baseline: -5.2% Memory: ✅ 43.901MB (SLO: <46.000MB -4.6%) vs baseline: +5.4% ✅ lower_noaspectTime: ✅ 234.101µs (SLO: <300.000µs 📉 -22.0%) vs baseline: -1.9% Memory: ✅ 43.998MB (SLO: <46.000MB -4.4%) vs baseline: +5.9% ✅ lstrip_aspectTime: ✅ 0.341ms (SLO: <3.000ms 📉 -88.6%) vs baseline: 📈 +23.9% Memory: ✅ 43.876MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ lstrip_noaspectTime: ✅ 0.176ms (SLO: <3.000ms 📉 -94.1%) vs baseline: -0.8% Memory: ✅ 43.948MB (SLO: <46.000MB -4.5%) vs baseline: +5.4% ✅ modulo_aspectTime: ✅ 14.286ms (SLO: <18.750ms 📉 -23.8%) vs baseline: +0.5% Memory: ✅ 43.916MB (SLO: <46.000MB -4.5%) vs baseline: +5.3% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.680ms (SLO: <19.350ms 📉 -24.1%) vs baseline: -0.5% Memory: ✅ 44.056MB (SLO: <46.000MB -4.2%) vs baseline: +5.6% ✅ modulo_aspect_for_bytesTime: ✅ 14.356ms (SLO: <18.900ms 📉 -24.0%) vs baseline: ~same Memory: ✅ 43.986MB (SLO: <46.000MB -4.4%) vs baseline: +5.1% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.525ms (SLO: <19.150ms 📉 -24.2%) vs baseline: -0.5% Memory: ✅ 44.057MB (SLO: <46.000MB -4.2%) vs baseline: +5.6% ✅ modulo_noaspectTime: ✅ 0.365ms (SLO: <3.000ms 📉 -87.8%) vs baseline: -3.2% Memory: ✅ 43.969MB (SLO: <46.000MB -4.4%) vs baseline: +5.5% ✅ replace_aspectTime: ✅ 18.297ms (SLO: <24.000ms 📉 -23.8%) vs baseline: -0.4% Memory: ✅ 43.926MB (SLO: <46.000MB -4.5%) vs baseline: +5.1% ✅ replace_noaspectTime: ✅ 304.927µs (SLO: <400.000µs 📉 -23.8%) vs baseline: +7.1% Memory: ✅ 43.924MB (SLO: <46.000MB -4.5%) vs baseline: +5.2% ✅ repr_aspectTime: ✅ 323.434µs (SLO: <420.000µs 📉 -23.0%) vs baseline: -4.0% Memory: ✅ 43.989MB (SLO: <46.000MB -4.4%) vs baseline: +5.1% ✅ repr_noaspectTime: ✅ 46.787µs (SLO: <90.000µs 📉 -48.0%) vs baseline: -0.4% Memory: ✅ 43.971MB (SLO: <46.000MB -4.4%) vs baseline: +5.5% ✅ rstrip_aspectTime: ✅ 378.691µs (SLO: <500.000µs 📉 -24.3%) vs baseline: -2.8% Memory: ✅ 43.971MB (SLO: <46.000MB -4.4%) vs baseline: +5.3% ✅ rstrip_noaspectTime: ✅ 184.211µs (SLO: <300.000µs 📉 -38.6%) vs baseline: +1.3% Memory: ✅ 43.968MB (SLO: <46.000MB -4.4%) vs baseline: +5.4% ✅ slice_aspectTime: ✅ 183.748µs (SLO: <300.000µs 📉 -38.8%) vs baseline: +1.1% Memory: ✅ 43.892MB (SLO: <46.000MB -4.6%) vs baseline: +5.2% ✅ slice_noaspectTime: ✅ 54.027µs (SLO: <90.000µs 📉 -40.0%) vs baseline: +0.5% Memory: ✅ 43.952MB (SLO: <46.000MB -4.5%) vs baseline: +5.3% ✅ stringio_aspectTime: ✅ 3.837ms (SLO: <5.000ms 📉 -23.3%) vs baseline: -1.4% Memory: ✅ 43.845MB (SLO: <46.000MB -4.7%) vs baseline: +5.1% ✅ stringio_noaspectTime: ✅ 344.602µs (SLO: <500.000µs 📉 -31.1%) vs baseline: -1.6% Memory: ✅ 43.871MB (SLO: <46.000MB -4.6%) vs baseline: +5.4% ✅ strip_aspectTime: ✅ 268.187µs (SLO: <350.000µs 📉 -23.4%) vs baseline: -1.5% Memory: ✅ 43.837MB (SLO: <46.000MB -4.7%) vs baseline: +5.3% ✅ strip_noaspectTime: ✅ 176.725µs (SLO: <240.000µs 📉 -26.4%) vs baseline: +0.6% Memory: ✅ 43.903MB (SLO: <46.000MB -4.6%) vs baseline: +5.5% ✅ swapcase_aspectTime: ✅ 334.933µs (SLO: <500.000µs 📉 -33.0%) vs baseline: -3.2% Memory: ✅ 43.804MB (SLO: <46.000MB -4.8%) vs baseline: +5.0% ✅ swapcase_noaspectTime: ✅ 271.614µs (SLO: <400.000µs 📉 -32.1%) vs baseline: -1.5% Memory: ✅ 44.012MB (SLO: <46.000MB -4.3%) vs baseline: +6.0% ✅ title_aspectTime: ✅ 318.683µs (SLO: <500.000µs 📉 -36.3%) vs baseline: -6.6% Memory: ✅ 43.836MB (SLO: <46.000MB -4.7%) vs baseline: +5.2% ✅ title_noaspectTime: ✅ 258.561µs (SLO: <400.000µs 📉 -35.4%) vs baseline: -2.7% Memory: ✅ 43.889MB (SLO: <46.000MB -4.6%) vs baseline: +5.4% ✅ translate_aspectTime: ✅ 561.968µs (SLO: <700.000µs 📉 -19.7%) vs baseline: +9.4% Memory: ✅ 43.929MB (SLO: <46.000MB -4.5%) vs baseline: +5.4% ✅ translate_noaspectTime: ✅ 423.278µs (SLO: <500.000µs 📉 -15.3%) vs baseline: -4.4% Memory: ✅ 43.834MB (SLO: <46.000MB -4.7%) vs baseline: +5.1% ✅ upper_aspectTime: ✅ 296.922µs (SLO: <500.000µs 📉 -40.6%) vs baseline: -6.3% Memory: ✅ 43.884MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% ✅ upper_noaspectTime: ✅ 235.871µs (SLO: <400.000µs 📉 -41.0%) vs baseline: -1.5% Memory: ✅ 43.928MB (SLO: <46.000MB -4.5%) vs baseline: +5.3% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 532.347µs (SLO: <700.000µs 📉 -24.0%) vs baseline: 📈 +23.9% Memory: ✅ 43.790MB (SLO: <46.000MB -4.8%) vs baseline: +5.3% ✅ ospathbasename_noaspectTime: ✅ 433.445µs (SLO: <700.000µs 📉 -38.1%) vs baseline: -0.9% Memory: ✅ 43.858MB (SLO: <46.000MB -4.7%) vs baseline: +5.9% ✅ ospathjoin_aspectTime: ✅ 625.967µs (SLO: <700.000µs 📉 -10.6%) vs baseline: -1.3% Memory: ✅ 43.873MB (SLO: <46.000MB -4.6%) vs baseline: +5.3% ✅ ospathjoin_noaspectTime: ✅ 641.536µs (SLO: <700.000µs -8.4%) vs baseline: -0.7% Memory: ✅ 43.817MB (SLO: <46.000MB -4.7%) vs baseline: +5.6% ✅ ospathnormcase_aspectTime: ✅ 350.231µs (SLO: <700.000µs 📉 -50.0%) vs baseline: -1.7% Memory: ✅ 43.873MB (SLO: <46.000MB -4.6%) vs baseline: +5.8% ✅ ospathnormcase_noaspectTime: ✅ 361.939µs (SLO: <700.000µs 📉 -48.3%) vs baseline: -0.8% Memory: ✅ 43.842MB (SLO: <46.000MB -4.7%) vs baseline: +5.7% ✅ ospathsplit_aspectTime: ✅ 488.035µs (SLO: <700.000µs 📉 -30.3%) vs baseline: -2.8% Memory: ✅ 43.831MB (SLO: <46.000MB -4.7%) vs baseline: +5.6% ✅ ospathsplit_noaspectTime: ✅ 500.795µs (SLO: <700.000µs 📉 -28.5%) vs baseline: -1.6% Memory: ✅ 43.868MB (SLO: <46.000MB -4.6%) vs baseline: +6.0% ✅ ospathsplitdrive_aspectTime: ✅ 372.275µs (SLO: <700.000µs 📉 -46.8%) vs baseline: -2.8% Memory: ✅ 43.812MB (SLO: <46.000MB -4.8%) vs baseline: +5.5% ✅ ospathsplitdrive_noaspectTime: ✅ 72.699µs (SLO: <700.000µs 📉 -89.6%) vs baseline: -1.2% Memory: ✅ 43.823MB (SLO: <46.000MB -4.7%) vs baseline: +5.6% ✅ ospathsplitext_aspectTime: ✅ 470.324µs (SLO: <700.000µs 📉 -32.8%) vs baseline: +1.3% Memory: ✅ 43.771MB (SLO: <46.000MB -4.8%) vs baseline: +5.5% ✅ ospathsplitext_noaspectTime: ✅ 471.936µs (SLO: <700.000µs 📉 -32.6%) vs baseline: -0.1% Memory: ✅ 43.843MB (SLO: <46.000MB -4.7%) vs baseline: +5.6% 🟡 Near SLO Breach (7 suites)🟡 djangosimple - 28/28✅ appsecTime: ✅ 19.704ms (SLO: <22.300ms 📉 -11.6%) vs baseline: ~same Memory: ✅ 71.564MB (SLO: <73.500MB -2.6%) vs baseline: +5.2% ✅ exception-replay-enabledTime: ✅ 1.367ms (SLO: <1.450ms -5.8%) vs baseline: -0.3% Memory: ✅ 69.579MB (SLO: <71.500MB -2.7%) vs baseline: +5.2% ✅ iastTime: ✅ 19.771ms (SLO: <22.250ms 📉 -11.1%) vs baseline: -0.3% Memory: ✅ 71.329MB (SLO: <75.000MB -4.9%) vs baseline: +5.1% ✅ profilerTime: ✅ 15.219ms (SLO: <16.550ms -8.0%) vs baseline: +0.6% Memory: ✅ 60.171MB (SLO: <61.000MB 🟡 -1.4%) vs baseline: +5.5% ✅ resource-renamingTime: ✅ 19.483ms (SLO: <21.750ms 📉 -10.4%) vs baseline: -0.5% Memory: ✅ 71.474MB (SLO: <73.500MB -2.8%) vs baseline: +5.2% ✅ span-code-originTime: ✅ 19.999ms (SLO: <28.200ms 📉 -29.1%) vs baseline: +1.1% Memory: ✅ 71.476MB (SLO: <75.000MB -4.7%) vs baseline: +5.4% ✅ tracerTime: ✅ 19.704ms (SLO: <21.750ms -9.4%) vs baseline: -0.3% Memory: ✅ 71.456MB (SLO: <75.000MB -4.7%) vs baseline: +5.3% ✅ tracer-and-profilerTime: ✅ 21.035ms (SLO: <23.500ms 📉 -10.5%) vs baseline: +0.5% Memory: ✅ 73.220MB (SLO: <75.000MB -2.4%) vs baseline: +4.9% ✅ tracer-dont-create-db-spansTime: ✅ 19.720ms (SLO: <21.500ms -8.3%) vs baseline: -0.4% Memory: ✅ 71.376MB (SLO: <75.000MB -4.8%) vs baseline: +5.3% ✅ tracer-minimalTime: ✅ 17.839ms (SLO: <18.500ms -3.6%) vs baseline: -0.6% Memory: ✅ 71.388MB (SLO: <75.000MB -4.8%) vs baseline: +5.1% ✅ tracer-no-cachesTime: ✅ 18.865ms (SLO: <19.650ms -4.0%) vs baseline: +0.2% Memory: ✅ 71.349MB (SLO: <75.000MB -4.9%) vs baseline: +5.2% ✅ tracer-no-databasesTime: ✅ 20.742ms (SLO: <21.100ms 🟡 -1.7%) vs baseline: +0.5% Memory: ✅ 71.145MB (SLO: <75.000MB -5.1%) vs baseline: +4.9% ✅ tracer-no-middlewareTime: ✅ 19.389ms (SLO: <21.500ms -9.8%) vs baseline: -0.3% Memory: ✅ 71.133MB (SLO: <75.000MB -5.2%) vs baseline: +4.8% ✅ tracer-no-templatesTime: ✅ 19.474ms (SLO: <22.000ms 📉 -11.5%) vs baseline: -5.8% Memory: ✅ 71.404MB (SLO: <73.500MB -2.9%) vs baseline: +5.1% 🟡 iastpropagation - 8/8✅ no-propagationTime: ✅ 48.750µs (SLO: <60.000µs 📉 -18.8%) vs baseline: +0.2% Memory: ✅ 41.209MB (SLO: <42.000MB 🟡 -1.9%) vs baseline: +5.3% ✅ propagation_enabledTime: ✅ 136.025µs (SLO: <190.000µs 📉 -28.4%) vs baseline: +0.3% Memory: ✅ 41.209MB (SLO: <42.000MB 🟡 -1.9%) vs baseline: +5.4% ✅ propagation_enabled_100Time: ✅ 1.556ms (SLO: <2.300ms 📉 -32.3%) vs baseline: +0.6% Memory: ✅ 41.229MB (SLO: <42.000MB 🟡 -1.8%) vs baseline: +5.3% ✅ propagation_enabled_1000Time: ✅ 29.081ms (SLO: <34.550ms 📉 -15.8%) vs baseline: -0.3% Memory: ✅ 41.366MB (SLO: <42.000MB 🟡 -1.5%) vs baseline: +5.7% 🟡 otelsdkspan - 24/24✅ add-eventTime: ✅ 40.363ms (SLO: <42.000ms -3.9%) vs baseline: -0.3% Memory: ✅ 39.046MB (SLO: <40.750MB -4.2%) vs baseline: +5.9% ✅ add-linkTime: ✅ 36.436ms (SLO: <38.550ms -5.5%) vs baseline: -0.5% Memory: ✅ 39.007MB (SLO: <40.750MB -4.3%) vs baseline: +6.1% ✅ add-metricsTime: ✅ 221.044ms (SLO: <232.000ms -4.7%) vs baseline: +0.3% Memory: ✅ 39.027MB (SLO: <40.750MB -4.2%) vs baseline: +6.1% ✅ add-tagsTime: ✅ 212.082ms (SLO: <221.600ms -4.3%) vs baseline: +0.2% Memory: ✅ 39.086MB (SLO: <40.750MB -4.1%) vs baseline: +6.1% ✅ get-contextTime: ✅ 29.269ms (SLO: <31.300ms -6.5%) vs baseline: +0.2% Memory: ✅ 38.987MB (SLO: <40.750MB -4.3%) vs baseline: +6.3% ✅ is-recordingTime: ✅ 29.322ms (SLO: <31.000ms -5.4%) vs baseline: +1.3% Memory: ✅ 38.987MB (SLO: <40.750MB -4.3%) vs baseline: +5.8% ✅ record-exceptionTime: ✅ 63.252ms (SLO: <65.850ms -3.9%) vs baseline: -0.4% Memory: ✅ 39.046MB (SLO: <40.750MB -4.2%) vs baseline: +5.2% ✅ set-statusTime: ✅ 31.909ms (SLO: <34.150ms -6.6%) vs baseline: -0.1% Memory: ✅ 38.968MB (SLO: <40.750MB -4.4%) vs baseline: +6.2% ✅ startTime: ✅ 29.645ms (SLO: <30.150ms 🟡 -1.7%) vs baseline: +3.1% Memory: ✅ 39.066MB (SLO: <40.750MB -4.1%) vs baseline: +5.9% ✅ start-finishTime: ✅ 34.093ms (SLO: <35.350ms -3.6%) vs baseline: +0.4% Memory: ✅ 38.928MB (SLO: <40.750MB -4.5%) vs baseline: +5.6% ✅ start-finish-telemetryTime: ✅ 34.076ms (SLO: <35.450ms -3.9%) vs baseline: -0.5% Memory: ✅ 38.987MB (SLO: <40.750MB -4.3%) vs baseline: +5.7% ✅ update-nameTime: ✅ 31.023ms (SLO: <33.400ms -7.1%) vs baseline: -0.3% Memory: ✅ 39.125MB (SLO: <40.750MB -4.0%) vs baseline: +6.0% 🟡 otelspan - 22/22✅ add-eventTime: ✅ 40.618ms (SLO: <47.150ms 📉 -13.9%) vs baseline: -1.1% Memory: ✅ 41.155MB (SLO: <47.000MB 📉 -12.4%) vs baseline: +5.1% ✅ add-metricsTime: ✅ 236.787ms (SLO: <344.800ms 📉 -31.3%) vs baseline: +0.3% Memory: ✅ 45.547MB (SLO: <47.500MB -4.1%) vs baseline: +4.9% ✅ add-tagsTime: ✅ 266.280ms (SLO: <330.000ms 📉 -19.3%) vs baseline: +2.0% Memory: ✅ 45.666MB (SLO: <47.500MB -3.9%) vs baseline: +4.9% ✅ get-contextTime: ✅ 83.335ms (SLO: <92.350ms -9.8%) vs baseline: -0.4% Memory: ✅ 41.543MB (SLO: <46.500MB 📉 -10.7%) vs baseline: +5.3% ✅ is-recordingTime: ✅ 39.019ms (SLO: <44.500ms 📉 -12.3%) vs baseline: -0.6% Memory: ✅ 41.126MB (SLO: <47.500MB 📉 -13.4%) vs baseline: +5.6% ✅ record-exceptionTime: ✅ 61.142ms (SLO: <67.650ms -9.6%) vs baseline: -0.6% Memory: ✅ 41.826MB (SLO: <47.000MB 📉 -11.0%) vs baseline: +5.5% ✅ set-statusTime: ✅ 44.945ms (SLO: <50.400ms 📉 -10.8%) vs baseline: ~same Memory: ✅ 41.173MB (SLO: <47.000MB 📉 -12.4%) vs baseline: +5.3% ✅ startTime: ✅ 40.015ms (SLO: <44.500ms 📉 -10.1%) vs baseline: +3.1% Memory: ✅ 41.112MB (SLO: <47.000MB 📉 -12.5%) vs baseline: +5.5% ✅ start-finishTime: ✅ 90.100ms (SLO: <92.000ms -2.1%) vs baseline: +0.1% Memory: ✅ 38.732MB (SLO: <46.500MB 📉 -16.7%) vs baseline: +5.3% ✅ start-finish-telemetryTime: ✅ 91.686ms (SLO: <93.000ms 🟡 -1.4%) vs baseline: ~same Memory: ✅ 38.850MB (SLO: <46.500MB 📉 -16.5%) vs baseline: +5.5% ✅ update-nameTime: ✅ 40.160ms (SLO: <45.150ms 📉 -11.1%) vs baseline: -0.7% Memory: ✅ 41.194MB (SLO: <47.000MB 📉 -12.4%) vs baseline: +5.2% 🟡 recursivecomputation - 8/8✅ deepTime: ✅ 311.691ms (SLO: <320.950ms -2.9%) vs baseline: -0.2% Memory: ✅ 37.356MB (SLO: <38.750MB -3.6%) vs baseline: +5.1% ✅ deep-profiledTime: ✅ 329.197ms (SLO: <359.150ms -8.3%) vs baseline: +0.2% Memory: ✅ 43.765MB (SLO: <46.000MB -4.9%) vs baseline: +5.3% ✅ mediumTime: ✅ 7.351ms (SLO: <7.450ms 🟡 -1.3%) vs baseline: -0.5% Memory: ✅ 36.294MB (SLO: <38.000MB -4.5%) vs baseline: +5.5% ✅ shallowTime: ✅ 1.039ms (SLO: <1.050ms 🟡 -1.1%) vs baseline: +1.4% Memory: ✅ 36.176MB (SLO: <38.000MB -4.8%) vs baseline: +5.4% 🟡 span - 25/25✅ add-eventTime: ✅ 19.546ms (SLO: <22.500ms 📉 -13.1%) vs baseline: -1.8% Memory: ✅ 38.418MB (SLO: <53.000MB 📉 -27.5%) vs baseline: +5.4% ✅ add-metricsTime: ✅ 89.183ms (SLO: <93.500ms -4.6%) vs baseline: -0.6% Memory: ✅ 42.959MB (SLO: <53.000MB 📉 -18.9%) vs baseline: +5.5% ✅ add-tagsTime: ✅ 137.241ms (SLO: <155.000ms 📉 -11.5%) vs baseline: ~same Memory: ✅ 42.925MB (SLO: <53.000MB 📉 -19.0%) vs baseline: +5.2% ✅ get-contextTime: ✅ 18.722ms (SLO: <20.500ms -8.7%) vs baseline: -1.9% Memory: ✅ 38.429MB (SLO: <53.000MB 📉 -27.5%) vs baseline: +6.2% ✅ is-recordingTime: ✅ 18.931ms (SLO: <20.500ms -7.7%) vs baseline: -1.2% Memory: ✅ 38.284MB (SLO: <53.000MB 📉 -27.8%) vs baseline: +5.5% ✅ record-exceptionTime: ✅ 38.674ms (SLO: <41.000ms -5.7%) vs baseline: ~same Memory: ✅ 38.818MB (SLO: <53.000MB 📉 -26.8%) vs baseline: +5.6% ✅ set-statusTime: ✅ 20.403ms (SLO: <22.000ms -7.3%) vs baseline: -1.6% Memory: ✅ 38.353MB (SLO: <53.000MB 📉 -27.6%) vs baseline: +5.4% ✅ startTime: ✅ 19.726ms (SLO: <20.500ms -3.8%) vs baseline: +3.5% Memory: ✅ 38.307MB (SLO: <53.000MB 📉 -27.7%) vs baseline: +5.3% ✅ start-finishTime: ✅ 58.092ms (SLO: <58.500ms 🟡 -0.7%) vs baseline: -0.1% Memory: ✅ 36.215MB (SLO: <38.000MB -4.7%) vs baseline: +5.4% ✅ start-finish-telemetryTime: ✅ 59.298ms (SLO: <60.000ms 🟡 -1.2%) vs baseline: -0.7% ✅ start-finish-traceid128Time: ✅ 60.584ms (SLO: <62.000ms -2.3%) vs baseline: -0.8% Memory: ✅ 36.196MB (SLO: <38.000MB -4.7%) vs baseline: +5.3% ✅ start-traceid128Time: ✅ 18.654ms (SLO: <22.500ms 📉 -17.1%) vs baseline: -2.6% Memory: ✅ 38.454MB (SLO: <53.000MB 📉 -27.4%) vs baseline: +6.0% ✅ update-nameTime: ✅ 19.276ms (SLO: <22.000ms 📉 -12.4%) vs baseline: -1.7% Memory: ✅ 38.376MB (SLO: <53.000MB 📉 -27.6%) vs baseline: +5.3% 🟡 tracer - 6/6✅ largeTime: ✅ 33.104ms (SLO: <33.950ms -2.5%) vs baseline: -0.6% Memory: ✅ 37.808MB (SLO: <39.250MB -3.7%) vs baseline: +6.1% ✅ mediumTime: ✅ 3.329ms (SLO: <3.500ms -4.9%) vs baseline: -0.3% Memory: ✅ 36.255MB (SLO: <38.750MB -6.4%) vs baseline: +5.3% ✅ smallTime: ✅ 387.359µs (SLO: <390.000µs 🟡 -0.7%) vs baseline: +3.4% Memory: ✅ 36.196MB (SLO: <38.750MB -6.6%) vs baseline: +5.0%
|
avara1986
added
the
changelog/no-changelog
🎉 All green!❄️ No new flaky tests detected 🔗 Commit SHA: f4dd6de | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback! |
Move the native _stacktrace C extension from ddtrace/appsec/_iast/ to ddtrace/appsec/_shared/ so it can be reused by both IAST and SCA without creating a dependency from SCA into IAST internals. Split out from #17156 to keep PRs incremental and reviewable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…h_utils Move rel_path() and the frame-walking logic (_compute_file_line) from VulnerabilityBase in _iast/taint_sinks/_base.py to shared functions in _patch_utils.py so both IAST and SCA can reuse them without depending on IAST internals. Split out from #17156 to keep PRs incremental and reviewable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…h_utils Move rel_path() and the frame-walking logic (_compute_file_line) from VulnerabilityBase in _iast/taint_sinks/_base.py to shared functions in _patch_utils.py so both IAST and SCA can reuse them without depending on IAST internals. Split out from #17156 to keep PRs incremental and reviewable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…h_utils Move rel_path() and the frame-walking logic (_compute_file_line) from VulnerabilityBase in _iast/taint_sinks/_base.py to shared functions in _patch_utils.py so both IAST and SCA can reuse them without depending on IAST internals. Also migrates insecure_cookie.py and updates test_weak_hash.py mock target accordingly. Split out from #17156 to keep PRs incremental and reviewable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…h_utils (#17334) ## Summary - Extract `rel_path()` and `_compute_file_line()` from `VulnerabilityBase` in `_iast/taint_sinks/_base.py` into shared functions (`rel_path` and `get_caller_frame_info`) in `_patch_utils.py`. - Migrate `insecure_cookie.py` to use the shared `get_caller_frame_info()` instead of `cls._compute_file_line()`. - Update `test_weak_hash.py` mock target from `get_info_frame` to `get_caller_frame_info`. - Both IAST and SCA can now reuse these functions without depending on IAST internals. Split out from #17156 to keep PRs incremental and reviewable. > **Important:** Before merging this PR, DataDog/datadog-lambda-python#761 must be merged first. ## Test plan - [ ] Existing IAST vulnerability tests pass (they call `VulnerabilityBase.report()` which now delegates to `get_caller_frame_info()`) - [ ] IAST cookie tests pass (`insecure_cookie.py` now uses shared function) - [ ] `test_weak_hash.py` edge case test passes with updated mock target 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: alberto.vara <alberto.vara@datadoghq.com>
) ## Summary - Move the native `_stacktrace` C extension from `ddtrace/appsec/_iast/` to `ddtrace/appsec/_shared/` so it can be reused by both IAST and SCA without creating a dependency from SCA into IAST internals. - Update all imports, build configuration (`setup.py`), and test references to use the new `ddtrace.appsec._shared._stacktrace` path. Split out from #17156 to keep PRs incremental and reviewable. > **Important:** Before merging this PR, DataDog/datadog-lambda-python#761 must be merged first, since `datadog-lambda-python` imports `ddtrace.appsec._iast._stacktrace` and needs to be updated to the new path. ## Test plan - [ ] Existing IAST stacktrace tests pass (`tests/appsec/iast/test_stacktrace.py`) - [ ] IAST memcheck tests pass (`tests/appsec/iast_memcheck/test_iast_mem_check.py`) - [ ] Architecture loading module test passes (`tests/appsec/architectures/test_appsec_loading_modules.py`) - [ ] Serverless import test passes (`tests/internal/test_serverless.py`) - [ ] Native C extension builds correctly from new path 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: alberto.vara <alberto.vara@datadoghq.com>
avara1986
force-pushed
the
avara1986/sca-telemetry-rereporting-and-module
branch
3 times, most recently
from
c9fcd49 to
71fdc56
Compare
Implements Runtime SCA Reachability — the tracer reports which vulnerable symbols have actually been invoked at runtime, reducing false positives from static SCA analysis. - DependencyEntry model with reachability metadata tracking - DependencyTracker for heartbeat-based dependency reporting - Skip re-report scan when SCA is disabled (idle heartbeat optimization) - CVE loader, registry, resolver, and instrumenter for SCA hooks - SCA product integration with telemetry writer - SLO benchmark suite (benchmarks/telemetry_dependencies/) - Standalone benchmark scripts (perf_bench_heartbeat_cycles/sca_telemetry) - Riot env and suitespec entries for SCA tests and benchmarks - Flask integration test for SCA telemetry via testagent Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
avara1986
force-pushed
the
avara1986/sca-telemetry-rereporting-and-module
branch
from
39f37ce to
15db88d
Compare
…mpty When _get_caller_info() returns empty (e.g., deep wrapt/gevent stack where the native C frame walker can't find user code), the hook now falls back to the target function's own qualified name (module path + symbol) instead of passing empty path to attach_dependency_metadata. Previously, empty path caused add_metadata's `if path` guard to silently drop the finding, leaving the reached array permanently empty — even though the hook was firing correctly. Adds test_instrumenter.py with 8 tests covering: - inject_hook fires on direct calls and through wrapt wrappers - Full SCA detection hook → registry → telemetry flow - Caller info propagation to reached array - Fallback behavior when caller info is unavailable Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds execution_time and max_rss_usage thresholds for the 8 telemetry dependency reporting scenarios. Thresholds set with ~3-5x headroom over observed local measurements to account for CI variability. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
On Python <3.11, co_firstlineno points to the `def` line which has no bytecode instructions. inject_hook needs the first instruction's line (the function body). Added _first_instr_line() helper that handles the cross-version difference (starts_line is int on <3.12, bool on >=3.12). Also fixes the test to use the same helper instead of co_firstlineno. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
os.register_at_fork(after_in_child=...) callbacks fire AFTER ddtrace's forksafe mechanism calls product.restart(). This meant: 1. restart() ran and set up _registry 2. CPython fork callback wiped _registry=None 3. _registry was never set again — hook silently returned Fix: Remove os.register_at_fork from _instrumenter.py and _registry.py. Instead, restart() in product.py explicitly calls both reset functions before re-initializing, ensuring correct ordering. Adds 3 tests for the fork callback ordering guarantee. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
avara1986
force-pushed
the
avara1986/sca-telemetry-rereporting-and-module
branch
from
f8227fb to
962eb72
Compare
…17514) ## Summary This PR extracts the telemetry dependency tracking changes from the larger SCA PR (#17156) to reduce its size and make review more manageable. - Introduces `DependencyTracker` to manage dependency state, deduplication, and re-reporting logic for SCA telemetry - Refactors dependency collection out of the telemetry writer into a dedicated module - Updates the benchmark scenario for `packages_update_imported_dependencies` - Adds comprehensive tests for the new dependency tracker and updated dependency logic **Parent PR:** #17156 — `feat(sca): runtime SCA reachability` ## Test plan - [ ] Existing telemetry tests pass - [ ] New `tests/telemetry/test_dependency.py` tests pass - [ ] `tests/telemetry/test_data.py` tests pass - [ ] Benchmark scenario runs without errors ## Checklist - [x] PR description/title adhere to [contributing guidelines](https://ddtrace.readthedocs.io/en/stable/contributing.html) - [x] Tests provided or no tests needed 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: alberto.vara <alberto.vara@datadoghq.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 639230db3a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
gh-worker-dd-mergequeue-cf854d
Bot
deleted the
avara1986/sca-telemetry-rereporting-and-module
branch
## Summary The `telemetry_dependencies` benchmark was failing against the baseline wheel after #17156 merged, with: ``` ModuleNotFoundError: No module named 'ddtrace.internal.telemetry.dependency' ``` Root cause: `scenario._run_old_api` had a function-scope `from ddtrace.internal.telemetry.dependency import DependencyEntry as DE` that Python evaluates on every call. The baseline wheel predates the SCA-reachability PR, so that module does not exist and the import crashes before any phase branching. The `else` branch also populated `already_imported` with `DependencyEntry` instances, but the baseline's `update_imported_dependencies(already_imported: Dict[str, str], ...)` expects version strings — it would break if ever exercised. ## Changes - Drop the broken top-level import. - Rewrite the idle/fallback branch to populate `already_imported` as `{name: version_str}`, matching the baseline contract. The candidate (new-API) path is unaffected — it runs `_run_new_api` because `HAS_DEPENDENCY_TRACKER` is True on current `main`. ## Test plan - [ ] Re-run the benchmark platform job for `telemetry_dependencies` and confirm it completes for both baseline and candidate. - [ ] Spot-check that the `telemetry_add_metric` job (which was already passing) is still unaffected. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Description Implements **Runtime SCA Reachability** — the tracer reports which vulnerable symbols (functions/methods in third-party libraries with known CVEs) have actually been invoked at runtime, reducing false positives from static SCA analysis. **RFC**: https://docs.google.com/document/d/1xDw9iG6h41VCEgJGTqoJdruRaNS4pYgNifO6nhiizWA/edit?tab=t.a9gurws0d8ua ### How it works When `DD_APPSEC_SCA_ENABLED=true`: 1. **CVE data loading** — At tracer startup, reads `_cve_data.json` containing vulnerability targets (symbol + CVE + version constraint). Filters against installed packages. 2. **Runtime instrumentation** — For each applicable CVE target, applies bytecode injection (`inject_hook`) to the vulnerable function. Supports both eager (already-imported modules) and lazy (via `ModuleWatchdog` for deferred imports). 3. **CVE registration** — Immediately registers all applicable CVEs on their dependencies with `reached: []` in the telemetry `app-dependencies-loaded` payload. The backend knows which CVEs apply before any symbol is hit. 4. **Reachability detection** — When an instrumented function executes, the hook captures the caller's file path, method name, and line number, then attaches it to the CVE's `reached` array. Only the first occurrence is reported per CVE (RFC: "reporting a single occurrence is sufficient"). 5. **Telemetry reporting** — On each heartbeat (default 60s), dependencies with new reachability data are re-reported with all their metadata via `app-dependencies-loaded`. ### Telemetry payload (RFC v3) ```json { "request_type": "app-dependencies-loaded", "payload": { "dependencies": [ { "name": "requests", "version": "2.31.0", "metadata": [ { "type": "reachability", "value": "{\"id\":\"CVE-2024-35195\",\"reached\":[{\"path\":\"myapp/views.py\",\"method\":\"handle_request\",\"line\":42}]}" } ] } ] } } ``` Before any symbol is hit, CVEs are reported with `"reached":[]`. When a symbol executes, the first caller info is added to the `reached` array. ## Performance Benchmark: `scripts/perf_bench_heartbeat_cycles.py` Measures `collect_report()` execution time per heartbeat cycle under different scenarios. The overhead is paid only in the background telemetry thread (every 60s), never in user request paths. - **SCA OFF (main)**: baseline on `main` branch — old `update_imported_dependencies()` path, no `DependencyTracker`, no re-report scan - **SCA OFF**: this branch with `DD_APPSEC_SCA_ENABLED=false` — new `DependencyTracker` with `metadata=None`, re-report scan skipped - **SCA ON**: this branch with `DD_APPSEC_SCA_ENABLED=true` — new `DependencyTracker` with `metadata=[]` - **Overhead**: `(SCA ON − SCA OFF) / SCA OFF` ### 1,000 dependencies (typical large application) | Heartbeat Cycle | SCA OFF (main) | SCA OFF | SCA ON | Overhead | |---|---|---|---|---| | First heartbeat (all new) | 4.56ms | 4.63ms | 5.10ms | **+10.1%** | | Idle (nothing to report) | 0.2us | 72.2us | 239.6us | **+231.8%** | | CVE registration (100 CVEs, reached=[]) | 0.3us | 73.5us | 1.20ms | **+1527.5%** | | SCA hits (100 hits on 50 deps) | 0.3us | 73.7us | 1.44ms | **+1857.1%** | ### 10,000 dependencies (extreme scale) | Heartbeat Cycle | SCA OFF (main) | SCA OFF | SCA ON | Overhead | |---|---|---|---|---| | First heartbeat (all new) | 53.15ms | 57.18ms | 62.12ms | **+8.6%** | | Idle (nothing to report) | 0.3us | 742.6us | 2.20ms | **+195.7%** | | CVE registration (1,000 CVEs, reached=[]) | 0.3us | 720.7us | 13.51ms | **+1774.4%** | | SCA hits (1,000 hits on 500 deps) | 0.3us | 718.9us | 15.56ms | **+2064.2%** | ### Payload size | Scenario (1,000 deps) | Entries | Size | |---|---|---| | First heartbeat SCA OFF | 1,000 | 62 KB | | First heartbeat SCA ON | 1,000 | 62 KB | | CVE registration (100 CVEs) | 50 | 10 KB | | SCA hits (100 hits) | 50 | 16 KB | ### Memory overhead | Scenario (1,000 deps) | SCA OFF | SCA ON | Delta | |---|---|---|---| | Idle heartbeat | 155.3 KB | 192.5 KB | +37.2 KB | | CVE registration (100 CVEs) | 136.4 KB | 205.8 KB | +69.4 KB | | SCA hits (100 hits) | 136.2 KB | 208.5 KB | +72.3 KB | > **Note on overhead**: The percentage overhead for idle, CVE registration, and SCA hits appears very high because the SCA OFF baseline includes mock/benchmark harness overhead (~72us at 1K deps) that dominates the measurement. In absolute terms: > > - **First heartbeat**: nearly identical across all three columns (4.56ms → 4.63ms → 5.10ms at 1K deps), confirming the `DependencyTracker` refactor adds **minimal overhead** to initial dependency discovery. > - **Idle heartbeat with SCA OFF**: ~72us includes benchmark mock overhead; measured independently at ~8us (lock acquisition + `get_newly_imported_modules()` + lazy config check). The re-report scan is **completely skipped** when SCA is disabled. > - **SCA ON worst case** at 1,000 dependencies: **1.44ms per 60s heartbeat** — 0.002% of the cycle. > - **SCA ON worst case** at 10,000 dependencies with 1,000 CVEs: **16ms** — 0.027% of the heartbeat interval. > > All overhead runs entirely in the background telemetry thread and does not affect user request latency. ### SLO benchmark A CI-integrated benchmark suite is available at `benchmarks/telemetry_dependencies/` with 8 scenarios covering first/idle/cve/hits phases at 100 and 1,000 dependencies. It is triggered automatically when `ddtrace/internal/telemetry/dependency*.py` or `ddtrace/appsec/sca/*` files change and compares performance between the base branch and this PR. ## Risks - **Bytecode injection**: Uses the existing `inject_hook` infrastructure from dd-trace-py. The hook is exception-safe (wrapped in try/except) and never raises in user code. - **Memory**: `DependencyEntry` objects add ~150 bytes per dependency vs plain strings. At 1,000 deps this is ~150KB total — negligible. - **Lock contention**: The `DependencyTracker._lock` is held briefly during `attach_metadata` calls from the SCA hook. After the first hit per CVE (max reached=1), subsequent hook invocations short-circuit before any lock acquisition. ## Additional Notes - Merge this PR first: DataDog/datadog-lambda-python#761 - Previous model PR benchmarks: #17092 - The static CVE JSON (`_cve_data.json`) will be replaced by Remote Config in the long-term solution Co-authored-by: alberto.vara <alberto.vara@datadoghq.com>
## Summary The `telemetry_dependencies` benchmark was failing against the baseline wheel after #17156 merged, with: ``` ModuleNotFoundError: No module named 'ddtrace.internal.telemetry.dependency' ``` Root cause: `scenario._run_old_api` had a function-scope `from ddtrace.internal.telemetry.dependency import DependencyEntry as DE` that Python evaluates on every call. The baseline wheel predates the SCA-reachability PR, so that module does not exist and the import crashes before any phase branching. The `else` branch also populated `already_imported` with `DependencyEntry` instances, but the baseline's `update_imported_dependencies(already_imported: Dict[str, str], ...)` expects version strings — it would break if ever exercised. ## Changes - Drop the broken top-level import. - Rewrite the idle/fallback branch to populate `already_imported` as `{name: version_str}`, matching the baseline contract. The candidate (new-API) path is unaffected — it runs `_run_new_api` because `HAS_DEPENDENCY_TRACKER` is True on current `main`. ## Test plan - [ ] Re-run the benchmark platform job for `telemetry_dependencies` and confirm it completes for both baseline and candidate. - [ ] Spot-check that the `telemetry_add_metric` job (which was already passing) is still unaffected. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Description
Implements Runtime SCA Reachability — the tracer reports which vulnerable symbols (functions/methods in third-party libraries with known CVEs) have actually been invoked at runtime, reducing false positives from static SCA analysis.
RFC: https://docs.google.com/document/d/1xDw9iG6h41VCEgJGTqoJdruRaNS4pYgNifO6nhiizWA/edit?tab=t.a9gurws0d8ua
How it works
When
DD_APPSEC_SCA_ENABLED=true:_cve_data.jsoncontaining vulnerability targets (symbol + CVE + version constraint). Filters against installed packages.inject_hook) to the vulnerable function. Supports both eager (already-imported modules) and lazy (viaModuleWatchdogfor deferred imports).reached: []in the telemetryapp-dependencies-loadedpayload. The backend knows which CVEs apply before any symbol is hit.reachedarray. Only the first occurrence is reported per CVE (RFC: "reporting a single occurrence is sufficient").app-dependencies-loaded.Telemetry payload (RFC v3)
{ "request_type": "app-dependencies-loaded", "payload": { "dependencies": [ { "name": "requests", "version": "2.31.0", "metadata": [ { "type": "reachability", "value": "{\"id\":\"CVE-2024-35195\",\"reached\":[{\"path\":\"myapp/views.py\",\"method\":\"handle_request\",\"line\":42}]}" } ] } ] } }Before any symbol is hit, CVEs are reported with
"reached":[]. When a symbol executes, the first caller info is added to thereachedarray.Performance
Benchmark:
scripts/perf_bench_heartbeat_cycles.pyMeasures
collect_report()execution time per heartbeat cycle under different scenarios. The overhead is paid only in the background telemetry thread (every 60s), never in user request paths.mainbranch — oldupdate_imported_dependencies()path, noDependencyTracker, no re-report scanDD_APPSEC_SCA_ENABLED=false— newDependencyTrackerwithmetadata=None, re-report scan skippedDD_APPSEC_SCA_ENABLED=true— newDependencyTrackerwithmetadata=[](SCA ON − SCA OFF) / SCA OFF1,000 dependencies (typical large application)
10,000 dependencies (extreme scale)
Payload size
Memory overhead
SLO benchmark
A CI-integrated benchmark suite is available at
benchmarks/telemetry_dependencies/with 8 scenarios covering first/idle/cve/hits phases at 100 and 1,000 dependencies. It is triggered automatically whenddtrace/internal/telemetry/dependency*.pyorddtrace/appsec/sca/*files change and compares performance between the base branch and this PR.Risks
inject_hookinfrastructure from dd-trace-py. The hook is exception-safe (wrapped in try/except) and never raises in user code.DependencyEntryobjects add ~150 bytes per dependency vs plain strings. At 1,000 deps this is ~150KB total — negligible.DependencyTracker._lockis held briefly duringattach_metadatacalls from the SCA hook. After the first hit per CVE (max reached=1), subsequent hook invocations short-circuit before any lock acquisition.Additional Notes
_cve_data.json) will be replaced by Remote Config in the long-term solution