Skip to content

test(profiling): fix flaky exception raise test#19139

Open
gyuheon0h wants to merge 2 commits into
mainfrom
gyuheon0h/harden-flaky-exc-tests
Open

test(profiling): fix flaky exception raise test#19139
gyuheon0h wants to merge 2 commits into
mainfrom
gyuheon0h/harden-flaky-exc-tests

Conversation

@gyuheon0h

@gyuheon0h gyuheon0h commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Description

When sampling_interval=1, skip Poisson sampling so every exception is collected instead of allowing random gaps of 2+. That removes the flake where a stray exception could consume the sample slot and miss intentional exceptions thrown. Used claude for verif script

Testing

ran the exception test file 100x...

# Build -k expression from test function names in the file, then loop 100x
scripts/ddtest bash -c '
set -e
KEXPR=$(python3 - <<'"'"'PY'"'"'
import ast, pathlib
src = pathlib.Path("tests/profiling/collector/test_exception.py").read_text()
names = [n.name for n in ast.walk(ast.parse(src)) if isinstance(n, ast.FunctionDef) and n.name.startswith("test_")]
print(" or ".join(names))
PY
)
echo "Filter: $KEXPR"
echo "Collect check:"
riot -v run --pass-env -s daba82d -- -k "$KEXPR" --collect-only -q 2>&1 | tail -30

fail=0
passed=0
for i in $(seq 1 100); do
  if riot -v run --pass-env -s daba82d -- -k "$KEXPR" --tb=line -q >/tmp/exc_file.out 2>&1; then
    passed=$((passed+1))
    echo "PASS $i/100"
  else
    echo "FAIL on run $i/100"
    tail -80 /tmp/exc_file.out
    fail=1
    break
  fi
done
echo "RESULT: passed=$passed/100 fail=$fail"
exit $fail
'
PASS 1/100
PASS 2/100
PASS 3/100
PASS 4/100
...
PASS 99/100
PASS 100/100

Risks

Additional Notes

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@gyuheon0h gyuheon0h changed the title interval = 1 means to sample **every** exception chore(profiling): sample every exception when internal is 1 Jul 17, 2026
@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 45 circular imports that already exist on the base branch and have not been changed by this PR.

Show existing cycles (showing 5 of 45 shortest)
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.kafka -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.kombu -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.aiokafka -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.botocore -> ddtrace.internal.datastreams
ddtrace.internal.core -> ddtrace._trace.span -> ddtrace.internal.core

To see all cycles, download the cycles-base.json and cycles-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/cycles.py compare cycles-base.json cycles-pr.json

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Pipelines  Tests

⚠️ Warnings

🚦 10 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-py | build linux serverless: [arm64, cp315-cp315, v113741357-d2b8243-manylinux2014_aarch64, 1]   View in Datadog   GitLab

DataDog/apm-reliability/dd-trace-py | build linux serverless: [arm64, cp315-cp315, v113741589-d2b8243-musllinux_1_2_aarch64, 1]   View in Datadog   GitLab

DataDog/apm-reliability/dd-trace-py | build linux: [amd64, cp315-cp315, v113741238-d2b8243-manylinux2014_x86_64]   View in Datadog   GitLab

View all 10 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ff3d8d2 | Docs | Datadog PR Page | Give us feedback!

@gyuheon0h gyuheon0h added the changelog/no-changelog A changelog entry is not required for this PR. label Jul 17, 2026
@gyuheon0h gyuheon0h changed the title chore(profiling): sample every exception when internal is 1 chore(profiling): fix flaky exception raise test Jul 17, 2026
@KowalskiThomas KowalskiThomas changed the title chore(profiling): fix flaky exception raise test test(profiling): fix flaky exception raise test Jul 17, 2026
@KowalskiThomas KowalskiThomas added the Profiling Continous Profling label Jul 17, 2026
@gyuheon0h
gyuheon0h marked this pull request as ready for review July 17, 2026 13:06
@gyuheon0h
gyuheon0h requested a review from a team as a code owner July 17, 2026 13:06
@gyuheon0h
gyuheon0h requested a review from vlad-scherbich July 17, 2026 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/no-changelog A changelog entry is not required for this PR. Profiling Continous Profling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants