Skip to content

Commit 70f2f22

Browse files
committed
[None][test] CBTS coverage HTML: show per-line test contexts
Enable coverage.py's show_contexts in the rendered HTML report so each covered line carries a clickable list of the pytest nodeids that exercised it. The data was already being captured -- cbts_plugin and sitecustomize call cov.switch_context(nodeid) per test on both the pytest main and MPI worker sides -- but the post-job HTML rendering in L0_MergeRequest's "Test Coverage" stage was not asking for it. Three .coveragerc-touching changes: * jenkins/L0_MergeRequest.groovy "Test Coverage" stage: rewrite the combine-time rcfile as a heredoc, add [html] show_contexts = True, and pass --show-contexts on the coverage html invocation. No-op for legacy non-CBTS stages whose .coverage carries no context info. * jenkins/L0_Test.groovy isCbtsStage(...) branch and jenkins/scripts/cbts/coverage_utils/coveragerc.template: same [html] block so a manually-rendered HTML report uses the per-stage rcfile. No behavior change to test execution. Affects only the rendering step that runs after combine on the Jenkins controller. Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
1 parent 9deb14c commit 70f2f22

3 files changed

Lines changed: 36 additions & 4 deletions

File tree

jenkins/L0_MergeRequest.groovy

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1173,16 +1173,34 @@ def collectTestResults(pipeline, testFilter)
11731173
sh "cp llm/examples/openai_triton/manual_plugin/fmha_triton.py llm/examples/openai_triton/plugin_autogen/"
11741174
def coverageConfigFile = "cov/.coveragerc"
11751175
sh """
1176-
echo '[paths]' > ${coverageConfigFile}
1177-
echo 'source1=\n ${CUR_PATH}/llm/examples/\n */TensorRT-LLM/src/examples/' >> ${coverageConfigFile}
1178-
echo 'source2=\n ${CUR_PATH}/llm/tensorrt_llm/\n */tensorrt_llm/' >> ${coverageConfigFile}
1176+
cat > ${coverageConfigFile} <<'EOF'
1177+
[paths]
1178+
source1 =
1179+
${CUR_PATH}/llm/examples/
1180+
*/TensorRT-LLM/src/examples/
1181+
source2 =
1182+
${CUR_PATH}/llm/tensorrt_llm/
1183+
*/tensorrt_llm/
1184+
1185+
[html]
1186+
# Per-line "which tests covered this" annotations in the rendered HTML
1187+
# report. Pairs with cov.switch_context() calls in cbts_plugin /
1188+
# sitecustomize that tag each line with the pytest nodeid that
1189+
# triggered it. No-op for non-CBTS stages whose .coverage files
1190+
# carry no context info.
1191+
show_contexts = True
1192+
EOF
11791193
cat ${coverageConfigFile}
11801194
"""
11811195

11821196
sh "cd cov && coverage combine"
11831197
sh "cd cov && find . -type f"
11841198
sh "cd cov && coverage report -i" // -i: ignore errors. Ignore the error that the source code file cannot be found.
1185-
sh "cd cov && coverage html -d test_coverage_html -i"
1199+
// --show-contexts pairs with show_contexts=True in .coveragerc
1200+
// [html] section: each line in the rendered HTML gets a
1201+
// clickable list of test nodeids that covered it. For non-CBTS
1202+
// stages whose .coverage carries no context info this is a no-op.
1203+
sh "cd cov && coverage html -d test_coverage_html -i --show-contexts"
11861204
trtllm_utils.uploadArtifacts("cov/test_coverage_html/*", "${UPLOAD_PATH}/test-results/coverage-report/")
11871205
echo "Test coverage report: https://urm.nvidia.com/artifactory/${UPLOAD_PATH}/test-results/coverage-report/index.html"
11881206
} // Test coverage

jenkins/L0_Test.groovy

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1144,6 +1144,13 @@ source =
11441144
11451145
[report]
11461146
ignore_errors = True
1147+
1148+
[html]
1149+
# Per-line "which tests covered this" annotations in the rendered HTML
1150+
# report (clickable contexts list). Pairs with cov.switch_context()
1151+
# calls in cbts_plugin / sitecustomize that tag each line with the
1152+
# pytest nodeid that triggered it.
1153+
show_contexts = True
11471154
EOF
11481155
cat ./.coveragerc
11491156
"""

jenkins/scripts/cbts/coverage_utils/coveragerc.template

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,10 @@ source =
4848

4949
[report]
5050
ignore_errors = True
51+
52+
[html]
53+
# Per-line "which tests covered this" annotations in the rendered HTML
54+
# report (clickable contexts list). Pairs with cov.switch_context()
55+
# calls in cbts_plugin / sitecustomize that tag each line with the
56+
# pytest nodeid that triggered it.
57+
show_contexts = True

0 commit comments

Comments
 (0)