Skip to content

fix: allow _with_context tests to collect sample rows#981

Open
joostboon wants to merge 2 commits intomasterfrom
fix/with-context-tests-sample-rows
Open

fix: allow _with_context tests to collect sample rows#981
joostboon wants to merge 2 commits intomasterfrom
fix/with-context-tests-sample-rows

Conversation

@joostboon
Copy link
Copy Markdown
Contributor

@joostboon joostboon commented Apr 9, 2026

Summary

  • _with_context tests are Elementary-namespaced but behave like regular dbt tests (they return failing rows rather than handling their own result row collection)
  • The test materialization was early-returning for all elementary-namespaced tests, bypassing the sampling logic entirely
  • This meant _with_context tests never wrote rows to test_result_rows, even when tagged with show_sample_rows

Fix

Check if the test name ends with _with_context before early-returning, allowing these tests to go through the standard handle_dbt_test sampling path.

Test plan

  • Run a failing _with_context test (e.g. not_null_with_context) and verify rows appear in test_result_rows
  • Verify non-_with_context Elementary tests (anomaly, schema change) still early-return and handle their own result rows

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Narrowed special-case handling for certain elementary tests so only recognized elementary test types use the custom materialization path; other elementary-namespaced tests (including those with "_with_context") now follow the standard test processing flow.

Elementary-namespaced tests were early-returning from the test
materialization, bypassing all sampling logic. _with_context tests
behave like regular dbt tests (they return failing rows) so they
should go through the standard handle_dbt_test sampling path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

👋 @joostboon
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in the elementary repository.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 9, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a5f2e3d9-46f0-492b-a448-b7187423f8e7

📥 Commits

Reviewing files that changed from the base of the PR and between 7e3c6b1 and 37988b0.

📒 Files selected for processing (1)
  • macros/edr/materializations/test/test.sql

📝 Walkthrough

Walkthrough

Updated the materialize_test hook to determine whether to early-return to the materialization_macro() by computing a lowercase short_name from test_metadata.name, calling elementary.get_elementary_test_type({"short_name": short_name, "test_namespace": "elementary"}), and only short-circuiting when that call returns a truthy value; otherwise tests (including elementary-namespaced ones without a detected elementary test type) follow the normal flatten_testhandle_dbt_test flow.

Changes

Cohort / File(s) Summary
Test Materialization Hook
macros/edr/materializations/test/test.sql
Narrowed early-bypass condition: compute short_name = lower(test_metadata.name) and call elementary.get_elementary_test_type({"short_name": short_name, "test_namespace": "elementary"}); only return materialization_macro() when that call is truthy; otherwise proceed to standard flatten_testhandle_dbt_test processing.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 I peered at names both short and neat,
Lowercased hops made decisions sweet,
Only true elementary types take flight,
The rest hop onwards—handled right! ✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main change: allowing _with_context tests to collect sample rows, which directly addresses the fix implemented in the materialization hook.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/with-context-tests-sample-rows

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

…alization

Instead of checking test name suffix, use the existing get_elementary_test_type
macro to determine if an elementary-namespaced test handles its own result row
collection. Only anomaly detection and schema change tests early-return; all
other elementary tests (e.g. _with_context) go through the standard sampling path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant