Skip to content

fix(weave): support eval_results filtering on inputs#7006

Open
domphan-wandb wants to merge 1 commit into
masterfrom
dom/eval-filtering-on-io
Open

fix(weave): support eval_results filtering on inputs#7006
domphan-wandb wants to merge 1 commit into
masterfrom
dom/eval-filtering-on-io

Conversation

@domphan-wandb
Copy link
Copy Markdown
Contributor

@domphan-wandb domphan-wandb commented May 29, 2026

Description

This PR fixes a few issues with filtering and sorting capabilities on imperatively logged evals because they don't have an actual dataset.

Problem

Filtering and sorting on inputs.* was returning empty results whose row_digest had no matching row in table_rows.

  • The query builder buildsresolved_inputs with COALESCE(tr.val_dump, JSONExtractRaw( inputs_dump, 'example'))
    • tr is the result of a LEFT JOIN on the row digest.
    • If the row digest is null, like in the case of imperatively logged evals, then Clickhouse fills the result with the default value, '', because val_dump is a non nullable string.

COALESCE will always return the empty string, which makes sorting and filtering broken on inputs.

To fix this, the PR adds nullIf calls on the val_dump so that the COALESCE can actually fallthrough to the example.

Testing

How was this PR tested?

  • Against dev frontend
  • unit tests updated

@domphan-wandb domphan-wandb requested a review from a team as a code owner May 29, 2026 00:26
@codecov
Copy link
Copy Markdown

codecov Bot commented May 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@domphan-wandb domphan-wandb force-pushed the dom/eval-filtering-on-io branch from b011963 to e374861 Compare May 29, 2026 01:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant