Skip to content

fix: unify fuzzy_match metrics into shared module#71

Merged
abrichr merged 1 commit into
mainfrom
fix/unify-fuzzy-match-metrics
Mar 2, 2026
Merged

fix: unify fuzzy_match metrics into shared module#71
abrichr merged 1 commit into
mainfrom
fix/unify-fuzzy-match-metrics

Conversation

@abrichr
Copy link
Copy Markdown
Member

@abrichr abrichr commented Mar 2, 2026

Summary

  • Create evaluation/metrics.py as single source of truth for metric functions
  • client.py fallback now delegates to shared module (was word-set overlap, now rapidfuzz with fallback)
  • evaluate_endpoint.py StandaloneMetrics now wraps shared module
  • Add 15 tests for the shared metrics module

Resolves the eval path divergence documented in client.py and STATUS.md (Tier 2).

What changed

Metric Before (client.py) Before (endpoint) After (both)
fuzzy_match word-set intersection rapidfuzz + substring fallback rapidfuzz + substring fallback
exact_match actual == expected actual == expected + strip actual == expected + strip
contains case-insensitive substring case-insensitive substring case-insensitive substring

Test plan

  • All 26 existing evaluate_endpoint tests pass
  • All 15 new metrics tests pass
  • 507 total tests pass (22 pre-existing failures in test_setup_handlers.py)

🤖 Generated with Claude Code

Extract metric functions (exact_match, fuzzy_match, contains, boolean,
file_exists) into evaluation/metrics.py as the single source of truth.
Both client.py and evaluate_endpoint.py now delegate to this module,
eliminating the divergence between word-set overlap and rapidfuzz
implementations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@abrichr abrichr merged commit 2e1c547 into main Mar 2, 2026
1 check passed
@abrichr abrichr deleted the fix/unify-fuzzy-match-metrics branch March 2, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant