Skip to content

export EvaluationModuleError and wrap _compute failures#774

Open
RavSinghChandan wants to merge 2 commits into
huggingface:mainfrom
RavSinghChandan:fix/export-evaluation-module-error
Open

export EvaluationModuleError and wrap _compute failures#774
RavSinghChandan wants to merge 2 commits into
huggingface:mainfrom
RavSinghChandan:fix/export-evaluation-module-error

Conversation

@RavSinghChandan

Copy link
Copy Markdown

Fixes #758.

EvaluationModuleError existed in module.py but was never exported from the public API, so raw sklearn/numpy exceptions like ValueError leaked out of EvaluationModule.compute() with no evaluate context in the traceback.

Two changes:

module.py — added EvaluationModuleError class (it did not exist before) and wrapped the _compute call to catch unexpected exceptions and re-raise them as EvaluationModuleError. Already-raised EvaluationModuleError instances pass through unchanged.

__init__.py — added EvaluationModuleError to the from .module import ... line so it is importable as evaluate.EvaluationModuleError.

After this change callers can write:

import evaluate

acc = evaluate.load("accuracy")
try:
    acc.compute(predictions=[], references=[])
except evaluate.EvaluationModuleError as e:
    print(f"metric failed: {e}")

No logic changes to how metrics are computed. Only the exception surface is affected.

Add Args/Returns/Raises/Example docstrings to filename_prefix_for_name,
filename_prefix_for_split, filepattern_for_dataset_split,
filename_for_dataset_split, filepath_for_dataset_split
EvaluationModuleError was defined internally but not exported, making it
impossible for callers to catch evaluate-specific errors without catching
broad Exception. Raw sklearn/numpy errors leaked out with no evaluate
context in the traceback.

- add EvaluationModuleError to module.py with a docstring example
- wrap _compute call to re-raise unexpected exceptions as EvaluationModuleError
- export EvaluationModuleError from evaluate/__init__.py

fixes huggingface#758
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

EvaluationModuleError not exported from public API — sklearn errors leak to users

1 participant