Skip to content

Commit d0989f1

Browse files
fix(evaluations): non-numeric score fold no longer aborts aggregation (#1096)
A single fold with a non-numeric score (e.g. an error fold returning "failed") made pandas infer the whole score column as object dtype, so groupby.agg(mean) raised TypeError and took down the entire evaluation. Coerce numeric aggregation columns with pd.to_numeric(errors="coerce") before agg so bad folds become NaN and are skipped by mean. Closes #1095
1 parent df789b8 commit d0989f1

3 files changed

Lines changed: 22 additions & 0 deletions

File tree

docs/source/whats_new.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@ Bugs
6868
- Cache Figshare's file listing in :func:`moabb.datasets.download.fs_get_file_list` (process-level ``lru_cache``) and persist it on disk next to the data for MAMEM (:class:`moabb.datasets.MAMEM1`/``MAMEM2``/``MAMEM3``). Once a dataset has been downloaded, subsequent calls never contact Figshare; pass ``force_update=True`` to bypass both layers (by `Bruno Aristimunha`_).
6969
- Fix Windows download path sanitization that changed absolute paths like ``C:\data`` into relative ``C-\data`` paths (:gh:`1079` by `Anton Andreev`_).
7070
- Fix missing electrode positions (NaN xyz) in six motor-imagery datasets so topographic maps, interpolation, and spatial methods work: :class:`moabb.datasets.Forenzo2023` and :class:`moabb.datasets.GuttmannFlury2025_MI`/``_ME`` normalize Neuroscan ALL_CAPS labels and apply ``standard_1005`` (CB1/CB2 kept as ``misc``); :class:`moabb.datasets.Dreyer2023` falls back to ``standard_1005`` when the BIDS archive ships no ``electrodes.tsv``; :class:`moabb.datasets.BNCI2003_004` maps its 26 legacy Berlin channel labels to their modern 10-5 equivalents for exact positions; :class:`moabb.datasets.BNCI2014_002` applies an approximate 3x5 grid for its unlabeled small-Laplacian channels; and :class:`moabb.datasets.Zhang2017` applies the ``GSN-HydroCel-32`` montage in EGI sensor order. Adds the shared :func:`moabb.datasets.utils.set_neuroscan_montage` helper (:gh:`1089` by `Bruno Aristimunha`_).
71+
- Fix ``BaseEvaluation._aggregate_fold_results`` aborting the whole evaluation with ``TypeError: agg function failed [how->mean,dtype->object]`` when a single fold contributes a non-numeric ``score`` (e.g. an error fold). The numeric aggregation columns are now coerced with ``pandas.to_numeric(errors="coerce")`` before ``groupby.agg``, so a bad fold becomes ``NaN`` and is skipped instead of taking down every subject/pipeline (:gh:`1095` by `Bruno Aristimunha`_).
7172
Code health
7273
~~~~~~~~~~~
7374
- Install CPU-only PyTorch wheels in CI by setting ``UV_TORCH_BACKEND=cpu`` in the test, braindecode, and docs workflows, so runners no longer download multi-GB CUDA builds of ``torch`` (pulled transitively via the ``deeplearning`` extra / braindecode) (:gh:`1083` by `Bhargav Kowshik`_).

moabb/evaluations/base.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -848,6 +848,13 @@ def _aggregate_fold_results(fold_results):
848848

849849
has_carbon = "carbon_emission" in df.columns
850850

851+
# A single error fold may put a non-numeric value (e.g. "failed") in an
852+
# aggregation column, making pandas infer the whole column as object and
853+
# mean() raise. Coerce so bad folds become NaN and are skipped by mean.
854+
for col in agg_ops:
855+
if df[col].dtype == object:
856+
df[col] = pd.to_numeric(df[col], errors="coerce")
857+
851858
grouped = df.groupby(group_keys, sort=False)
852859
agg_df = grouped.agg(agg_ops)
853860

moabb/tests/test_evaluations.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1034,3 +1034,17 @@ def test_error_folds_multi_metric_use_fallback(self):
10341034
np.testing.assert_almost_equal(res["score_accuracy"], 0.4)
10351035
# score_f1: avg(0.7, 0.0) = 0.35
10361036
np.testing.assert_almost_equal(res["score_f1"], 0.35)
1037+
1038+
def test_non_numeric_score_fold_does_not_abort(self):
1039+
"""A fold with a non-numeric score becomes NaN, not a fatal TypeError."""
1040+
from moabb.evaluations.base import BaseEvaluation
1041+
1042+
folds = [
1043+
self._make_fold(1, "0", "csp", 0.7),
1044+
# A degenerate/error fold may leave a non-numeric value in "score".
1045+
self._make_fold(1, "0", "csp", "failed", is_error=True),
1046+
]
1047+
agg = BaseEvaluation._aggregate_fold_results(folds)
1048+
assert len(agg) == 1
1049+
# Non-numeric fold is coerced to NaN and skipped by mean -> only 0.7 left.
1050+
np.testing.assert_almost_equal(agg[0]["score"], 0.7)

0 commit comments

Comments
 (0)