Skip to content

Allow a metadata-aware splitter as cv_class (used directly, not wrapped as the inner groups-CV) #1104

Description

@bruAristimunha

Summary

CrossSubjectEvaluation (and the CrossSubjectSplitter/CrossSessionSplitter wrappers)
always treat cv_class as the inner CV: they instantiate it and call
split(X=index, y, [groups=subject]), forwarding only the subject column. There is no
public way to plug in a metadata-aware top-level splitter — one that needs split(y, metadata)
and other columns (e.g. session) to build the fold.

Why this matters

We want a single-target cross-subject fold that restricts the test set to one session while
excluding the target's other sessions from training (test = target@session, train = other
subjects). That decision needs the session column, which the inner-CV path never passes.

A splitter that already follows the moabb convention (subclasses BaseCrossValidator, declares
metadata_columns = ("subject", "session"), implements split(self, y, metadata)) cannot be
passed via cv_class:

import pandas as pd
from moabb.evaluations.splitters import CrossSubjectSplitter
# MySplitter: BaseCrossValidator with metadata_columns=("subject","session") and split(self, y, metadata)
md = pd.DataFrame({"subject": [1, 1, 2, 2], "session": ["0", "1", "0", "1"]})
list(CrossSubjectSplitter(cv_class=MySplitter, target=2, test_session="1").split(y=None, metadata=md))
# TypeError: MySplitter.split() got an unexpected keyword argument 'X'

Current workaround

Subclass the evaluation and override _create_splitter to use cv_class directly:

class TargetSubjectEvaluation(CrossSubjectEvaluation):
    def _create_splitter(self):
        return self.cv_class(**self.cv_kwargs) if self.cv_class else super()._create_splitter()

Proposed

The evaluation already inspects cv_class.metadata_columns in __init__ (to extend
additional_columns). Extend that: when cv_class declares metadata_columns, use it directly
as the top-level splitter — call split(y, metadata) — instead of wrapping it as the inner
groups-CV. Backward compatible (group CVs without metadata_columns keep wrapping), and it lets
metadata-driven folds be configured through the public cv_class / cv_kwargs API with no subclass.

Happy to open a PR if this direction sounds good.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions