Summary
CrossSubjectEvaluation (and the CrossSubjectSplitter/CrossSessionSplitter wrappers)
always treat cv_class as the inner CV: they instantiate it and call
split(X=index, y, [groups=subject]), forwarding only the subject column. There is no
public way to plug in a metadata-aware top-level splitter — one that needs split(y, metadata)
and other columns (e.g. session) to build the fold.
Why this matters
We want a single-target cross-subject fold that restricts the test set to one session while
excluding the target's other sessions from training (test = target@session, train = other
subjects). That decision needs the session column, which the inner-CV path never passes.
A splitter that already follows the moabb convention (subclasses BaseCrossValidator, declares
metadata_columns = ("subject", "session"), implements split(self, y, metadata)) cannot be
passed via cv_class:
import pandas as pd
from moabb.evaluations.splitters import CrossSubjectSplitter
# MySplitter: BaseCrossValidator with metadata_columns=("subject","session") and split(self, y, metadata)
md = pd.DataFrame({"subject": [1, 1, 2, 2], "session": ["0", "1", "0", "1"]})
list(CrossSubjectSplitter(cv_class=MySplitter, target=2, test_session="1").split(y=None, metadata=md))
# TypeError: MySplitter.split() got an unexpected keyword argument 'X'
Current workaround
Subclass the evaluation and override _create_splitter to use cv_class directly:
class TargetSubjectEvaluation(CrossSubjectEvaluation):
def _create_splitter(self):
return self.cv_class(**self.cv_kwargs) if self.cv_class else super()._create_splitter()
Proposed
The evaluation already inspects cv_class.metadata_columns in __init__ (to extend
additional_columns). Extend that: when cv_class declares metadata_columns, use it directly
as the top-level splitter — call split(y, metadata) — instead of wrapping it as the inner
groups-CV. Backward compatible (group CVs without metadata_columns keep wrapping), and it lets
metadata-driven folds be configured through the public cv_class / cv_kwargs API with no subclass.
Happy to open a PR if this direction sounds good.
Summary
CrossSubjectEvaluation(and theCrossSubjectSplitter/CrossSessionSplitterwrappers)always treat
cv_classas the inner CV: they instantiate it and callsplit(X=index, y, [groups=subject]), forwarding only thesubjectcolumn. There is nopublic way to plug in a metadata-aware top-level splitter — one that needs
split(y, metadata)and other columns (e.g.
session) to build the fold.Why this matters
We want a single-target cross-subject fold that restricts the test set to one session while
excluding the target's other sessions from training (test = target@session, train = other
subjects). That decision needs the
sessioncolumn, which the inner-CV path never passes.A splitter that already follows the moabb convention (subclasses
BaseCrossValidator, declaresmetadata_columns = ("subject", "session"), implementssplit(self, y, metadata)) cannot bepassed via
cv_class:Current workaround
Subclass the evaluation and override
_create_splitterto usecv_classdirectly:Proposed
The evaluation already inspects
cv_class.metadata_columnsin__init__(to extendadditional_columns). Extend that: whencv_classdeclaresmetadata_columns, use it directlyas the top-level splitter — call
split(y, metadata)— instead of wrapping it as the innergroups-CV. Backward compatible (group CVs without
metadata_columnskeep wrapping), and it letsmetadata-driven folds be configured through the public
cv_class/cv_kwargsAPI with no subclass.Happy to open a PR if this direction sounds good.