Skip to content

Commit 5b5fb3f

Browse files
author
Gereon Elvers
committed
Add OSFDownloadMixin
Mirrors OhanaDownloadMixin (ensure_file, prefetch_files, manifest, retries) for datasets hosted on the Open Science Framework. Public OSF projects need no auth; OSF_TOKEN env var supported for private projects. Multi-component projects (OSF storage caps at ~50 GB per node) are supported via OSF_PROJECT_FALLBACKS — the manifest aggregates files across all configured nodes with the primary winning on duplicates. Used by the upcoming MEG-MASC dataset, which spans four sibling nodes (ag3kj, h2tzn, u5327, dr4wy).
1 parent dfaf6e6 commit 5b5fb3f

2 files changed

Lines changed: 501 additions & 0 deletions

File tree

pnpl/datasets/mixins/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313

1414
from .download import HFDownloadMixin
1515
from .ohana_download import OhanaDownloadMixin
16+
from .osf_download import OSFDownloadMixin
1617
from .standardization import StandardizationMixin
1718
from .continuous_h5 import ContinuousH5Mixin
1819
from .epoched_h5 import EpochedH5Mixin
@@ -21,6 +22,7 @@
2122
__all__ = [
2223
"HFDownloadMixin",
2324
"OhanaDownloadMixin",
25+
"OSFDownloadMixin",
2426
"StandardizationMixin",
2527
"ContinuousH5Mixin",
2628
"EpochedH5Mixin",

0 commit comments

Comments
 (0)