Adding PanLUNA to BioFoundation

README.md

@@ -13,6 +13,9 @@
   <a href="https://arxiv.org/abs/2603.19100">
     <img src="https://img.shields.io/badge/arXiv-2603.19100-b31b1b.svg" alt="LuMamba Paper">
   </a>
+  <a href="https://arxiv.org/pdf/2604.04297">
+    <img src="https://img.shields.io/badge/arXiv-2604.04297-b31b1b.svg" alt="PanLUNA Paper">
+  </a>
   <a href="https://huggingface.co/PulpBio/FEMBA">
     <img src="https://img.shields.io/badge/HuggingFace-FEMBA-%23ffcc4d?logo=huggingface&logoColor=black" alt="Hugging Face: FEMBA">
   </a>
@@ -25,14 +28,17 @@
   <a href="https://huggingface.co/PulpBio/LuMamba">
     <img src="https://img.shields.io/badge/HuggingFace-LuMamba-%23ffcc4d?logo=huggingface&logoColor=black" alt="Hugging Face: LuMamba">
   </a>
+  <a href="https://huggingface.co/PulpBio/PanLUNA">
+    <img src="https://img.shields.io/badge/HuggingFace-PanLUNA-%23ffcc4d?logo=huggingface&logoColor=black" alt="Hugging Face: LuMamba">
+  </a>
   <a href="https://github.com/pulp-bio/BioFoundation">
     <img src="https://img.shields.io/github/stars/pulp-bio/BioFoundation?style=social" alt="GitHub Stars">
   </a>
 </p>
 
 Copyright (C) 2025-2026 ETH Zurich, Switzerland. SPDX-License-Identifier: Apache-2.0. See LICENSE file for details.
 
-Authors: Thorir Mar Ingolfsson, Anna Tegon, Berkay Döner, Xiaying Wang, Matteo Fasulo, Danaé Broustail, Yawei Li & Luca Benini.
+Authors: Thorir Mar Ingolfsson, Anna Tegon, Berkay Döner, Xiaying Wang, Matteo Fasulo, Danaé Broustail, Marija Zelic, Yawei Li & Luca Benini.
 
 ## About
 
@@ -50,6 +56,7 @@ Looking for ready-to-use weights of models? We host them on Hugging Face:
 - **LUNA** ([paper](https://arxiv.org/abs/2510.22257)) [![HF Model Card](https://img.shields.io/badge/Model%20Card-LUNA-ffcc4d?logo=huggingface&logoColor=black)](https://huggingface.co/PulpBio/LUNA)
 - **TinyMyo** ([paper](https://arxiv.org/abs/2512.15729)) [![HF Model Card](https://img.shields.io/badge/Model%20Card-TinyMyo-ffcc4d?logo=huggingface&logoColor=black)](https://huggingface.co/PulpBio/TinyMyo)
 - **LuMamba** ([paper](https://arxiv.org/abs/2603.19100)) [![HF Model Card](https://img.shields.io/badge/Model%20Card-LuMamba-ffcc4d?logo=huggingface&logoColor=black)](https://huggingface.co/PulpBio/LuMamba)
+- **PanLUNA** ([paper](https://arxiv.org/pdf/2604.04297)) [![HF Model Card](https://img.shields.io/badge/Model%20Card-PanLUNA-ffcc4d?logo=huggingface&logoColor=black)](https://huggingface.co/PulpBio/PanLUNA)
 
 #### Why FEMBA?
 
@@ -227,6 +234,46 @@ Tips:
   - Ensure `data_module:train/test/val` are initialized with the compatible dataset class.
   - Configuration file includes sufficient `#CHANGEME` tags and further instructions for a working example.
 
+## Why PanLUNA?
+
+* Extending LUNA's channel-unification mechanism from topology invariance to **cross-modal fusion**, jointly processing EEG, ECG, and PPG within a single shared encoder via sensor-type embeddings - no modality-specific backbones, no paired multimodal data required during pretraining.
+* Pretrained on ~40,000 hours of heterogeneous biosignal data (TUEG, Siena, MIMIC-IV, CODE-15%, PulseDB) with a masked signal reconstruction objective.
+* Strong performance on unimodal and multimodal experiments.
+
+➡️ Model hub: __https://huggingface.co/PulpBio/PanLUNA__ 📄 Model card: __[PanLUNA on Hugging Face](https://huggingface.co/PulpBio/PanLUNA)__ — variants, configs, and fine-tuning walkthrough. 📜 Weights license: CC BY-ND 4.0 (use + redistribute unmodified weights with attribution; no redistribution of modified weights) 🧑‍🍳 PR-gated improvements: If you fine-tune internally and want your variant to become an official PanLUNA release, open a PR with configs, logs, and evals. We'll review; if it looks good, we'll retrain/validate and publish an official PanLUNA checkpoint.
+
+### What you'll find on the hub
+
+* Pre-trained checkpoint.
+* Instructions to get started on fine-tuning experiments.
+
+### Quick download with `huggingface_hub`:
+
+```
+pip install huggingface_hub
+```
+
+```python
+from huggingface_hub import snapshot_download
+
+# downloads all pre-trained variants and safetensors into ./checkpoints/PanLUNA
+snapshot_download(repo_id="PulpBio/PanLUNA", repo_type="model", local_dir="checkpoints/PanLUNA")
+```
+
+Include the safetensors checkpoint path as input and run fine-tuning in the commandline:
+
+```bash
+python -u run_train.py +experiment=PanLUNA_finetune \
+  pretrained_safetensors_path=/absolute/path/to/checkpoints/PanLUNA/PanLUNA.safetensors
+```
+
+### Tips:
+
+* Data preprocessing scripts are provided in `/make_datasets` for various downstream datasets. Download the corresponding dataset, locate its preprocessing script by name matching, and adjust key parameters.
+* Adapt configuration file `config/experiment/PanLUNA_finetune.yaml` to your specific task with correct data module (for unimodal experiments `config/data_module/finetune_data_module_unimodal_PanLUNA` or multimodal experiments `config/data_module/finetune_data_module_multimodal_PanLUNA`), classification type (binary `bc`, multi-class `mcc` and mulit-label `mlp`) and change `model.num_classes` accordingly. For different fine-tuning strategies adjust `finetuning.mode` parameter with `lora` for Low-Rank Adaptation, `freeze_encoder` for frozen backbone or select `full` for complete adaptation. 
+  * Ensure `data_module:train/test/val` are initialized with the compatible dataset class. Leverage `config/data_module/finetune_data_module_multimodal_PanLUNA:data_module.train.channel_groups` to specify channels available in the dataset and `channel_start`/`channel_end` to tweak the used subset.
+  * Configuration file includes sufficient `#CHANGEME` tags and further instructions for a working example.
+
 ## Features
 
 - **Modular Design**: The repository is organized into modules for data loading, models, training tasks, and more, making it easy to extend and adapt for new research projects.
@@ -455,6 +502,15 @@ If you find this work useful, please cite the respective papers:
       primaryClass={cs.AI},
       url={https://arxiv.org/abs/2603.19100}, 
 }
+@misc{zelic2026panluna,
+      title={PanLUNA: An Efficient and Robust Query-Unified Multimodal Model for Edge Biosignal Intelligence}, 
+      author={Marija Zelic and Anna Tegon and Yawei Li and Thorir Mar Ingolfsson},
+      year={2026},
+      eprint={2604.04297},
+      archivePrefix={arXiv},
+      primaryClass={cs.AI},
+      url={https://arxiv.org/abs/2604.04297}, 
+}
 ```
 
 ## License

config/data_module/dataset_types.yaml

@@ -0,0 +1,33 @@
+dataset_types: # both pretraining datasets and unimodal finetuning datasets
+  tueg:
+    channels: [FP1-F7, F7-T3, T3-T5, T5-O1, FP2-F8, F8-T4, T4-T6, T6-O2, T3-C3, C3-CZ, CZ-C4, C4-T4, FP1-F3, F3-C3, C3-P3, P3-O1, FP2-F4, F4-C4, C4-P4, P4-O2, A1-T3, T4-A2]
+    location_fn: "eeg"
+    sensor_type: 1
+  siena:
+    channels: [FP1, FP2, F3, C3, P3, O1, F7, T3, T5, FC1, FC5, CP1, CP5, F9, FZ, CZ, PZ, F4, C4, P4, O2, F8, T4, T6, FC2,FC6, CP2, CP6, F10]
+    location_fn: "eeg"
+    sensor_type: 1
+  code15:
+    channels: [I, II, III, AVR, AVL, AVF, V1, V2, V3, V4, V5, V6]
+    location_fn: "ecg"
+    sensor_type: 0
+  mimicIV: 
+    channels: [I, II, III, AVR, AVF, AVL, V1, V2, V3, V4, V5, V6]
+    location_fn: "ecg"
+    sensor_type: 0
+  pulsedb:
+    channels: [II, PPG]
+    location_fn: "ecg"
+    sensor_type: [0, 2]
+  ptbxl:
+    channels: [I, II, III, AVR, AVF, AVL, V1, V2, V3, V4, V5, V6]
+    location_fn: "ecg"
+    sensor_type: 0
+  csn:
+    channels: [I, II, III, AVR, AVL, AVF, V1, V2, V3, V4, V5, V6]
+    location_fn: "ecg"
+    sensor_type: 0
+  cpsc2018: 
+    channels: [I, II, III, AVR, AVL, AVF, V1, V2, V3, V4, V5, V6]
+    location_fn: "ecg"
+    sensor_type: 0

config/data_module/finetune_data_module_multimodal_PanLUNA.yaml

@@ -0,0 +1,109 @@
+#*----------------------------------------------------------------------------*
+#* Copyright (C) 2026 ETH Zurich, Switzerland                                 *
+#* SPDX-License-Identifier: Apache-2.0                                        *
+#*                                                                            *
+#* Licensed under the Apache License, Version 2.0 (the "License");            *
+#* you may not use this file except in compliance with the License.           *
+#* You may obtain a copy of the License at                                    *
+#*                                                                            *
+#* http://www.apache.org/licenses/LICENSE-2.0                                 *
+#*                                                                            *
+#* Unless required by applicable law or agreed to in writing, software        *
+#* distributed under the License is distributed on an "AS IS" BASIS,          *
+#* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.   *
+#* See the License for the specific language governing permissions and        *
+#* limitations under the License.                                             *
+#*                                                                            *
+#* Author:  Marija Zelic                                                      *
+#* Author:  Thorir Mar Ingolfsson                                             *
+#*----------------------------------------------------------------------------*
+# @package _global_
+# This is example for SEED-VII dataset
+# To adjust for other multimodal datasets replace channel_groups field with appropriate modalities and their channels
+# Adjust slicing with channel start and end indexes
+data_module:
+  _target_: data_module.finetune_data_module.FinetuneDataModule
+  name: "eeg"
+  cfg:
+    num_workers: ${num_workers}
+    batch_size: ${batch_size}
+  train:
+    _target_: 'datasets.finetuning_multimodal_datasets_PanLUNA.FinetuningMultimodal_Dataset'
+    hdf5_file: "#CHANGEME"
+    <<: &seed_channels
+      channel_groups:
+        eeg:
+          - FP1 
+          - FPZ
+          - FP2
+          - AF3
+          - AF4
+          - F7
+          - F5
+          - F3
+          - F1
+          - FZ
+          - F2
+          - F4
+          - F6
+          - F8
+          - FT7
+          - FC5
+          - FC3
+          - FC1
+          - FCZ 
+          - FC2
+          - FC4 
+          - FC6 
+          - FT8 
+          - T7
+          - C5
+          - C3
+          - C1
+          - CZ
+          - C2
+          - C4
+          - C6
+          - T8
+          - TP7
+          - CP5
+          - CP3
+          - CP1
+          - CPZ
+          - CP2
+          - CP4
+          - CP6
+          - TP8
+          - P7
+          - P5
+          - P3
+          - P1
+          - PZ
+          - P2
+          - P4
+          - P6
+          - P8
+          - PO7
+          - PO5
+          - PO3
+          - POZ
+          - PO4
+          - PO6
+          - PO8
+          - CB1
+          - O1
+          - OZ
+          - O2
+          - CB2
+        ecg:
+          - II
+      channel_start: 0 # change for fraction of channels
+      channel_end: 63
+  val:
+    _target_: 'datasets.finetuning_multimodal_datasets_PanLUNA.FinetuningMultimodal_Dataset'
+    hdf5_file: "#CHANGEME"
+    <<: *seed_channels
+  test: 
+    _target_: 'datasets.finetuning_multimodal_datasets_PanLUNA.FinetuningMultimodal_Dataset'
+    hdf5_file: "#CHANGEME"
+    <<: *seed_channels

config/data_module/finetune_data_module_unimodal_PanLUNA.yaml

@@ -0,0 +1,49 @@
+#*----------------------------------------------------------------------------*
+#* Copyright (C) 2026 ETH Zurich, Switzerland                                 *
+#* SPDX-License-Identifier: Apache-2.0                                        *
+#*                                                                            *
+#* Licensed under the Apache License, Version 2.0 (the "License");            *
+#* you may not use this file except in compliance with the License.           *
+#* You may obtain a copy of the License at                                    *
+#*                                                                            *
+#* http://www.apache.org/licenses/LICENSE-2.0                                 *
+#*                                                                            *
+#* Unless required by applicable law or agreed to in writing, software        *
+#* distributed under the License is distributed on an "AS IS" BASIS,          *
+#* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.   *
+#* See the License for the specific language governing permissions and        *
+#* limitations under the License.                                             *
+#*                                                                            *
+#* Author:  Marija Zelic                                                      *
+#* Author:  Thorir Mar Ingolfsson                                             *
+#*----------------------------------------------------------------------------*
+# @package _global_
+defaults:
+  - dataset_types
+
+data_module:
+  _target_: data_module.finetune_data_module.FinetuneDataModule
+  cfg:
+    num_workers: ${num_workers}
+    batch_size: ${batch_size}
+  train:
+    _target_: 'datasets.finetuning_unimodal_datasets_PanLUNA.FinetuningUnimodal_Dataset'
+    hdf5_file: "#CHANGEME"
+    num_channels: 12 # example
+    channels: ${dataset_types.csn.channels} # example
+    location_fn: ${dataset_types.csn.location_fn}
+    sensor_type: ${dataset_types.csn.sensor_type}
+  val:
+    _target_: 'datasets.finetuning_unimodal_datasets_PanLUNA.FinetuningUnimodal_Dataset'
+    hdf5_file: "#CHANGEME"
+    num_channels: 12
+    channels: ${dataset_types.csn.channels}
+    location_fn: ${dataset_types.csn.location_fn}
+    sensor_type: ${dataset_types.csn.sensor_type}
+  test: 
+    _target_: 'datasets.finetuning_unimodal_datasets_PanLUNA.FinetuningUnimodal_Dataset'
+    hdf5_file: "#CHANGEME"
+    num_channels: 12
+    channels: ${dataset_types.csn.channels}
+    location_fn: ${dataset_types.csn.location_fn}
+    sensor_type: ${dataset_types.csn.sensor_type}