Skip to content

Commit 84cce91

Browse files
Merge pull request #31 from OpenScience-Collective/feature/issue-3-phase2-amica
phase2: amica + dipfit (code + CI wiring; data run in flight)
2 parents 387af54 + e019e3f commit 84cce91

22 files changed

Lines changed: 992 additions & 14 deletions

.github/workflows/matlab-tests.yml

Lines changed: 25 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
matlab-tests:
2121
name: matlab unit + smoke tests
2222
runs-on: ubuntu-latest
23-
timeout-minutes: 30
23+
timeout-minutes: 45
2424

2525
env:
2626
# Pin EEGLAB to a tagged release so CI is reproducible across runs.
@@ -44,7 +44,7 @@ jobs:
4444
uses: actions/cache@v4
4545
with:
4646
path: ${{ env.EEGLAB_DIR }}
47-
key: eeglab-${{ env.EEGLAB_TAG }}-${{ runner.os }}-v1
47+
key: eeglab-${{ env.EEGLAB_TAG }}-${{ runner.os }}-v2-amica
4848

4949
- name: Clone EEGLAB (cache miss)
5050
if: steps.cache-eeglab.outputs.cache-hit != 'true'
@@ -53,6 +53,18 @@ jobs:
5353
git clone --depth 1 --branch "$EEGLAB_TAG" --recurse-submodules \
5454
https://github.com/sccn/eeglab.git "$EEGLAB_DIR"
5555
56+
- name: Install MPI runtime for AMICA Linux binary
57+
# The AMICA plugin ships amica15ex (Linux Fortran binary) which
58+
# dynamically links against MPICH's Fortran library libmpifort.so.12.
59+
# Ubuntu runners do not have it preinstalled; without it AMICA
60+
# fails at "error while loading shared libraries: libmpifort.so.12".
61+
# libmpich-dev pulls the runtime plus headers; runtime-only is also
62+
# fine but the -dev package resolves regardless of Ubuntu version.
63+
run: |
64+
set -euo pipefail
65+
sudo apt-get update
66+
sudo apt-get install -y libmpich-dev
67+
5668
- uses: matlab-actions/setup-matlab@v2
5769
with:
5870
release: R2024a
@@ -63,16 +75,17 @@ jobs:
6375
command: |
6476
addpath('${{ env.EEGLAB_DIR }}');
6577
evalc('eeglab nogui');
66-
% Biosig is required for BDF I/O and is not bundled with EEGLAB.
67-
% plugin_askinstall runs headless and pulls the plugin into
68-
% EEGLAB_DIR/plugins/, where eeglab's path setup finds it on the
69-
% next call. The Biosig install is what enables pop_biosig and
70-
% pop_writeeeg.
71-
try
72-
plugin_askinstall('Biosig', [], true);
73-
catch ME
74-
warning('hbn:ci:biosig_install', ...
75-
'Biosig auto-install failed: %s', ME.message);
78+
% Plugins not bundled with EEGLAB are auto-installed here via
79+
% plugin_askinstall (headless). Biosig is needed for BDF I/O;
80+
% amica is needed for Phase 2 ICA decomposition; Fieldtrip-lite
81+
% is required by pop_dipfit_settings for dipole fitting.
82+
for plugin = {'Biosig', 'amica', 'Fieldtrip-lite'}
83+
try
84+
plugin_askinstall(plugin{1}, [], true);
85+
catch ME
86+
warning('hbn:ci:plugin_install', ...
87+
'%s auto-install failed: %s', plugin{1}, ME.message);
88+
end
7689
end
7790
evalc('eeglab nogui'); % re-init to pick up newly installed plugins
7891
addpath(genpath(fullfile(pwd, 'src', 'matlab')));

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ derivatives/**/*.txt
2121
derivatives/**/*.log
2222
derivatives/**/scratch*
2323
derivatives/**/_bids_import_scratch/
24+
derivatives/**/_amica/
2425
!derivatives/**/*.png
2526
!derivatives/**/qa_*.csv
2627
!derivatives/**/params.json
@@ -66,3 +67,4 @@ __pycache__/
6667
# and epic-dev state. Only the per-user permission file is private.
6768
.claude/settings.local.json
6869
logs/
70+
tmpflist

derivatives/amica/README.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Phase 2 AMICA: 3-subject re-run
2+
3+
AMICA decomposition + dipfit5 dipole localization for the same 3 Healthy Brain Network (HBN) Release 3 (R3) subjects processed by Phase 1 (`derivatives/preproc/`). Generated by `phase2_amica()` invoked from `scripts/run_phase2_three_subjects.m` on 2026-05-13.
4+
5+
## What is tracked in git
6+
7+
```
8+
derivatives/amica/
9+
README.md (this file)
10+
qa_amica.csv per-subject QA table, 3 rows
11+
params.json full parameter snapshot + tool versions
12+
sub-<ID>/figures/
13+
sub-<ID>_ic-topos.png first 30 independent component (IC) topographies
14+
sub-<ID>_dipoles.png dipole montage (top view)
15+
```
16+
17+
## What is NOT tracked
18+
19+
Heavy AMICA checkpoints and intermediate files stay out of git (`.set`, `.fdt`, `.mat`, plus the `_amica/` model directory per subject which contains the AMICA `W`, `S`, `LL`, `mods` binaries). To regenerate them, see the reproduce section.
20+
21+
```
22+
sub-<ID>/eeg/sub-<ID>_task-ThePresent_desc-amica_eeg.set (ignored)
23+
sub-<ID>/eeg/sub-<ID>_task-ThePresent_desc-amica_eeg.fdt (ignored)
24+
sub-<ID>/_amica/ (ignored)
25+
```
26+
27+
## Subjects in this run
28+
29+
| Participant | n_components | iterations | reached_max_iter | median_rv | wall-time (s) |
30+
| ---------------- | ------------ | ---------- | ---------------- | --------- | ------------- |
31+
| sub-NDARAA948VFH | 123 | 2000 | true | 0.368 | 1891 (~32 min) |
32+
| sub-NDARAC853DTE | 112 | 2000 | true | 0.420 | 1313 (~22 min) |
33+
| sub-NDARAD774HAZ | 98 | 2000 | true | 0.368 | 1156 (~19 min) |
34+
35+
`n_components` matches the post-rejection channel count from Phase 1 (no Principal Component Analysis (PCA) reduction was applied; AMICA decomposes at full rank). `Cz` is rejected upstream so it does not appear here.
36+
37+
### Known QA findings on this pilot
38+
39+
- **All 3 subjects hit `reached_max_iter=true` at iteration 2000.** AMICA did not reach the log-likelihood convergence criterion within the budget. This is expected at the data-size we have: ~210 s × 100 Hz = 21,000 samples for ~120 channels, which falls below the conventional 30 × N² floor for stable Independent Component Analysis (ICA). The reference pipeline concatenates several HBN tasks to enlarge the sample pool; this project intentionally processes `ThePresent` only, so under-convergence is structural.
40+
- **Median residual variance (RV) is 0.37 to 0.42 across subjects.** Above the agent's "interesting" floor (suspect when >0.50) but well above the ideal <0.30 for clean brain ICs. Read alongside the topographies: the first ~20 ICs per subject look component-like, the tail looks noisy.
41+
- **Subjects are kept on the pilot watch list, not dropped.** Phase 3 (ICLabel) will decide which ICs survive the brain-threshold filter. If a subject ends up with fewer than 5 brain ICs at the locked threshold of 0.69, it gets dropped from group stats (per `.context/research.md` L92-94 escalation path).
42+
43+
## Parameters
44+
45+
| Parameter | Value |
46+
| -------------------- | ---------------------------------------------------- |
47+
| Task | `ThePresent` |
48+
| Preproc input | `derivatives/preproc/sub-*/eeg/*_desc-clean_eeg.set` |
49+
| Independent Component Analysis method | `amica` |
50+
| AMICA models | 1 |
51+
| AMICA max iterations | 2000 |
52+
| AMICA threads | 14 (host has 14 logical cores) |
53+
| Time-point rejection | enabled, 5 rejection iterations, 4 sigma threshold |
54+
| Head model | EEGLAB standard MNI Boundary Element Model |
55+
| Dipole fit | `pop_multifit`, RV threshold 100 (keep everything) |
56+
57+
See `params.json` for the canonical snapshot plus MATLAB / EEGLAB versions, git SHA, and ISO timestamp.
58+
59+
## Reproduce
60+
61+
```bash
62+
# From the repo root
63+
matlab -batch "addpath('scripts'); run_phase2_three_subjects()"
64+
```
65+
66+
The runner regenerates Phase 1 `.set` checkpoints automatically if they are missing (the heavy checkpoints are gitignored). Wall-time on the local 14-core machine: ~75 minutes total for AMICA + dipfit across 3 subjects.
67+
68+
## Per-phase QA review
69+
70+
The `eeg-qa-neuroscientist` agent reviews these figures, the QA Comma-Separated Values table, and `params.json` before the PR opens. Findings live in the PR description, not here.

derivatives/amica/params.json

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"PreprocDir": "/Users/yahya/Documents/git/osc/hbn-phase2/derivatives/preproc",
3+
"OutDir": "/Users/yahya/Documents/git/osc/hbn-phase2/derivatives/amica",
4+
"Task": "ThePresent",
5+
"Subjects": [],
6+
"NumModels": 1,
7+
"MaxIter": 2000,
8+
"MaxThreads": 14,
9+
"DoReject": true,
10+
"NumRej": 5,
11+
"RejSig": 4,
12+
"NumTopoICs": 30,
13+
"EeglabRoot": "",
14+
"n_subjects_input": 3,
15+
"n_subjects_ok": 3,
16+
"n_subjects_failed": 0,
17+
"eeglab_root": "/Users/yahya/Documents/git/eeg/eeglab",
18+
"matlab_version": "25.2.0.3055257 (R2025b) Update 2",
19+
"eeglab_version": "dev",
20+
"git_sha": "4b6f73642d4eb8016d7c54e3aee50e0303f1d936",
21+
"timestamp_iso": "2026-05-13T07:53:41Z"
22+
}

derivatives/amica/qa_amica.csv

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
participant_id,status,ica_method,n_components,final_ll,iterations,reached_max_iter,median_rv,num_models,max_threads,duration_s,error_message
2+
sub-NDARAA948VFH,ok,amica,123,-1.481,2000,true,0.3682,1,14,1891,
3+
sub-NDARAC853DTE,ok,amica,112,-2.221,2000,true,0.4204,1,14,1313,
4+
sub-NDARAD774HAZ,ok,amica,98,-2.512,2000,true,0.3682,1,14,1156,
253 KB
Loading
382 KB
Loading
248 KB
Loading
389 KB
Loading
241 KB
Loading

0 commit comments

Comments
 (0)