phase2: amica + dipfit (code + CI wiring; data run in flight)#31
Merged
Conversation
Adds Phase 2 (AMICA + dipfit) on top of the Phase 1 + CI work already
on main. Smoke test wiring is covered by CI against the fixture; the
science-quality 3-subject derivatives are produced by a separate runner
against the full local R3 data (scripts/run_phase2_three_subjects).
src/matlab/phase2_amica.m (new entrypoint)
- Per-subject pipeline: load Phase 1 .set, run AMICA, attach weights,
fit dipoles, render IC topos + dipole PNGs, save checkpoint, write
qa_amica.csv row.
- MaxThreads defaults to feature('numcores') so the same code runs on
a 12-core Mac and a 2-core Actions runner.
src/matlab/+hbn/ helpers
- run_amica.m : wraps runamica15. Passes EEG.datfile (.fdt path) when
available; passing the EEG struct directly corrupts
argument parsing in the Fortran binary (forrtl-66
overflow on macOS). use_queue flag dropped (rejected
by this version with 'unknown flag').
- fit_dipoles.m: pop_dipfit_settings with standard MNI BEM + pop_multifit.
- save_ic_topo_figure.m : first N=30 IC topographies PNG.
- save_dipole_figure.m : pop_dipplot top view PNG.
- write_qa_amica_csv.m : participant_id, status, n_components,
final_ll, iterations, reached_max_iter,
median_rv, num_models, max_threads,
duration_s, error_message.
tests/matlab/test_phase2_smoke.m
- Chains phase1 -> phase2 on the 60 s fixture. MaxIter=100,
DoReject=false because the smoke test asserts wiring, not science.
Header warns against interpreting RV from this test.
tests/matlab/run_all_tests.m
- Adds test_phase2_smoke (5 tests total, all green locally).
.github/workflows/matlab-tests.yml
- Auto-installs Biosig, amica, Fieldtrip-lite. Cache key bumped to
v2-amica. timeout raised from 30 to 45 min.
scripts/run_phase2_three_subjects.m
- Standalone runner. Regenerates Phase 1 .set checkpoints if missing
(they are gitignored), then runs phase2 with MaxIter=2000, DoReject=
true on the full R3 data per the brief.
Refs #1, #3.
CI revealed a file-type detection mismatch in runamica15 on Linux: 'Need num_chans keyword argument with number of channels, unknown file type'. The .fdt path worked locally (macOS AMICA 1.7) but the CI build does not recognize the .fdt extension and would need num_chans passed alongside. The matrix path has no such ambiguity and the save-a-copy benefit of .fdt is negligible at our channel counts and recording durations. Drop the branch; keep the comment that documents why we never pass the EEG struct directly (the original macOS forrtl-66 crash). Refs #3.
The AMICA plugin ships amica15ex on Linux; that binary dynamically links against MPICH's Fortran library libmpifort.so.12. Ubuntu runners do not have it preinstalled, so AMICA fails immediately with 'error while loading shared libraries: libmpifort.so.12'. Add an apt step before MATLAB starts to install libmpich-dev. Refs #3.
CI cannot run AMICA reliably: the precompiled Linux binary amica15ex
needs MPI runtime libs that Ubuntu runners do not have in a known-good
combination (libmpifort.so.12 via libmpich-dev did not resolve cleanly).
Rather than fight library versions on every Ubuntu image bump, switch
the CI smoke test to Infomax (runica), which is bundled with EEGLAB
core and runs in pure MATLAB. Production runs keep using AMICA.
phase2_amica.m
- New opt IcaMethod (default 'amica', mustBeMember={'amica','runica'}).
Dispatches to hbn.run_amica (production) or hbn.run_infomax (CI/test).
Downstream dipfit + figures + checkpoint code does not branch on
method because both helpers populate EEG.icaweights / icasphere /
icawinv the same way.
- IcaMethod recorded in params.json so the QA agent can read it.
src/matlab/+hbn/run_infomax.m (new)
- Thin wrapper around pop_runica('icatype','runica'). Sets max_iter
from opts.MaxIter; reports stats with NaN final_ll/iterations
(Infomax does not expose a log-likelihood like AMICA does).
src/matlab/+hbn/write_qa_amica_csv.m
- Add ica_method column so a row's IcaMethod is recoverable from the
CSV alone (status='ok' + ica_method='runica' is the CI smoke shape;
ica_method='amica' is the production shape).
tests/matlab/test_phase2_smoke.m
- Sets IcaMethod='runica'.
- Skips the finite-final_ll / iterations assertions when ica_method
is not amica (Infomax does not expose those QA fields).
- requiredFields list includes IcaMethod.
Local: 5/5 tests green with runica on 60s fixture. Production 3-subject
runner (scripts/run_phase2_three_subjects) still uses the AMICA default.
Refs #3.
3-subject Phase 2 results plus the figure-export bug that the eeg-qa-neuroscientist agent caught. derivatives/amica/ (tracked via .gitignore allow-list) - README.md, qa_amica.csv, params.json - sub-<ID>/figures/sub-<ID>_ic-topos.png (first 30 ICs) - sub-<ID>/figures/sub-<ID>_dipoles.png (top view) Channel-count consistency: n_components matches Phase 1 n_channels_after for all 3 subjects (123 / 112 / 98). Per-subject AMICA + dipfit results: sub-NDARAA948VFH: 123 ICs, final_ll=-1.481, iter=2000, RV=0.368 sub-NDARAC853DTE: 112 ICs, final_ll=-2.221, iter=2000, RV=0.420 sub-NDARAD774HAZ: 98 ICs, final_ll=-2.512, iter=2000, RV=0.368 All 3 hit reached_max_iter=true. Expected at single-task data size. Subjects on pilot watch list; Phase 3 ICLabel filters. Figure-export bug (caught by eeg-qa-neuroscientist on the first pass): - save_ic_topo_figure.m and save_dipole_figure.m pre-allocated a figure handle and passed it to exportgraphics, but pop_topoplot and pop_dipplot open THEIR OWN figures. The pre-allocated handle stayed empty; exportgraphics wrote a blank canvas (byte-identical PNGs across subjects, the smoking gun). - Both helpers now snapshot figure handles before/after the pop_* call and capture the new figure for export. - save_ic_topo_figure drops sgtitle: it occluded ICs 1-5 (the highest-variance, most diagnostic components). pop_topoplot's own title is enough. - save_dipole_figure turns off axes in the captured figure to reduce pop_dipplot's axis bleed. scripts/regenerate_phase2_figures.m (new): rebuilds figures from existing .set checkpoints. Used to recover from the empty-canvas bug without redoing the 75 min of AMICA + dipfit. .gitignore: derivatives/**/_amica/ + tmpflist denylisted (AMICA binary outputs and scratch are heavy and gitignored). Refs #3, refs #1.
This was referenced May 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 2 of the HBN ERSP epic: AMICA decomposition + dipfit5 dipole localization, with a pluggable ICA dispatcher so CI (Ubuntu, no MPI libs) can run Infomax while production (local macOS, full MPI) uses AMICA.
Refs epic #1, closes #3.
What landed
Code
src/matlab/phase2_amica.m— entrypoint withIcaMethodopt (amicaorrunica, defaultamica). Per-subject: load Phase 1.set-> ICA -> attach weights -> dipfit -> figures -> AMICA checkpoint ->qa_amica.csvrow.src/matlab/+hbn/run_amica.m— wrapsrunamica15. PassesEEG.data(:,:)(matrix path); passing the struct or the.fdtpath either crashed the macOS binary (forrtl-66 in arg parsing) or tripped Linux file-type detection ("unknown file type"). Matrix path works everywhere.src/matlab/+hbn/run_infomax.m— runspop_runica('icatype','runica'). Used by the CI smoke test because AMICA's Linux binary needs MPI Fortran runtime that Ubuntu runners do not have in a stable combination.src/matlab/+hbn/fit_dipoles.m—pop_dipfit_settings(standard MNI BEM) +pop_multifit.src/matlab/+hbn/save_ic_topo_figure.m,save_dipole_figure.m— first 30 IC topographies + dipole top view.src/matlab/+hbn/write_qa_amica_csv.m— schema:participant_id, status, ica_method, n_components, final_ll, iterations, reached_max_iter, median_rv, num_models, max_threads, duration_s, error_message.scripts/run_phase2_three_subjects.m— production 3-subject runner.scripts/regenerate_phase2_figures.m— rebuilds figures from existing AMICA.setcheckpoints (used after the figure-export bug).tests/matlab/test_phase2_smoke.m— chains Phase 1 -> Phase 2 on the fixture withIcaMethod=runica,MaxIter=100,DoReject=false.CI workflow
.github/workflows/matlab-tests.ymlauto-installs Biosig, amica, Fieldtrip-lite. AMICA isn't usable on the runner anyway (libmpifort.so.12); install kept for parity / future probing. Cache keyeeglab-2026.0.0-v2-amica. Timeout 45 min.Derivatives
derivatives/amica/(tracked via the.gitignoreallow-list):README.md,qa_amica.csv,params.jsonsub-<ID>/figures/*.png(6 total)derivatives/**/_amica/(heavy AMICA model binaries) andtmpflist(AMICA scratch) added to denylist.3-subject AMICA results
n_componentsmatches Phase 1n_channels_afterfor every subject.Czis rejected upstream and absent from the ICA. AMICA hit the iteration ceiling on all 3 — expected at this data size (~210 s × 100 Hz vs 30×N² floor for stable ICA at 120 channels). Reference pipeline mitigates by concatenating tasks; we intentionally processThePresentonly.eeg-qa-neuroscientist review
Two passes; both attached for transparency.
Pass 1 verdict: FAIL
Root cause:
save_ic_topo_figureandsave_dipole_figurepre-allocated a figure handle and passed it toexportgraphics. Butpop_topoplotandpop_dipplotopen their own figures; the pre-allocated handle stayed empty. Both helpers now snapshot handles before/after thepop_*call and capture the new figure.Pass 2 verdict: PASS-WITH-FINDINGS
Critical: IC 1-5 in the topo montage were occluded by my added
sgtitle. Droppedsgtitle;pop_topoplot's own title arg is sufficient. Verified by reading the regeneratedsub-NDARAA948VFH_ic-topos.png— IC 1 through 30 all visible.Important (not blocking): tail ICs (beyond ~15) look noisy; consistent with AMICA's documented under-convergence. Phase 3 ICLabel filters at the locked 0.69 brain threshold. Dipole figures show
pop_dipplot's MRI volume render pinstripe pattern; helper now hides axes, residual pinstripe ispop_dipplot's own rendering style and acknowledged as cosmetic.Test plan
test_list_eligible_subjects,test_write_qa_channels_csv,test_phase1_no_subjects,test_phase1_smoke,test_phase2_smoke).IcaMethod=amica,MaxIter=2000. All 3 ok; channel counts consistent with Phase 1.matlab-testsworkflow green (usesIcaMethod=runica).Follow-up
Phase 3 ICLabel will pick up where this leaves off. AMICA non-convergence + RV findings stay on the subject watch list per
.context/research.mdL92-94 escalation path.