You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: paper.md
+14-15Lines changed: 14 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -72,27 +72,26 @@ It also computes mutual information reduction (MIR) and pairwise mutual informat
72
72
and the Amari distance [@amari1996new], a relabeling- and scale-invariant unmixing-matrix metric that needs no assignment step.
73
73
Both implementations were run for AMICA's usual 2000 iterations with Newton off (`do_newton=0`) and otherwise-default parameters
74
74
(settings transcribed between `pamica`'s JSON and Fortran's native text format).
75
-
Newton is disabled here to isolate the algorithm from initialization: with Newton on, independently seeded runs can settle a few under-determined components in different, equally likely optima, though a matched initialization recovers agreement (documentation).
76
-
The single-model headline (Table 1) uses an external recording (OpenNeuro ds002718, 70 channels) with a data-adequacy ratio $k\approx153$ ($k$ = frames divided by squared channel count), well past the ~60 threshold where cross-backend agreement plateaus;
77
-
the bundled 32-channel sample below ($k\approx30$) gives a consistent Amari distance.
75
+
Newton is disabled here to isolate the algorithm from initialization: with Newton on, independently seeded runs can settle a few under-determined components in different, equally likely optima, though a matched initialization recovers agreement ([documentation](https://eeglab.org/pAMICA/guides/validation/)).
76
+
The single-model comparison uses a well-determined external recording (OpenNeuro ds002718, $k\approx153$, where $k$ = frames over squared channel count), well past the ~60 threshold where cross-backend agreement plateaus, together with the bundled 32-channel sample ($k\approx30$); Table 1 gives each metric's dataset.
78
77
Score functions and per-block sufficient statistics are exact to floating-point resolution against the literal Fortran expressions on the bundled sample.
79
78
A mixture of ICA models is not partition-identifiable, so exact partition parity is the wrong bar for the multi-model case;
80
79
it is instead assessed by whether the two implementations sample a similar distribution of solutions, across ensembles of 20 runs each (\autoref{fig:ensemble}).
81
80
A permutation test finds no evidence that cross-implementation agreement is worse than Fortran's own run-to-run agreement.
82
81
Multi-model log-likelihood distributions still differ slightly at a matched iteration budget
83
82
(`pamica` needs about twice as many iterations to reach Fortran's mean), so full-likelihood similarity is not yet claimed.
84
83
85
-
| Regime | Metric | Result (mean) |
84
+
| Regime | Metric (dataset) | Result (mean) |
86
85
|---|---|---|
87
-
| Single-model | Log-likelihood gap to Fortran (mean per-sample-channel log-likelihood, $-3.6993$, $k\approx153$) | within ~0.0005 |
: Single-model parity (external ds002718, $k\approx153$; Amari and score-function checks on the bundled sample) and
94
-
multi-model distributional similarity (bundled sample) of `pamica` with the Fortran reference. All values are means
95
-
(sd = standard deviation, shown where relevant) over matched components (single-model) or within/cross-implementation pairs (190/400, multi-model).
94
+
: Parity of `pamica` with the Fortran reference. The two single-model conformity metrics use different recordings, hence the two data-adequacy ratios $k$: the correlation headline uses a well-determined external recording ($k\approx153$), while the Amari distance and score-function checks use the bundled 32-channel sample ($k\approx30$). Multi-model agreement is distributional, since a mixture of models is not partition-identifiable; the ensemble row is the mean difference between cross-implementation and within-Fortran agreement, with a run-level permutation $p$-value. Values are means (sd, standard deviation) over matched components or, for multi-model, over within/cross-implementation run pairs (190/400).
96
95
97
96
) shows cross-backend component agreement rising with frames per channel and plateauing near 0.98 once the decomposition is well-determined,
107
106
where two independent double-precision implementations agree at a mean of 0.995;
108
107
single-precision runs agree with float64 to four to five significant digits, so float64 stays the default for parity.
109
108
@@ -120,14 +119,14 @@ single-precision runs agree with float64 to four to five significant digits, so
120
119
: Single-model throughput on real 70-channel EEG (`n_mix`=3,
121
120
`pdftype`=0, `block_size`=512; warm, minimum of repeated runs).
122
121
CPU, MPS, and MLX on Apple Silicon; CUDA on a separate NVIDIA RTX 4090 (float32 comparable, ~36 ms).
123
-
The two native-Fortran rows are from a separate core-count sweep (documentation) at each backend's plateau;
122
+
The two native-Fortran rows are from a separate core-count sweep ([documentation](https://eeglab.org/pAMICA/guides/validation/)) at each backend's plateau;
124
123
the other CPU rows use platform-default threads and are not core-matched to Fortran.
125
124
Unlike the correctness comparison, this benchmark uses external data (OpenNeuro ds002718, one subject so far) and specific GPU hardware.
126
125
127
126
The correctness harness never uses synthetic data;
128
127
the multi-model and score-function checks need no external download (bundled sample only).
129
128
The full performance tables, per-run Amari-distance detail, data-size sweep,
130
-
and step-by-step reproduction commands are in the documentation (<https://eeglab.org/pAMICA/guides/validation/>).
129
+
and step-by-step reproduction commands are in the [documentation](https://eeglab.org/pAMICA/guides/validation/).
0 commit comments