Skip to content

Commit 9b90df2

Browse files
Clarify Table 1; hyperlink documentation refs
1 parent f902879 commit 9b90df2

1 file changed

Lines changed: 14 additions & 15 deletions

File tree

paper.md

Lines changed: 14 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -72,27 +72,26 @@ It also computes mutual information reduction (MIR) and pairwise mutual informat
7272
and the Amari distance [@amari1996new], a relabeling- and scale-invariant unmixing-matrix metric that needs no assignment step.
7373
Both implementations were run for AMICA's usual 2000 iterations with Newton off (`do_newton=0`) and otherwise-default parameters
7474
(settings transcribed between `pamica`'s JSON and Fortran's native text format).
75-
Newton is disabled here to isolate the algorithm from initialization: with Newton on, independently seeded runs can settle a few under-determined components in different, equally likely optima, though a matched initialization recovers agreement (documentation).
76-
The single-model headline (Table 1) uses an external recording (OpenNeuro ds002718, 70 channels) with a data-adequacy ratio $k\approx153$ ($k$ = frames divided by squared channel count), well past the ~60 threshold where cross-backend agreement plateaus;
77-
the bundled 32-channel sample below ($k\approx30$) gives a consistent Amari distance.
75+
Newton is disabled here to isolate the algorithm from initialization: with Newton on, independently seeded runs can settle a few under-determined components in different, equally likely optima, though a matched initialization recovers agreement ([documentation](https://eeglab.org/pAMICA/guides/validation/)).
76+
The single-model comparison uses a well-determined external recording (OpenNeuro ds002718, $k\approx153$, where $k$ = frames over squared channel count), well past the ~60 threshold where cross-backend agreement plateaus, together with the bundled 32-channel sample ($k\approx30$); Table 1 gives each metric's dataset.
7877
Score functions and per-block sufficient statistics are exact to floating-point resolution against the literal Fortran expressions on the bundled sample.
7978
A mixture of ICA models is not partition-identifiable, so exact partition parity is the wrong bar for the multi-model case;
8079
it is instead assessed by whether the two implementations sample a similar distribution of solutions, across ensembles of 20 runs each (\autoref{fig:ensemble}).
8180
A permutation test finds no evidence that cross-implementation agreement is worse than Fortran's own run-to-run agreement.
8281
Multi-model log-likelihood distributions still differ slightly at a matched iteration budget
8382
(`pamica` needs about twice as many iterations to reach Fortran's mean), so full-likelihood similarity is not yet claimed.
8483

85-
| Regime | Metric | Result (mean) |
84+
| Regime | Metric (dataset) | Result (mean) |
8685
|---|---|---|
87-
| Single-model | Log-likelihood gap to Fortran (mean per-sample-channel log-likelihood, $-3.6993$, $k\approx153$) | within ~0.0005 |
88-
| Single-model | Conformity with Fortran | 0.998 ($k\approx153$) / 0.006 (bundled, $k\approx30$) |
89-
| Single-model | Score functions (non-default families) and sufficient statistics | exact to floating-point resolution ($\sim\!10^{-15}$) |
90-
| Multi-model | Single-run magnitude (`pamica`-Fortran; Fortran-Fortran) | 0.65; 0.64 (sd 0.05) / 0.163; 0.174 (sd 0.02) |
91-
| Multi-model | Ensemble similarity, 20 runs each: mean difference, between $-$ within-Fortran (permutation $p$) | $+0.011$ ($p=0.96$) / $-0.011$ ($p>0.999$) |
86+
| Single-model | Log-likelihood gap to Fortran (ds002718, $k\approx153$) | within ~0.0005 of $-3.6993$ |
87+
| Single-model | Hungarian-matched component correlation (ds002718, $k\approx153$) | 0.998 |
88+
| Single-model | Amari distance (bundled 32-channel sample, $k\approx30$) | 0.006 |
89+
| Single-model | Score functions and sufficient statistics (bundled sample) | exact, $\sim\!10^{-15}$ |
90+
| Multi-model | Component correlation, single run: `pamica`-Fortran; Fortran-Fortran (bundled) | 0.65; 0.64 (sd 0.05) |
91+
| Multi-model | Amari distance, single run: `pamica`-Fortran; Fortran-Fortran (bundled) | 0.163; 0.174 (sd 0.02) |
92+
| Multi-model | Ensemble agreement, cross-implementation $-$ within-Fortran, 20 runs each (bundled) | correlation $+0.011$ ($p=0.96$); Amari $-0.011$ ($p>0.999$) |
9293

93-
: Single-model parity (external ds002718, $k\approx153$; Amari and score-function checks on the bundled sample) and
94-
multi-model distributional similarity (bundled sample) of `pamica` with the Fortran reference. All values are means
95-
(sd = standard deviation, shown where relevant) over matched components (single-model) or within/cross-implementation pairs (190/400, multi-model).
94+
: Parity of `pamica` with the Fortran reference. The two single-model conformity metrics use different recordings, hence the two data-adequacy ratios $k$: the correlation headline uses a well-determined external recording ($k\approx153$), while the Amari distance and score-function checks use the bundled 32-channel sample ($k\approx30$). Multi-model agreement is distributional, since a mixture of models is not partition-identifiable; the ensemble row is the mean difference between cross-implementation and within-Fortran agreement, with a run-level permutation $p$-value. Values are means (sd, standard deviation) over matched components or, for multi-model, over within/cross-implementation run pairs (190/400).
9695

9796
![Multi-model solution-ensemble partition-correlation distributions (panel A) and log-likelihood distributions (panel B) for 20 `pamica` and 20 Fortran fits of the sample EEG; dashed lines mark each distribution's mean.
9897
The within-Fortran, within-`pamica`, and between-implementation correlation distributions overlap,
@@ -103,7 +102,7 @@ All backends converge to the same single-model log-likelihood on real EEG (maxim
103102
On Apple Silicon, MLX is the fastest backend and flat with channel count (Table 2); PyTorch-MPS is never a win.
104103
Double-precision CUDA is the reproducible NVIDIA path; native Fortran scales with CPU cores and, with enough cores pinned,
105104
can beat the GPU on a larger, hotter host, though it does not match Apple's MLX on laptop hardware.
106-
A data-size sweep (documentation) shows cross-backend component agreement rising with frames per channel and plateauing near 0.98 once the decomposition is well-determined,
105+
A data-size sweep ([documentation](https://eeglab.org/pAMICA/guides/validation/)) shows cross-backend component agreement rising with frames per channel and plateauing near 0.98 once the decomposition is well-determined,
107106
where two independent double-precision implementations agree at a mean of 0.995;
108107
single-precision runs agree with float64 to four to five significant digits, so float64 stays the default for parity.
109108

@@ -120,14 +119,14 @@ single-precision runs agree with float64 to four to five significant digits, so
120119
: Single-model throughput on real 70-channel EEG (`n_mix`=3,
121120
`pdftype`=0, `block_size`=512; warm, minimum of repeated runs).
122121
CPU, MPS, and MLX on Apple Silicon; CUDA on a separate NVIDIA RTX 4090 (float32 comparable, ~36 ms).
123-
The two native-Fortran rows are from a separate core-count sweep (documentation) at each backend's plateau;
122+
The two native-Fortran rows are from a separate core-count sweep ([documentation](https://eeglab.org/pAMICA/guides/validation/)) at each backend's plateau;
124123
the other CPU rows use platform-default threads and are not core-matched to Fortran.
125124
Unlike the correctness comparison, this benchmark uses external data (OpenNeuro ds002718, one subject so far) and specific GPU hardware.
126125

127126
The correctness harness never uses synthetic data;
128127
the multi-model and score-function checks need no external download (bundled sample only).
129128
The full performance tables, per-run Amari-distance detail, data-size sweep,
130-
and step-by-step reproduction commands are in the documentation (<https://eeglab.org/pAMICA/guides/validation/>).
129+
and step-by-step reproduction commands are in the [documentation](https://eeglab.org/pAMICA/guides/validation/).
131130

132131
# State of the field
133132

0 commit comments

Comments
 (0)