Skip to content

Commit 68b0a7e

Browse files
torwagerclaude
andauthored
Docs: document the matlab.unittest test harness in CLAUDE.md (#83)
* CLAUDE.md: document the matlab.unittest harness and CI The Tests section still claimed "there is no test harness." Master now has canlab_run_all_tests (matlab.unittest), a canlab_test_*.m naming/discovery convention, a fault-tolerant walkthrough integration tier, and GitHub Actions CI (test.yml, tests-walkthroughs.yml, codespell.yml). Document the runner options, layout, Passed/Failed/Incomplete semantics, and prerequisites (CanlabCore + Neuroimaging_Pattern_Masks + SPM on path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * CLAUDE.md: note walkthrough tier's RobustToolbox/CANlab_help_examples deps Reflects the dependency surfaced by the nightly walkthrough fix (#85): the walkthrough tier needs RobustToolbox on the path (weighted_corrcoef) in addition to the unit-tier prerequisites, and the nightly CI checks out CANlab_help_examples and RobustToolbox. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 0e6c3cd commit 68b0a7e

1 file changed

Lines changed: 25 additions & 5 deletions

File tree

CLAUDE.md

Lines changed: 25 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
44

55
## What this is
66

7-
CanlabCore is a MATLAB toolbox for MRI/fMRI/PET analysis from the Cognitive and Affective Neuroscience Lab (PI: Tor Wager). The core abstraction is a small set of object types that wrap neuroimaging data and provide a consistent, high-level method surface (`plot`, `predict`, `ica`, `threshold`, `apply_atlas`, `montage`, `surface`, ...). There is **no build system, no test runner, and no linter**code is loaded onto the MATLAB path and exercised interactively or via user scripts.
7+
CanlabCore is a MATLAB toolbox for MRI/fMRI/PET analysis from the Cognitive and Affective Neuroscience Lab (PI: Tor Wager). The core abstraction is a small set of object types that wrap neuroimaging data and provide a consistent, high-level method surface (`plot`, `predict`, `ica`, `threshold`, `apply_atlas`, `montage`, `surface`, ...). There is no build system and no linter; code is loaded onto the MATLAB path and exercised interactively or via user scripts. There **is** now a `matlab.unittest` test runner and GitHub Actions CI (see Tests).
88

99
## Setup and "running" the toolbox
1010

@@ -14,10 +14,30 @@ CanlabCore is a MATLAB toolbox for MRI/fMRI/PET analysis from the Cognitive and
1414

1515
## Tests
1616

17-
There is no test harness. What exists:
18-
- `CanlabCore/Unit_tests/` — three standalone scripts (`check_roi_extraction.m`, `jackknife_similarity_unit_test.m`, `resampling_pattern_expression_unit_test1.m`). Run by `cd`'ing in MATLAB and calling the function name.
19-
- `@fmri_data/predict_test_suite.m` and `@fmri_data/validate_object.m` — broader sanity checks invoked on a constructed object (e.g. `validate_object(my_fmri_data)`).
20-
- For ad-hoc verification, the canonical pattern is to load a sample dataset (`load_image_set('emotionreg')` or files under `CanlabCore/Sample_datasets/`) and run the method end-to-end.
17+
There is a real `matlab.unittest` harness under `CanlabCore/Unit_tests/`. Prerequisite: CanlabCore **and** `Neuroimaging_Pattern_Masks` **and** SPM (SPM12 or SPM25) must be on the path — most tests load atlases/sample images and will error otherwise. The walkthrough tier additionally needs `RobustToolbox` on the path (it supplies `weighted_corrcoef`, used by `plot_correlation_samefig`'s robust branch).
18+
19+
**Runner.** `canlab_run_all_tests.m` is the entry point (returns a `matlab.unittest.TestResult` array):
20+
```matlab
21+
cd CanlabCore/Unit_tests
22+
results = canlab_run_all_tests; % unit tier; walkthroughs SKIPPED by default
23+
results = canlab_run_all_tests('Walkthroughs','only'); % only the slow walkthrough integration tests
24+
results = canlab_run_all_tests('Walkthroughs','include');% both tiers
25+
results = canlab_run_all_tests('Tag','Core'); % select/deselect by tag ('~RequiresMasks' to skip)
26+
results = canlab_run_all_tests('JUnit','test-results.xml'); % write JUnit XML (CI)
27+
```
28+
It globs `canlab_test_*.m` recursively (excluding `old_to_integrate/`) and runs each via `TestSuite.fromFile`. To run/debug one file directly, use `runtests('canlab_test_xxx.m')`.
29+
30+
**Naming convention matters.** Test files are namespaced `canlab_test_*.m` to avoid path collisions. `matlab.unittest`'s `TestSuite.fromFolder` only auto-discovers files whose names *start or end* with "test", so the prefix-`canlab_` form is invisible to it — that's why the runner does its own glob. Name any new test `canlab_test_<thing>.m` and it is picked up automatically.
31+
32+
**Layout.** `Unit_tests/<class-or-area>/canlab_test_*.m` (e.g. `fmri_data/`, `image_vector/`, `statistic_image/`, `atlas/`, `predictive_model/`, `region/`, `workflows/`); `Unit_tests/helpers/` (shared fixtures like `canlab_get_sample_thresholded_t`); `Unit_tests/walkthroughs/` (slow end-to-end integration tier). Legacy still present: the old standalone scripts, `@fmri_data/predict_test_suite.m`, `@fmri_data/validate_object.m`.
33+
34+
**Result semantics.** CI gates on **Failed only** (`if any([results.Failed]); error(...); end`). **Incomplete** = "Filtered by assumption": the test called `assumeFail`/`assumeTrue` and was skipped because an environment precondition (a data file, a toolbox, a display) was missing — not a failure. A couple of Incomplete results is normal and environment-dependent. Verified locally (R2026a + SPM25): unit tier **261 passed / 0 failed / 2 incomplete** (~9 min); walkthrough tier **10/10 passed**.
35+
36+
**Walkthrough tier is fault-tolerant by design.** `canlab_run_walkthrough_snapshot` runs *vendored snapshot copies* of the CANlab_help_examples scripts (`walkthroughs/private/canlab_help_*.m`) section by section, so the tier does not depend on the external help repo. `canlab_classify_environment_error` downgrades environment-limited sections (missing toolbox, no graphics/display, invalid surface handles) to logged **skips** rather than failures — so a headless or partially-provisioned machine still passes. Don't "fix" a skipped graphics section by forcing it to run.
37+
38+
**CI.** `.github/workflows/test.yml` runs the unit tier (R2024b + Statistics + Signal Processing, clones SPM25, checks out `Neuroimaging_Pattern_Masks`, uploads JUnit XML). `tests-walkthroughs.yml` runs the walkthrough tier nightly (cron) and additionally checks out `CANlab_help_examples` and `RobustToolbox` — the latter supplies `weighted_corrcoef`, needed by the regression walkthrough; without it the nightly fails with an `UndefinedFunction` error. `codespell.yml` is a spell-check action over comments/docs.
39+
40+
For quick ad-hoc verification (not a formal test), the canonical pattern is still to load a sample dataset (`load_image_set('emotionreg')` or files under `CanlabCore/Sample_datasets/`) and run the method end-to-end.
2141

2242
## Object architecture (the part you must understand to be productive)
2343

0 commit comments

Comments
 (0)