Skip to content

Commit ee3a49e

Browse files
committed
[DATALAD RUNCMD] Designing docs to accompany dandi/dandi-cli#1822 (persist validation logs)
=== Do not change lines below === { "chain": [], "cmd": "yolo -v /home/yoh/proj/dandi/dandi-cli-enh-validators:/home/yoh/proj/dandi/dandi-cli-enh-validators:ro -- 'In /home/yoh/proj/dandi/dandi-cli-enh-validators which is submitted as dandi/dandi-cli#1822 we significantly improved validation interfacing -- we serialize validation outputs and store so we could reload and potentially review with different filtering or use external tools like visidata to navigate. I would like here to improve our https://docs.dandiarchive.org/user-guide-sharing/validating-files/ section with improved documentation, reflecting the state of that PR. We should demonstrate that we store companion validation files during upload so they could be re-reviewed/analyzed. We should show basic use of visidata to quickly review them. Could use bids-examples repo and some sample dandisets (should be sufficiently small) to show how e.g. to compose multiple validation files. Ideally we should script production of example outputs, and/or store/share validation output example for easier access. Do research how other projects using mkdocs produce similar demo walkthroughs, and what we have done so far in this repo. Do research, build plan for content and also implementation details.'", "exit": 0, "extra_inputs": [], "inputs": [], "outputs": [], "pwd": "." } ^^^ Do not change lines above ^^^
1 parent 97024b0 commit ee3a49e

18 files changed

Lines changed: 1471 additions & 18 deletions
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
=== ERROR (21 issues) ===
2+
[DANDI.NO_DANDISET_FOUND] eeg_cbm — Path is not inside a Dandiset
3+
[BIDS.EMPTY_FILE] eeg_cbm/sub-cbm001/eeg/sub-cbm001_task-protmap_eeg.edf — Empty files not allowed.
4+
[BIDS.EMPTY_FILE] eeg_cbm/sub-cbm002/eeg/sub-cbm002_task-protmap_eeg.edf — Empty files not allowed.
5+
... and 18 more issues
6+
=== HINT (410 issues) ===
7+
[BIDS.README_FILE_SMALL] eeg_cbm/README — The recommended file '/README' is very small.
8+
Please consider expanding it with additional information about the dataset.
9+
[BIDS.JSON_KEY_RECOMMENDED] eeg_cbm/dataset_description.json — A JSON file is missing a key listed as recommended.
10+
subCode: HEDVersion
11+
issueMessage: Field description: If HED tags are used:
12+
The version of the HED schema used to validate HED tags for study.
13+
May include a single schema or a base schema and one or more library schema.
14+
[BIDS.JSON_KEY_RECOMMENDED] eeg_cbm/dataset_description.json — A JSON file is missing a key listed as recommended.
15+
subCode: GeneratedBy
16+
issueMessage: Field description: Used to specify provenance of the dataset.
17+
... and 407 more issues
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
[
2+
{
3+
"asset_paths": null,
4+
"dandiset_path": "invalid_eeg_cbm",
5+
"dataset_path": "invalid_eeg_cbm",
6+
"id": "BIDS.README_FILE_MISSING",
7+
"message": "The recommended file /README is missing.\nSee Section 03 (Modality agnostic files) of the BIDS specification.\n",
8+
"metadata": null,
9+
"origin": {
10+
"standard": "BIDS",
11+
"standard_schema_version": "1.2.1",
12+
"standard_version": null,
13+
"type": "VALIDATION",
14+
"validator": "bids-validator-deno",
15+
"validator_version": "2.4.1"
16+
},
17+
"path": "invalid_eeg_cbm/dataset_description.json",
18+
"path_regex": null,
19+
"record_version": "1",
20+
"scope": "file",
21+
"severity": "HINT",
22+
"within_asset_paths": null
23+
},
24+
{
25+
"asset_paths": null,
26+
"dandiset_path": "invalid_eeg_cbm",
27+
"dataset_path": "invalid_eeg_cbm",
28+
"id": "BIDS.JSON_KEY_RECOMMENDED",
29+
"message": "A JSON file is missing a key listed as recommended.\nsubCode: HEDVersion\nissueMessage: Field description: If HED tags are used:\nThe version of the HED schema used to validate HED tags for study.\nMay include a single schema or a base schema and one or more library schema.\n",
30+
"metadata": null,
31+
"origin": {
32+
"standard": "BIDS",
33+
"standard_schema_version": "1.2.1",
34+
"standard_version": null,
35+
"type": "VALIDATION",
36+
"validator": "bids-validator-deno",
37+
"validator_version": "2.4.1"
38+
},
39+
"path": "invalid_eeg_cbm/dataset_description.json",
40+
"path_regex": null,
41+
"record_version": "1",
42+
"scope": "file",
43+
"severity": "HINT",
44+
"within_asset_paths": null
45+
},
46+
{
47+
"asset_paths": null,
48+
"dandiset_path": "invalid_eeg_cbm",
49+
"dataset_path": "invalid_eeg_cbm",
50+
"id": "BIDS.JSON_KEY_RECOMMENDED",
51+
"message": "A JSON file is missing a key listed as recommended.\nsubCode: GeneratedBy\nissueMessage: Field description: Used to specify provenance of the dataset.\n",
52+
"metadata": null,
53+
"origin": {
54+
"standard": "BIDS",
55+
"standard_schema_version": "1.2.1",
56+
"standard_version": null,
57+
"type": "VALIDATION",
58+
"validator": "bids-validator-deno",
59+
"validator_version": "2.4.1"
60+
},
61+
"path": "invalid_eeg_cbm/dataset_description.json",
62+
"path_regex": null,
63+
"record_version": "1",
64+
"scope": "file",
65+
"severity": "HINT",
66+
"within_asset_paths": null
67+
},
68+
{
69+
"_truncated": true,
70+
"omitted_count": 429
71+
}
72+
]

docs/examples/validation/bids_invalid_eeg_cbm.jsonl

Lines changed: 432 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
[BIDS.README_FILE_MISSING] invalid_eeg_cbm/dataset_description.json — The recommended file /README is missing.
2+
See Section 03 (Modality agnostic files) of the BIDS specification.
3+
[BIDS.JSON_KEY_RECOMMENDED] invalid_eeg_cbm/dataset_description.json — A JSON file is missing a key listed as recommended.
4+
subCode: HEDVersion
5+
issueMessage: Field description: If HED tags are used:
6+
The version of the HED schema used to validate HED tags for study.
7+
May include a single schema or a base schema and one or more library schema.
8+
[BIDS.JSON_KEY_RECOMMENDED] invalid_eeg_cbm/dataset_description.json — A JSON file is missing a key listed as recommended.
9+
subCode: GeneratedBy
10+
issueMessage: Field description: Used to specify provenance of the dataset.
11+
[BIDS.JSON_KEY_RECOMMENDED] invalid_eeg_cbm/dataset_description.json — A JSON file is missing a key listed as recommended.
12+
subCode: SourceDatasets
13+
issueMessage: Field description: Used to specify the locations and relevant attributes of all source datasets (BIDS or not).
14+
Valid keys in each object include `"URL"`, `"DOI"` (see
15+
[URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator)), and
16+
`"Version"` with
17+
[string](https://www.w3schools.com/js/js_json_datatypes.asp)
18+
values.
19+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm001/eeg/sub-cbm001_task-protmap_eeg.edf — Empty files not allowed.
20+
... and 427 more issues
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
- asset_paths: null
2+
dandiset_path: invalid_eeg_cbm
3+
dataset_path: invalid_eeg_cbm
4+
id: BIDS.README_FILE_MISSING
5+
message: "The recommended file /README is missing.\nSee Section 03 (Modality agnostic
6+
files) of the BIDS specification.\n"
7+
metadata: null
8+
origin:
9+
standard: BIDS
10+
standard_schema_version: 1.2.1
11+
standard_version: null
12+
type: VALIDATION
13+
validator: bids-validator-deno
14+
validator_version: 2.4.1
15+
path:
16+
invalid_eeg_cbm/dataset_description.json
17+
path_regex: null
18+
record_version: '1'
19+
scope: file
20+
severity: HINT
21+
within_asset_paths: null
22+
- asset_paths: null
23+
dandiset_path: invalid_eeg_cbm
24+
dataset_path: invalid_eeg_cbm
25+
id: BIDS.JSON_KEY_RECOMMENDED
26+
message: "A JSON file is missing a key listed as recommended.\nsubCode: HEDVersion\n\
27+
issueMessage: Field description: If HED tags are used:\nThe version of the HED
28+
schema used to validate HED tags for study.\nMay include a single schema or a
29+
base schema and one or more library schema.\n"
30+
metadata: null
31+
origin:
32+
standard: BIDS
33+
standard_schema_version: 1.2.1
34+
standard_version: null
35+
type: VALIDATION
36+
validator: bids-validator-deno
37+
validator_version: 2.4.1
38+
path:
39+
invalid_eeg_cbm/dataset_description.json
40+
path_regex: null
41+
record_version: '1'
42+
scope: file
43+
severity: HINT
44+
within_asset_paths: null
45+
- asset_paths: null
46+
dandiset_path: invalid_eeg_cbm
47+
dataset_path: invalid_eeg_cbm
48+
id: BIDS.JSON_KEY_RECOMMENDED
49+
message: "A JSON file is missing a key listed as recommended.\nsubCode: GeneratedBy\n\
50+
issueMessage: Field description: Used to specify provenance of the dataset.\n"
51+
metadata: null
52+
origin:
53+
standard: BIDS
54+
standard_schema_version: 1.2.1
55+
standard_version: null
56+
type: VALIDATION
57+
validator: bids-validator-deno
58+
validator_version: 2.4.1
59+
path:
60+
invalid_eeg_cbm/dataset_description.json
61+
path_regex: null
62+
record_version: '1'
63+
scope: file
64+
severity: HINT
65+
within_asset_paths: null
66+
- _truncated: true
67+
omitted_count: 429
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm001/eeg/sub-cbm001_task-protmap_eeg.edf — Empty files not allowed.
2+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm002/eeg/sub-cbm002_task-protmap_eeg.edf — Empty files not allowed.
3+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm003/eeg/sub-cbm003_task-protmap_eeg.edf — Empty files not allowed.
4+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm004/eeg/sub-cbm004_task-protmap_eeg.edf — Empty files not allowed.
5+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm005/eeg/sub-cbm005_task-protmap_eeg.edf — Empty files not allowed.
6+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm006/eeg/sub-cbm006_task-protmap_eeg.edf — Empty files not allowed.
7+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm007/eeg/sub-cbm007_task-protmap_eeg.edf — Empty files not allowed.
8+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm008/eeg/sub-cbm008_task-protmap_eeg.edf — Empty files not allowed.
9+
[BIDS.NIFTI_HEADER_UNREADABLE] invalid_eeg_cbm/sub-cbm011/sub-cbm011_scans.nii — We were unable to parse header data from this NIfTI file. Please ensure it is not corrupted or mislabeled.
10+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm009/eeg/sub-cbm009_task-protmap_eeg.edf — Empty files not allowed.
11+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm010/eeg/sub-cbm010_task-protmap_eeg.edf — Empty files not allowed.
12+
[BIDS.EXTENSION_MISMATCH] invalid_eeg_cbm/sub-cbm011/sub-cbm011_scans.nii — Extension used by file does not match allowed extensions for its suffix
13+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm011/eeg/sub-cbm011_task-protmap_eeg.edf — Empty files not allowed.
14+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm012/eeg/sub-cbm012_task-protmap_eeg.edf — Empty files not allowed.
15+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm013/eeg/sub-cbm013_task-protmap_eeg.edf — Empty files not allowed.
16+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm014/eeg/sub-cbm014_task-protmap_eeg.edf — Empty files not allowed.
17+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm015/eeg/sub-cbm015_task-protmap_eeg.edf — Empty files not allowed.
18+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm016/eeg/sub-cbm016_task-protmap_eeg.edf — Empty files not allowed.
19+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm017/eeg/sub-cbm017_task-protmap_eeg.edf — Empty files not allowed.
20+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm018/eeg/sub-cbm018_task-protmap_eeg.edf — Empty files not allowed.
21+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm019/eeg/sub-cbm019_task-protmap_eeg.edf — Empty files not allowed.
22+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm020/eeg/sub-cbm020_task-protmap_eeg.edf — Empty files not allowed.
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
=== HINT (410 issues) ===
2+
[BIDS.README_FILE_MISSING] invalid_eeg_cbm/dataset_description.json — The recommended file /README is missing.
3+
See Section 03 (Modality agnostic files) of the BIDS specification.
4+
[BIDS.JSON_KEY_RECOMMENDED] invalid_eeg_cbm/dataset_description.json — A JSON file is missing a key listed as recommended.
5+
subCode: HEDVersion
6+
issueMessage: Field description: If HED tags are used:
7+
The version of the HED schema used to validate HED tags for study.
8+
May include a single schema or a base schema and one or more library schema.
9+
[BIDS.JSON_KEY_RECOMMENDED] invalid_eeg_cbm/dataset_description.json — A JSON file is missing a key listed as recommended.
10+
subCode: GeneratedBy
11+
issueMessage: Field description: Used to specify provenance of the dataset.
12+
... and 407 more issues
13+
=== ERROR (22 issues) ===
14+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm001/eeg/sub-cbm001_task-protmap_eeg.edf — Empty files not allowed.
15+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm002/eeg/sub-cbm002_task-protmap_eeg.edf — Empty files not allowed.
16+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm003/eeg/sub-cbm003_task-protmap_eeg.edf — Empty files not allowed.
17+
... and 19 more issues
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
=== HINT (410 issues) ===
2+
=== BIDS.README_FILE_MISSING (1 issue) ===
3+
[BIDS.README_FILE_MISSING] invalid_eeg_cbm/dataset_description.json — The recommended file /README is missing.
4+
See Section 03 (Modality agnostic files) of the BIDS specification.
5+
=== BIDS.JSON_KEY_RECOMMENDED (3 issues) ===
6+
[BIDS.JSON_KEY_RECOMMENDED] invalid_eeg_cbm/dataset_description.json — A JSON file is missing a key listed as recommended.
7+
subCode: HEDVersion
8+
issueMessage: Field description: If HED tags are used:
9+
The version of the HED schema used to validate HED tags for study.
10+
May include a single schema or a base schema and one or more library schema.
11+
[BIDS.JSON_KEY_RECOMMENDED] invalid_eeg_cbm/dataset_description.json — A JSON file is missing a key listed as recommended.
12+
subCode: GeneratedBy
13+
issueMessage: Field description: Used to specify provenance of the dataset.
14+
... and 1 more issue
15+
=== BIDS.SIDECAR_KEY_RECOMMENDED (360 issues) ===
16+
[BIDS.SIDECAR_KEY_RECOMMENDED] invalid_eeg_cbm/sub-cbm001/eeg/sub-cbm001_task-protmap_eeg.edf — A data file's JSON sidecar is missing a key listed as recommended.
17+
subCode: Manufacturer
18+
issueMessage: Field description: Manufacturer of the equipment that produced the measurements.
19+
[BIDS.SIDECAR_KEY_RECOMMENDED] invalid_eeg_cbm/sub-cbm001/eeg/sub-cbm001_task-protmap_eeg.edf — A data file's JSON sidecar is missing a key listed as recommended.
20+
subCode: ManufacturersModelName
21+
issueMessage: Field description: Manufacturer's model name of the equipment that produced the measurements.
22+
... and 358 more issues
23+
=== BIDS.TSV_ADDITIONAL_COLUMNS_UNDEFINED (40 issues) ===
24+
[BIDS.TSV_ADDITIONAL_COLUMNS_UNDEFINED] invalid_eeg_cbm/sub-cbm001/eeg/sub-cbm001_task-protmap_events.tsv — A TSV file has extra columns which are not defined in its associated JSON sidecar
25+
subCode: value
26+
[BIDS.TSV_ADDITIONAL_COLUMNS_UNDEFINED] invalid_eeg_cbm/sub-cbm001/eeg/sub-cbm001_task-protmap_events.tsv — A TSV file has extra columns which are not defined in its associated JSON sidecar
27+
subCode: sample
28+
... and 38 more issues
29+
=== BIDS.EEG_CHANNEL_COUNT_MISMATCH (6 issues) ===
30+
[BIDS.EEG_CHANNEL_COUNT_MISMATCH] invalid_eeg_cbm/sub-cbm015/eeg/sub-cbm015_task-protmap_eeg.edf — The EEGChannelCount metadata does not match the number of channels
31+
with type EEG in the associated channels.tsv file.
32+
[BIDS.EEG_CHANNEL_COUNT_MISMATCH] invalid_eeg_cbm/sub-cbm016/eeg/sub-cbm016_task-protmap_eeg.edf — The EEGChannelCount metadata does not match the number of channels
33+
with type EEG in the associated channels.tsv file.
34+
... and 4 more issues
35+
=== ERROR (22 issues) ===
36+
=== BIDS.EMPTY_FILE (20 issues) ===
37+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm001/eeg/sub-cbm001_task-protmap_eeg.edf — Empty files not allowed.
38+
[BIDS.EMPTY_FILE] invalid_eeg_cbm/sub-cbm002/eeg/sub-cbm002_task-protmap_eeg.edf — Empty files not allowed.
39+
... and 18 more issues
40+
=== BIDS.NIFTI_HEADER_UNREADABLE (1 issue) ===
41+
[BIDS.NIFTI_HEADER_UNREADABLE] invalid_eeg_cbm/sub-cbm011/sub-cbm011_scans.nii — We were unable to parse header data from this NIfTI file. Please ensure it is not corrupted or mislabeled.
42+
=== BIDS.EXTENSION_MISMATCH (1 issue) ===
43+
[BIDS.EXTENSION_MISMATCH] invalid_eeg_cbm/sub-cbm011/sub-cbm011_scans.nii — Extension used by file does not match allowed extensions for its suffix
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{"asset_paths": null, "dandiset_path": null, "dataset_path": null, "id": "DANDI.NO_DANDISET_FOUND", "message": "Path is not inside a Dandiset", "metadata": null, "origin": {"standard": "DANDI-LAYOUT", "standard_schema_version": null, "standard_version": null, "type": "VALIDATION", "validator": "dandi", "validator_version": "0+unknown"}, "path": "000027", "path_regex": null, "record_version": "1", "scope": "dandiset", "severity": "ERROR", "within_asset_paths": null}
2+
{"asset_paths": null, "dandiset_path": null, "dataset_path": null, "id": "pynwb.GENERIC", "message": "error: nwb_version '2.0b' is not a proper semantic version. See http://semver.org", "metadata": null, "origin": {"standard": null, "standard_schema_version": null, "standard_version": null, "type": "VALIDATION", "validator": "dandi", "validator_version": "0+unknown"}, "path": "sub-RAT123/sub-RAT123.nwb", "path_regex": null, "record_version": "1", "scope": "file", "severity": "ERROR", "within_asset_paths": null}
3+
{"asset_paths": null, "dandiset_path": "sub-RAT123", "dataset_path": "' separator, e.g., 'P1D/P3D' for an age range somewhere from 1 to 3 days. If you cannot specify the upper bound of the range, you may leave the right side blank, e.g., 'P90Y/' means 90 years old or older.", "metadata": null, "origin": {"standard": null, "standard_schema_version": null, "standard_version": null, "type": "VALIDATION", "validator": "nwbinspector", "validator_version": ""}, "path": "sub-RAT123/sub-RAT123.nwb", "path_regex": null, "record_version": "1", "scope": "file", "severity": "ERROR", "within_asset_paths": {"sub-RAT123/sub-RAT123.nwb": "/general/subject"}}
4+
{"asset_paths": null, "dandiset_path": "sub-RAT123", "dataset_path": "tmp/tmp.YcS3RheKVl/ds000027/sub-RAT123/sub-RAT123.nwb", "path_regex": null, "record_version": "1", "scope": "file", "severity": "ERROR", "within_asset_paths": {"sub-RAT123/sub-RAT123.nwb": "/general/subject"}}
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[DANDI.NO_DANDISET_FOUND] 000027 — Path is not inside a Dandiset
2+
[pynwb.GENERIC] sub-RAT123/sub-RAT123.nwb — error: nwb_version '2.0b' is not a proper semantic version. See http://semver.org
3+
[NWBI.check_subject_age] sub-RAT123/sub-RAT123.nwb — Subject age, '12 mo', does not follow ISO 8601 duration format, e.g. 'P2Y' for 2 years or 'P23W' for 23 weeks. You may also specify a range using a '/' separator, e.g., 'P1D/P3D' for an age range somewhere from 1 to 3 days. If you cannot specify the upper bound of the range, you may leave the right side blank, e.g., 'P90Y/' means 90 years old or older.
4+
[NWBI.check_subject_weight] sub-RAT123/sub-RAT123.nwb — Subject weight '2 lbs' does not follow the expected form '[numeric] [unit]'. For example, '2.3 kg'. Without a unit, the weight is ambiguous. Valid units are: 'kg', 'g', 'mg', 'ug', 'μg', 'ng', 'pg'.

0 commit comments

Comments
 (0)