Skip to content

feat: add option to dump sanity check results into JSON #167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ktro2828 merged 3 commits into from
Aug 18, 2025
Merged

feat: add option to dump sanity check results into JSON #167

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
497ea07
feat: add option to save exceptions as JSON
ktro2828 Aug 17, 2025
96f9369
docs: update document
ktro2828 Aug 17, 2025
ff1640c
chore: use uppercase
ktro2828 Aug 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 33 additions & 2 deletions docs/tutorials/cli/t4sanity.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ $ t4sanity -h
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --version -v Show the application version and exit. │
│ --output -o TEXT Path to output JSON file. [default: None] │
│ --revision -rv TEXT Specify if you want to load the specific version. [default: None] │
│ --include-warning -iw Indicates whether to report any warnings. │
│ --install-completion Install completion for the current shell. │
Expand Down Expand Up @@ -62,7 +63,7 @@ $ t4sanity <DATA_ROOT>
>>>Sanity checking...: 2it [00:00, 18.69it/s]
⚠️ Encountered some exceptions!!
+-----------+---------+--------+------------------------------------------------------------------------------------------------+
| DatasetID | Version | status | Message |
| DatasetID | Version | Status | Message |
+-----------+---------+--------+------------------------------------------------------------------------------------------------+
| dataset1 | 2 | ERROR | bbox must be (xmin, ymin, xmax, ymax) and xmin <= xmax && ymin <= ymax: (1532, 198, 1440, 265) |
| dataset2 | 1 | OK | |
Expand All @@ -79,9 +80,39 @@ $ t4sanity <DATA_ROOT> -iw
>>>Sanity checking...: 2it [00:00, 21.54it/s]
⚠️ Encountered some exceptions!!
+-----------+---------+---------+------------------------------------------------------------------------------------------------+
| DatasetID | Version | status | Message |
| DatasetID | Version | Status | Message |
+-----------+---------+---------+------------------------------------------------------------------------------------------------+
| dataset1 | 2 | ERROR | bbox must be (xmin, ymin, xmax, ymax) and xmin <= xmax && ymin <= ymax: (1532, 198, 1440, 265) |
| dataset2 | 1 | WARNING | Category token is empty for surface ann: 0c15d9c143fb2723c16ac7e0c735b0a8 |
+-----------+---------+---------+------------------------------------------------------------------------------------------------+
```

### Dump Results as JSON

To dump results into JSON, use the `-o; --output` option:

```shell
$ t4sanity <DATA_ROOT> -o results.json

>>>Sanity checking...: 2it [00:00, 21.54it/s]
...
```

Then a JSON file named `results.json` will be generated:

```json
[
{
"dataset_id": "dataset1",
"version": 2,
"status": "ERROR",
"message": "bbox must be (xmin, ymin, xmax, ymax) and xmin <= xmax && ymin <= ymax: (1532, 198, 1440, 265)"
},
{
"dataset_id": "dataset2",
"version": 1,
"status": "WARNING",
"message": "Category token is empty for surface ann: 0c15d9c143fb2723c16ac7e0c735b0a8"
}
]
```
9 changes: 8 additions & 1 deletion t4_devkit/cli/sanity.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@
from tabulate import tabulate
from tqdm import tqdm

from t4_devkit.common.io import save_json
from t4_devkit.common.sanity import DBException, sanity_check
from t4_devkit.common.serialize import serialize_dataclasses

from .version import version_callback

Expand Down Expand Up @@ -41,6 +43,7 @@ def main(
is_eager=True,
),
db_parent: str = typer.Argument(..., help="Path to parent directory of the databases."),
output: str | None = typer.Option(None, "-o", "--output", help="Path to output JSON file."),
revision: str | None = typer.Option(
None, "-rv", "--revision", help="Specify if you want to check the specific version."
),
Expand All @@ -54,6 +57,10 @@ def main(
print("✅ No exceptions occurred!!")
else:
print("⚠️ Encountered some exceptions!!")
headers = ["DatasetID", "Version", "status", "Message"]
headers = ["DatasetID", "Version", "Status", "Message"]
table = [[e.dataset_id, e.version, e.status, e.message] for e in exceptions]
print(tabulate(table, headers=headers, tablefmt="pretty"))

if output:
serialized = serialize_dataclasses(exceptions)
save_json(serialized, output)
Loading