Skip to content

Commit 57bb5e7

Browse files
K-saifpre-commit-ci[bot]Bordaclaude[bot]
authored
Add adaptive TP/FP/FN validation mosaic export (#2271)
- remove top-level cv2/annotator imports; lazy-load inside rendering functions - remove save_result_images bool; save_directory_path is now keyword-only after metric_target - drop hardcoded result/ subdirectory from benchmark output path - propagate metric_target into _split_detections_by_outcome for correct OBB IoU dispatch - add filename collision UserWarning in benchmark loop - remove dead/unreachable combined None-check in _split_detections_by_outcome - add Google-style docstrings to all 5 new private visualization functions - add TestSplitDetectionsByOutcome covering 7 edge cases (empty inputs, cross-class, confidence-None) - fix FP/FN pixel assertions to check interior box pixels rather than border/title regions - fix benchmark_a_model.md: full panel names, add Visual Benchmarking section, update API examples --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: jirka <6035284+Borda@users.noreply.github.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
1 parent 2aa43bc commit 57bb5e7

3 files changed

Lines changed: 646 additions & 2 deletions

File tree

docs/how_to/benchmark_a_model.md

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -329,6 +329,20 @@ Here, predictions in purple are targets (ground truth), and predictions in teal
329329

330330
See [annotator documentation](https://supervision.roboflow.com/latest/detection/annotators/) for even more options.
331331

332+
## Visual Benchmarking
333+
334+
To inspect where a model succeeds and fails, pass `save_directory_path` to `sv.ConfusionMatrix.benchmark(...)`. For every dataset image it writes a 2x2 result grid — `Ground Truth`, `True Positives`, `False Positives`, and `False Negatives` panels — directly into that directory, reusing the original image filenames. This makes it easy to skim through per-image outcomes alongside the aggregate confusion matrix.
335+
336+
```python
337+
import supervision as sv
338+
339+
confusion_matrix = sv.ConfusionMatrix.benchmark(
340+
dataset=test_set,
341+
callback=callback,
342+
save_directory_path="./results",
343+
)
344+
```
345+
332346
## Benchmarking Metrics
333347

334348
With multiple models, fine details matter. Visual inspection may not be enough. `supervision` provides a collection of metrics that help obtain precise numerical results of model performance.
@@ -462,7 +476,7 @@ Yes, if you want to evaluate their bounding boxes. Convert model outputs to `Det
462476

463477
### What is a ConfusionMatrix and how do I use it?
464478

465-
`sv.ConfusionMatrix` visualizes true positives, false positives, and false negatives per class. Create one with `sv.ConfusionMatrix.from_detections(predictions=predictions, targets=targets, classes=classes, conf_threshold=0.5, iou_threshold=0.5)`, then call `metric.plot()` to render a heatmap.
479+
`sv.ConfusionMatrix` visualizes true positives, false positives, and false negatives per class. Create one with `sv.ConfusionMatrix.from_detections(predictions=predictions, targets=targets, classes=classes, conf_threshold=0.5, iou_threshold=0.5)`, then call `confusion_matrix.plot()` to render a heatmap. If you want per-image validation visualizations saved to disk, pass `save_directory_path="./results"` to `sv.ConfusionMatrix.benchmark(...)`; it will write 2x2 result grids directly into that directory using the original image filenames, with `Ground Truth`, `True Positives`, `False Positives`, and `False Negatives` panels.
466480

467481
## Author
468482

0 commit comments

Comments
 (0)