Binary class usage

[tridivb]

In which format does the library expect Binary logit predictions to be for metric modules like Accuracy? I am a bit confused with the description here,

https://github.com/PyTorchLightning/metrics/blob/3613a21dfaed5257656b18879dc12973027a7caa/torchmetrics/utilities/checks.py#L310

My current binary segmentation model outputs are of the form Nx1xHxW (sigmoids of predicted logits) and targets are NxHxW.

Currently, the following code snippet uses the pred.ndims to assign the case i.e. BINARY or others which seems counter-intuitive for my case.

https://github.com/PyTorchLightning/metrics/blob/3613a21dfaed5257656b18879dc12973027a7caa/torchmetrics/utilities/checks.py#L74

Moreover, due to my output being single channel sigmoid probabilities, the implied_classes are also assigned wrongly.

https://github.com/PyTorchLightning/metrics/blob/3613a21dfaed5257656b18879dc12973027a7caa/torchmetrics/utilities/checks.py#L83

Previously, with pytorch lightning 1.1, this wasn't an issue as the _input_format_classification used to either threshold (for binary) or take the argmax (for multiclass) predictions and assign the num_classes from there.

Is there a specific reason to move away from the previous approach and how are we supposed to use the Metrics for the above case now?

[tridivb]

I just realized, this seems to be a bug because as per the docstring below, the prediction and target should both be flatttened arrays.

https://github.com/PyTorchLightning/metrics/blob/3613a21dfaed5257656b18879dc12973027a7caa/torchmetrics/utilities/checks.py#L310

The intended classes would then end up being wrong in any case as it only assigns the very first value of the array, in turn failing the class consistency check.

https://github.com/PyTorchLightning/metrics/blob/3613a21dfaed5257656b18879dc12973027a7caa/torchmetrics/utilities/checks.py#L168

[tridivb]

Is there an answer for this?

[Borda]

@tridivb — the shape handling issue (Nx1xHxW vs NxHxW) was a real pain point with the old unified API. Completely resolved with task-specific metrics in v1.9.0.

For binary segmentation with sigmoid output (Nx1xHxW):

from torchmetrics.classification import BinaryAccuracy, BinaryF1Score, BinaryJaccardIndex

# Squeeze the channel dim
preds = model(x).squeeze(1)  # (N, H, W) — sigmoid probabilities
target = target               # (N, H, W) — 0 or 1

acc = BinaryAccuracy(multidim_average="global")
f1 = BinaryF1Score(multidim_average="global")
iou = BinaryJaccardIndex()

acc(preds, target)
f1(preds, target)
iou(preds, target)

Key differences from the old API:

• No more num_classes for binary tasks
• No more multiclass=True/False confusion
• No more implied_classes heuristic — the metric type tells TM exactly what format to expect
• Threshold default is 0.5 — pass probabilities, not argmax

The old Accuracy(num_classes=1) / Accuracy(num_classes=2) patterns are fully deprecated.

Docs: Accuracy