feat(yolo26-sem): YOLO26 semantic segmentation via inference_models (ONNX + TorchScript + TRT) (#2372)

* feat(yolo26-sem): YOLO26 semantic segmentation via inference_models (ONNX + TorchScript)

Adds public-pretrained YOLO26-Sem support (Ultralytics 8.4.52/53,
Cityscapes pretrains, 1024x1024) to inference through inference_models.
This PR covers the ONNX and TorchScript backends; the TRT backend ships
in a follow-up once we can build and validate the engines.

Pieces:

1. Shared semantic-seg post-processing helper
   `post_process_semantic_segmentation_logits()` in
   `inference_models/models/common/roboflow/post_processing.py`. Handles
   the softmax -> argmax -> letterbox-crop -> resize pipeline. All three
   DeepLabV3+ backends (ONNX, Torch, TRT) refactored to delegate here,
   replacing 3 near-identical inline implementations.

2. YOLO26ForSemanticSegmentation{Onnx,TorchScript} classes inheriting
   SemanticSegmentationModel and delegating post-processing to the
   shared helper. Reuses INFERENCE_MODELS_YOLO26_DEFAULT_CONFIDENCE
   (no new constant).

3. Two registry tuples in models_registry.py for
   (yolo26, semantic-segmentation, {ONNX, TORCH_SCRIPT}). One dispatch
   entry in inference/models/utils.py routing
   ("semantic-segmentation", "yolo26") -> existing generic
   InferenceModelsSemanticSegmentationAdapter.

Tests:
- Unit: import smoke for ONNX + TorchScript, registry lookup for both
  backends, synthetic-tensor coverage for the shared post-processing
  helper (winning-class collapse + sub-threshold background fallback).
- Integration ONNX path validated end-to-end against staging for all
  five sizes (yolo26{n,s,m,l,x}-sem-1024) with the bus.jpg test image
  through /infer/semantic_segmentation.

Follow-up PR adds the TRT backend (class + registry tuple + integration
test) once the trt-compiler image is rebuilt against this PR and the
engine packages are available to test against.

* fix(yolo26-sem): require background class in semantic-seg packages

Replace the silent `background_class_id = -1` fallback with a shared
`resolve_background_class_id()` helper that raises CorruptedModelPackageError
when `class_names.txt` has no `background` entry.

A negative background id is never a valid output: it aliases a real class
via negative indexing in downstream consumers (`class_names[-1]`, palette
LUTs) and breaks the platform 0=background convention, silently corrupting
the segmentation map. Failing loud at load time surfaces a misbuilt package
instead. The conversion side (roboflow-model-conversion#92) guarantees
`background` is prepended, so correctly-built packages are unaffected.

Wires the helper into both YOLO26-sem classes and all three DeepLabV3+
backends, removing the duplicated try/except idiom.

* feat(yolo26-sem): TRT backend for YOLO26 semantic segmentation (#2379)

* fix(yolo26-sem): guard TorchScript load with torchscript_global_lock

torch.jit.load shares a non-thread-safe process-global; wrap the load in
torchscript_global_lock(torchscript_state_global_lock) and accept the lock
in from_pretrained, matching the other YOLO26 TorchScript classes.
Addresses review feedback on #2372.

* bump requirements inference_models==0.28.7

---------

Co-authored-by: Paweł Pęczek <146137186+PawelPeczek-Roboflow@users.noreply.github.com>

Author: leeclemnet

inference/models/utils.py

@@ -990,6 +990,12 @@ def get_roboflow_model(*args, **kwargs):
                 category=InferenceModelsStackMissing,
             )
 
+    # YOLO26 semantic segmentation is inference_models-only (no legacy implementation),
+    # so we add entries directly rather than swapping existing ones.
+    ROBOFLOW_MODEL_TYPES[("semantic-segmentation", "yolo26")] = (
+        InferenceModelsSemanticSegmentationAdapter
+    )
+
     # YOLOLite is inference_models-only (no legacy implementation),
     # so we add entries directly rather than swapping existing ones.
     for variant in [

inference_models/docs/changelog.md

@@ -1,5 +1,9 @@
 # Changelog
 
+## `0.28.7`
+
+- Added YOLO26 semantic segmentation support (ONNX, TorchScript, and TensorRT backends).
+
 ## `0.28.6`
 
 - torch.jit.load/script share a process-global which is not thread-safe, introduced lock to prevent race conditions when loading SAM3 and other torchscript models

inference_models/inference_models/models/auto_loaders/models_registry.py

@@ -255,6 +255,18 @@ class RegistryEntry:
         module_name="inference_models.models.yolo26.yolo26_instance_segmentation_trt",
         class_name="YOLO26ForInstanceSegmentationTRT",
     ),
+    ("yolo26", SEMANTIC_SEGMENTATION_TASK, BackendType.ONNX): LazyClass(
+        module_name="inference_models.models.yolo26.yolo26_semantic_segmentation_onnx",
+        class_name="YOLO26ForSemanticSegmentationOnnx",
+    ),
+    ("yolo26", SEMANTIC_SEGMENTATION_TASK, BackendType.TORCH_SCRIPT): LazyClass(
+        module_name="inference_models.models.yolo26.yolo26_semantic_segmentation_torch_script",
+        class_name="YOLO26ForSemanticSegmentationTorchScript",
+    ),
+    ("yolo26", SEMANTIC_SEGMENTATION_TASK, BackendType.TRT): LazyClass(
+        module_name="inference_models.models.yolo26.yolo26_semantic_segmentation_trt",
+        class_name="YOLO26ForSemanticSegmentationTRT",
+    ),
     ("yololite", OBJECT_DETECTION_TASK, BackendType.ONNX): RegistryEntry(
         model_class=LazyClass(
             module_name="inference_models.models.yololite.yololite_object_detection_onnx",

inference_models/inference_models/models/common/roboflow/model_packages.py

@@ -28,6 +28,29 @@ def parse_class_names_file(class_names_path: str) -> List[str]:
         ) from error
 
 
+def resolve_background_class_id(class_names: List[str]) -> int:
+    """Return the index of the `background` class for a semantic-segmentation
+    package.
+
+    Roboflow semantic segmentation maps every pixel to a class id, with one
+    class reserved for background (sub-threshold / unlabeled pixels). A valid,
+    in-range background id is required - a negative sentinel would alias a real
+    class via negative indexing in downstream consumers (`class_names[-1]`,
+    palette LUTs, the 0=background platform convention), silently corrupting
+    the segmentation map. Packages must therefore declare a `background` class.
+    """
+    try:
+        return [c.lower() for c in class_names].index("background")
+    except ValueError as error:
+        raise CorruptedModelPackageError(
+            message="Semantic segmentation model package does not define a `background` class in "
+            "`class_names.txt`. A background class is required so that sub-threshold pixels map to a "
+            "valid class id. If you created the model package manually, prepend `background` to the class "
+            "names. If the weights are hosted on the Roboflow platform - contact support.",
+            help_url="https://inference-models.roboflow.com/errors/model-loading/#corruptedmodelpackageerror",
+        ) from error
+
+
 PADDING_VALUES_MAPPING = {
     "black edges": 0,
     "grey edges": 127,

inference_models/inference_models/models/common/roboflow/post_processing.py

@@ -7,6 +7,9 @@
 from inference_models.configuration import INFERENCE_MODELS_DEFAULT_CONFIDENCE
 from inference_models.entities import Confidence, ImageDimensions
 from inference_models.logger import LOGGER
+from inference_models.models.base.semantic_segmentation import (
+    SemanticSegmentationResult,
+)
 from inference_models.models.common.rle_utils import torch_mask_to_coco_rle
 from inference_models.models.common.roboflow.model_packages import (
     PreProcessingMetadata,
@@ -723,3 +726,148 @@ def _resolve_fallback_threshold(
         ):
             return recommended_parameters.confidence
         return default_confidence
+
+
+def post_process_semantic_segmentation_logits(
+    model_results: torch.Tensor,
+    pre_processing_meta: List[PreProcessingMetadata],
+    class_names: List[str],
+    background_class_id: int,
+    device: torch.device,
+    confidence: Confidence,
+    recommended_parameters: Optional[RecommendedParameters],
+    default_confidence: float,
+) -> List[SemanticSegmentationResult]:
+    """Shared post-processing for semantic-segmentation models that emit
+    (B, K, H, W) float logits. Used by DeepLabV3+ and YOLO26-sem.
+
+    Steps: crop out letterbox padding → resize back to pre-letterbox size →
+    softmax over classes → argmax → place into original-image canvas if
+    static_crop was applied → apply per-class confidence threshold
+    (sub-threshold pixels collapse to background_class_id).
+    """
+    confidence_filter = ConfidenceFilter(
+        confidence=confidence,
+        recommended_parameters=recommended_parameters,
+        default_confidence=default_confidence,
+    )
+    results: List[SemanticSegmentationResult] = []
+    for image_results, image_metadata in zip(model_results, pre_processing_meta):
+        inference_size = image_metadata.inference_size
+        mask_h_scale = model_results.shape[2] / inference_size.height
+        mask_w_scale = model_results.shape[3] / inference_size.width
+        mask_pad_top, mask_pad_bottom, mask_pad_left, mask_pad_right = (
+            round(mask_h_scale * image_metadata.pad_top),
+            round(mask_h_scale * image_metadata.pad_bottom),
+            round(mask_w_scale * image_metadata.pad_left),
+            round(mask_w_scale * image_metadata.pad_right),
+        )
+        _, mh, mw = image_results.shape
+        if (
+            mask_pad_top < 0
+            or mask_pad_bottom < 0
+            or mask_pad_left < 0
+            or mask_pad_right < 0
+        ):
+            image_results = torch.nn.functional.pad(
+                image_results,
+                (
+                    abs(min(mask_pad_left, 0)),
+                    abs(min(mask_pad_right, 0)),
+                    abs(min(mask_pad_top, 0)),
+                    abs(min(mask_pad_bottom, 0)),
+                ),
+                "constant",
+                background_class_id,
+            )
+            padded_mask_offset_top = max(mask_pad_top, 0)
+            padded_mask_offset_bottom = max(mask_pad_bottom, 0)
+            padded_mask_offset_left = max(mask_pad_left, 0)
+            padded_mask_offset_right = max(mask_pad_right, 0)
+            image_results = image_results[
+                :,
+                padded_mask_offset_top : image_results.shape[1]
+                - padded_mask_offset_bottom,
+                padded_mask_offset_left : image_results.shape[2]
+                - padded_mask_offset_right,
+            ]
+        else:
+            image_results = image_results[
+                :,
+                mask_pad_top : mh - mask_pad_bottom,
+                mask_pad_left : mw - mask_pad_right,
+            ]
+        if (
+            image_results.shape[1] != image_metadata.size_after_pre_processing.height
+            or image_results.shape[2] != image_metadata.size_after_pre_processing.width
+        ):
+            image_results = functional.resize(
+                image_results,
+                [
+                    image_metadata.size_after_pre_processing.height,
+                    image_metadata.size_after_pre_processing.width,
+                ],
+                interpolation=functional.InterpolationMode.BILINEAR,
+            )
+        image_results = torch.nn.functional.softmax(image_results, dim=0)
+        image_confidence, image_class_ids = torch.max(image_results, dim=0)
+        if len(class_names) == image_results.shape[0] + 1:
+            image_class_ids = image_class_ids + 1
+        if (
+            image_metadata.static_crop_offset.offset_x > 0
+            or image_metadata.static_crop_offset.offset_y > 0
+        ):
+            original_size_confidence_canvas = torch.zeros(
+                (
+                    image_metadata.original_size.height,
+                    image_metadata.original_size.width,
+                ),
+                device=device,
+                dtype=image_confidence.dtype,
+            )
+            original_size_confidence_canvas[
+                image_metadata.static_crop_offset.offset_y : image_metadata.static_crop_offset.offset_y
+                + image_confidence.shape[0],
+                image_metadata.static_crop_offset.offset_x : image_metadata.static_crop_offset.offset_x
+                + image_confidence.shape[1],
+            ] = image_confidence
+            original_size_confidence_class_id_canvas = (
+                torch.ones(
+                    (
+                        image_metadata.original_size.height,
+                        image_metadata.original_size.width,
+                    ),
+                    device=device,
+                    dtype=image_class_ids.dtype,
+                )
+                * background_class_id
+            )
+            original_size_confidence_class_id_canvas[
+                image_metadata.static_crop_offset.offset_y : image_metadata.static_crop_offset.offset_y
+                + image_class_ids.shape[0],
+                image_metadata.static_crop_offset.offset_x : image_metadata.static_crop_offset.offset_x
+                + image_class_ids.shape[1],
+            ] = image_class_ids
+            image_class_ids = original_size_confidence_class_id_canvas
+            image_confidence = original_size_confidence_canvas
+        threshold = confidence_filter.get_threshold(class_names)
+        if isinstance(threshold, torch.Tensor):
+            threshold = threshold.to(
+                dtype=image_confidence.dtype, device=image_confidence.device
+            )
+        below = image_confidence < (
+            threshold[image_class_ids.long()]
+            if isinstance(threshold, torch.Tensor)
+            else threshold
+        )
+        image_class_ids = image_class_ids.clone()
+        image_confidence = image_confidence.clone()
+        image_class_ids[below] = background_class_id
+        image_confidence[below] = 0.0
+        results.append(
+            SemanticSegmentationResult(
+                segmentation_map=image_class_ids,
+                confidence=image_confidence,
+            )
+        )
+    return results

inference_models/inference_models/models/deep_lab_v3_plus/deep_lab_v3_plus_segmentation_onnx.py

@@ -2,7 +2,6 @@
 from typing import List, Optional, Tuple, Union
 
 import torch
-from torchvision.transforms import functional
 
 from inference_models import ColorFormat, SemanticSegmentationModel
 from inference_models.configuration import (
@@ -31,8 +30,11 @@
     ResizeMode,
     parse_class_names_file,
     parse_inference_config,
+    resolve_background_class_id,
+)
+from inference_models.models.common.roboflow.post_processing import (
+    post_process_semantic_segmentation_logits,
 )
-from inference_models.models.common.roboflow.post_processing import ConfidenceFilter
 from inference_models.models.common.roboflow.pre_processing import (
     pre_process_network_input,
 )
@@ -92,10 +94,7 @@ def from_pretrained(
         class_names = parse_class_names_file(
             class_names_path=model_package_content["class_names.txt"]
         )
-        try:
-            background_class_id = [c.lower() for c in class_names].index("background")
-        except ValueError:
-            background_class_id = -1
+        background_class_id = resolve_background_class_id(class_names)
         inference_config = parse_inference_config(
             config_path=model_package_content["inference_config.json"],
             allowed_resize_modes={
@@ -194,128 +193,13 @@ def post_process(
         confidence: Confidence = "default",
         **kwargs,
     ) -> List[SemanticSegmentationResult]:
-        confidence_filter = ConfidenceFilter(
+        return post_process_semantic_segmentation_logits(
+            model_results=model_results,
+            pre_processing_meta=pre_processing_meta,
+            class_names=self._class_names,
+            background_class_id=self._background_class_id,
+            device=self._device,
             confidence=confidence,
             recommended_parameters=self.recommended_parameters,
             default_confidence=INFERENCE_MODELS_DEEP_LAB_V3_PLUS_DEFAULT_CONFIDENCE,
         )
-        results = []
-        for image_results, image_metadata in zip(model_results, pre_processing_meta):
-            inference_size = image_metadata.inference_size
-            mask_h_scale = model_results.shape[2] / inference_size.height
-            mask_w_scale = model_results.shape[3] / inference_size.width
-            mask_pad_top, mask_pad_bottom, mask_pad_left, mask_pad_right = (
-                round(mask_h_scale * image_metadata.pad_top),
-                round(mask_h_scale * image_metadata.pad_bottom),
-                round(mask_w_scale * image_metadata.pad_left),
-                round(mask_w_scale * image_metadata.pad_right),
-            )
-            _, mh, mw = image_results.shape
-            if (
-                mask_pad_top < 0
-                or mask_pad_bottom < 0
-                or mask_pad_left < 0
-                or mask_pad_right < 0
-            ):
-                image_results = torch.nn.functional.pad(
-                    image_results,
-                    (
-                        abs(min(mask_pad_left, 0)),
-                        abs(min(mask_pad_right, 0)),
-                        abs(min(mask_pad_top, 0)),
-                        abs(min(mask_pad_bottom, 0)),
-                    ),
-                    "constant",
-                    self._background_class_id,
-                )
-                padded_mask_offset_top = max(mask_pad_top, 0)
-                padded_mask_offset_bottom = max(mask_pad_bottom, 0)
-                padded_mask_offset_left = max(mask_pad_left, 0)
-                padded_mask_offset_right = max(mask_pad_right, 0)
-                image_results = image_results[
-                    :,
-                    padded_mask_offset_top : image_results.shape[1]
-                    - padded_mask_offset_bottom,
-                    padded_mask_offset_left : image_results.shape[2]
-                    - padded_mask_offset_right,
-                ]
-            else:
-                image_results = image_results[
-                    :,
-                    mask_pad_top : mh - mask_pad_bottom,
-                    mask_pad_left : mw - mask_pad_right,
-                ]
-            if (
-                image_results.shape[1]
-                != image_metadata.size_after_pre_processing.height
-                or image_results.shape[2]
-                != image_metadata.size_after_pre_processing.width
-            ):
-                image_results = functional.resize(
-                    image_results,
-                    [
-                        image_metadata.size_after_pre_processing.height,
-                        image_metadata.size_after_pre_processing.width,
-                    ],
-                    interpolation=functional.InterpolationMode.BILINEAR,
-                )
-            image_results = torch.nn.functional.softmax(image_results, dim=0)
-            image_confidence, image_class_ids = torch.max(image_results, dim=0)
-            if (
-                image_metadata.static_crop_offset.offset_x > 0
-                or image_metadata.static_crop_offset.offset_y > 0
-            ):
-                original_size_confidence_canvas = torch.zeros(
-                    (
-                        image_metadata.original_size.height,
-                        image_metadata.original_size.width,
-                    ),
-                    device=self._device,
-                    dtype=image_confidence.dtype,
-                )
-                original_size_confidence_canvas[
-                    image_metadata.static_crop_offset.offset_y : image_metadata.static_crop_offset.offset_y
-                    + image_confidence.shape[0],
-                    image_metadata.static_crop_offset.offset_x : image_metadata.static_crop_offset.offset_x
-                    + image_confidence.shape[1],
-                ] = image_confidence
-                original_size_confidence_class_id_canvas = (
-                    torch.ones(
-                        (
-                            image_metadata.original_size.height,
-                            image_metadata.original_size.width,
-                        ),
-                        device=self._device,
-                        dtype=image_class_ids.dtype,
-                    )
-                    * self._background_class_id
-                )
-                original_size_confidence_class_id_canvas[
-                    image_metadata.static_crop_offset.offset_y : image_metadata.static_crop_offset.offset_y
-                    + image_class_ids.shape[0],
-                    image_metadata.static_crop_offset.offset_x : image_metadata.static_crop_offset.offset_x
-                    + image_class_ids.shape[1],
-                ] = image_class_ids
-                image_class_ids = original_size_confidence_class_id_canvas
-                image_confidence = original_size_confidence_canvas
-            threshold = confidence_filter.get_threshold(self.class_names)
-            if isinstance(threshold, torch.Tensor):
-                threshold = threshold.to(
-                    dtype=image_confidence.dtype, device=image_confidence.device
-                )
-            below = image_confidence < (
-                threshold[image_class_ids.long()]
-                if isinstance(threshold, torch.Tensor)
-                else threshold
-            )
-            image_class_ids = image_class_ids.clone()
-            image_confidence = image_confidence.clone()
-            image_class_ids[below] = self._background_class_id
-            image_confidence[below] = 0.0
-            results.append(
-                SemanticSegmentationResult(
-                    segmentation_map=image_class_ids,
-                    confidence=image_confidence,
-                )
-            )
-        return results