Integrate Ultralytics Backend (OD + IS) by kprokofi · Pull Request #6648 · open-edge-platform/geti

kprokofi · 2026-06-02T13:24:29Z

Integrate Ultralytics Backend for Instance Segmentation and Object Detection

Summary

How to test

Checklist

The PR title and description are clear and descriptive
I have manually tested the changes
All changes are covered by automated tests
All related issues are linked to this PR (if applicable)
Documentation has been updated (if applicable)

- Add 'ultralytics>=8.2,<9' optional dependency group to library/pyproject.toml - Regenerate uv.lock (ultralytics 8.4.41 + ultralytics-thop 2.0.18) - Existing opencv-python exclusion handles ultralytics transitive dep - No conflicts with pinned torch/torchvision/numpy - Add benchmark script and COCO-to-YOLO converter in scripts/

- Create UltralyticsModel base class with lazy YOLO initialization - Create UltralyticsDetectionModel (yolo11s.pt, task=detect) - Create UltralyticsInstSegModel (yolo11s-seg.pt, task=segment) - Implement UltralyticsEngine with train/test/predict/export methods - Register UltralyticsEngine in create_engine() with guarded import - Update MODEL type union in types/types.py with guarded import - Metric key mappings from Ultralytics -> getitune format - All files lint-clean (ruff + pyrefly, 0 new errors) - 1206 unit tests pass with no regressions

…r stubs - Create custom trainer stubs (DetectionTrainer, SegmentationTrainer) with DataModule slot for Phase 2 data bridge wiring - Add trainer_cls attribute to UltralyticsModel base and both model wrappers - Route engine.train() through custom trainer via _make_bound_trainer() factory (dynamic subclass with DataModule as class variable) - Normalize engine.export() to accept ExportFormat + Precision and return concrete file paths (.xml for OpenVINO, .onnx for ONNX) - Extract _convert_predictions(), _normalize_openvino_export(), _normalize_onnx_export() helpers - Move imports to module scope, use f-strings, clean up docstrings - All lint (ruff + pyrefly) and 1227 unit tests pass

…ators Implement the data bridge layer connecting getitune VisionDataset to Ultralytics trainer/validator internals: - data/adapter.py: UltralyticsDatasetAdapter wrapping VisionDataset samples into Ultralytics sample dicts (float32 CHW passthrough) - data/collate.py: custom collate_fn producing Ultralytics batch dicts with correct batch_idx, bypassing getitune SampleBatch - data/geometry.py: xyxy_abs_to_xywh_norm and build_ratio_pad utilities - trainers/detection.py: full override of get_dataset, build_dataset, get_dataloader, preprocess_batch (skip /255), get_validator, set_class_weights, plot_training_labels, auto_batch - trainers/instance_segmentation.py: same pattern with include_masks=True and overlap_mask=False - validators/detection.py: DetectionValidator with preprocess override skipping /255 normalization - validators/instance_segmentation.py: SegmentationValidator with same plus masks-to-float cast - Updated pyrefly baseline for intentional DataLoader return type widening

Bug fixes: - Set close_mosaic=0 to prevent DataLoader.reset() crash - Wire engine.test() to DataModule via bound validator pattern - Wire engine.predict() to iterate DataModule test/val subset - Fix geometry: use img_info.img_shape for ratio_pad, not tensor shape PR review fixes: - Use VisionDataset type in adapter instead of TorchDataset - Keep masks in torch flow, remove numpy roundtrip - Remove unused logger, batch_idx from adapter - Remove task param from __init__, make ClassVar property only - Remove yolo.setter and no-op _apply_label_info - Add validator_cls to model classes - Rewrite docstrings: short, precise, Google-style

- Use multiprocessing_context='spawn' + persistent_workers in DataLoaders to prevent Datumaro dataset hang with workers>0 (matches Lightning backend) - Set _datamodule on validator in get_validator() so final_eval() uses our custom DataModule path instead of downloading coco8.yaml - Resolve work_dir to absolute path so Ultralytics save_dir logic skips the runs/detect/ prefix - Fix Profile import from ultralytics.utils.ops (was torch_utils) in validators

- make UltralyticsModel honest about pretrained=False by requiring a model config instead of silently accepting checkpoint weights - switch DataModule-backed predict() to predict_dataloader() so Ultralytics inference follows the getitune DataModule contract - preserve DataModule images/img_info when converting predictions back to getitune Prediction objects - add focused unit coverage for the data bridge, model validation, and DataModule-backed prediction flow

- Engine._resolve_device() now returns torch.device; resolution order: XPU > CUDA > CPU (matches getitune convention). Accepts DeviceType enum and all common string forms (xpu, xpu:0, cuda, 0, cpu, auto). torch.device objects bypass Ultralytics select_device() validation. - XPUAwareTrainerMixin added to DetectionTrainer and SegmentationTrainer MRO. Overrides: * train(): wraps training loop in torch.amp.autocast('xpu', bf16) * _setup_train(): replaces CUDA GradScaler with disabled XPU scaler * _get_memory(): torch.xpu.memory_reserved() branch * _clear_memory(): torch.xpu.empty_cache() branch * preprocess_batch(): non_blocking on both CUDA and XPU - 26 new unit tests covering all device resolution paths, GradScaler replacement, memory utilities, autocast wrapping, and non_blocking.

…Phase 5) - UltralyticsConfig / UltralyticsTrainConfig / etc. dataclasses in config.py - UltralyticsConfigurator with from_recipe(), create_model(), create_engine() - Geti bridge: apply_geti_overrides() maps UI hyper_parameters to Ultralytics config, to_geti_config() produces GetiConfigConverter-compatible output dict - Base data recipe _base_/data/ultralytics_detection.yaml (no ImageNet norm) - YOLO26 detection recipes: yolo26_n, yolo26_s, yolo26_m - 53 configurator unit tests (config, parsing, overrides, Geti bridge, model/engine creation, real recipe validation)

- Delete ultralytics_config_adapter.py; GetiConfigConverter now handles Ultralytics recipes directly using the same hyper_parameters keys as Lightning (learning_rate, batch_size, max_epochs, weight_decay, etc.) - Delete ultralytics_detection.yaml; YOLO26 recipes now reference the shared _base_/data/detection.yaml with inline overrides.data section - Revert all model.pt handling; store_model_artifacts always copies to model.ckpt regardless of source extension; __build_model_weights_path and _get_variant_binary_files only look for model.ckpt - Update tests: rename converter test, remove model.pt parametrization from model_service integration test, remove prefers_pt trainer test, store_model_artifacts always asserts model.ckpt

… externally Recipes now use .yaml model configs (not .pt) with pretrained=false so Ultralytics never downloads weights internally. BaseWeightsService downloads .pt files from manifest URLs, and the new UltralyticsModel.load_checkpoint() injects them into the model built from the architecture config. - Recipes: model_name .pt -> .yaml, pretrained: false - UltralyticsModel: add load_checkpoint(weights_path) via YOLO.load() - UltralyticsConfigurator.create_model(): build from config, then load checkpoint when weights_path provided (remove _resolve_model_name) - UltralyticsDetectionModel: default_model_name .pt -> .yaml - Tests: update configurator/model tests for new flow

Removes all _apply_ultralytics_* methods and _set_ultralytics_cpu_augmentation from geti_config_converter.py (~190 lines deleted). The mapping from standard Geti hyper_parameters (learning_rate, batch_size, max_epochs, etc.) to Ultralytics-specific config fields (lr0, batch, epochs, etc.) now lives in the library at UltralyticsConfigAdapter. The application converter just calls UltralyticsConfigAdapter.convert(), keeping the same contract for all backends. This ensures adding new backends does not require hardcoding backend-specific knowledge in the application. - New: library/src/getitune/backend/ultralytics/config_adapter.py - New: library/tests/unit/backend/ultralytics/test_config_adapter.py (27 tests) - Simplified _build_ultralytics_configurator to one-liner delegation - Removed unused yaml/copy imports from converter

The adapter was too thin (~40 lines of actual logic after augmentation handling was removed) to justify a separate module. All methods now live on UltralyticsConfigurator as instance/class methods: - is_ultralytics_recipe() -> static method - convert() -> classmethod (from_recipe + apply_hyper_parameters + to_config_dict) - apply_hyper_parameters() -> instance method (mutates self) - to_config_dict() -> instance method (reads self.config/data_config) Deleted config_adapter.py and its test file; merged all 24 adapter tests into test_configurator.py. Updated app-side imports in geti_config_converter.py. All 130 library + 40 converter + 44 trainer tests pass, lint clean.

…st() signature - Add Engine.best_checkpoint property (non-abstract, returns None by default) - Override in LightningEngine (returns checkpoint callback path) and UltralyticsEngine (returns _last_train_checkpoint with fallback) - Add metric parameter to UltralyticsEngine.test() for API compatibility (accepted but ignored — Ultralytics computes metrics internally) - Extract _train_lightning_model() for symmetry with _train_ultralytics_model() - Remove isinstance(engine, UltralyticsEngine) check from evaluate_model() - Remove UltralyticsEngine import from app trainer and tests - Fix test fixture: yolo26n.pt -> yolo26n.yaml - All backends now accessed through uniform Engine interface

Add ModelAPI-compatible metadata embedding for Ultralytics exports so that Model.create_model(adapter) auto-detects YOLO11 wrapper and the Geti inference pipeline works without code changes. Key changes: - New export.py: build_export_metadata() + embed_{openvino,onnx}_metadata() - Engine exports with end2end=False (pre-NMS output for ModelAPI) - Fix scale_values to use ModelAPI divisor convention (255.0, not 1/255) - Fix cascading import breakages from upstream OTX→getitune renames (TaskType, VisionDataset, DataModule, ExportFormat, Precision, etc.) - Add export_model_type/export_task_type properties to UltralyticsModel - 17 new unit tests for metadata building and embedding - Update existing export tests for end2end=False and metadata mocking

…xport Wire Ultralytics export thresholds through recipe/config into metadata so ModelAPI postprocessing uses the configured confidence and IoU values. Default values now match the upstream YOLO11 wrapper (0.25 / 0.7). Support FP16-compatible ModelAPI export by keeping the exported graph FP16 internally while casting OpenVINO and ONNX result tensors back to f32, which satisfies the YOLO11 wrapper output precision contract without adding a custom wrapper. Also blocks instance-segmentation export until its ModelAPI wrapper contract is validated and adds unit coverage for the new export paths.

…cs-backend

…esolution OVModel._resolve_model_type() reads model_type from IR rt_info metadata (e.g. YOLO11) and falls back to the class default (e.g. SSD) for older IRs without the field. _FP32OpenvinoAdapter no longer rejects uint8-scale preprocessing (scale_values=[255]) so YOLO models work through the standard OV inference path. Also cleans up Ultralytics export metadata: uses model.export_model_type property instead of a dict lookup, renames otx_version to getitune_version, and adds export_task_type to UltralyticsInstSegModel.

…tning Replace the function-based export.py with a class-based UltralyticsModelExporter(ModelExporter) that follows the same architecture as LightningModelExporter: - Add data_input_params and _export_parameters properties to UltralyticsModel, using DataInputParams and TaskLevelExportParameters (same dataclasses as Lightning models). - Create exporter/ package with UltralyticsModelExporter that inherits ModelExporter's metadata embedding, preprocessing params, and postprocessing pipeline — no hardcoded constants. - FP16 handled by openvino.save_model(compress_to_fp16=True) and onnxconverter_common, not manual Cast-node insertion. - Simplify engine.py export() to delegate to the new exporter class; remove 6 helper methods and _ULTRALYTICS_FORMAT_MAP. - Delete export.py entirely (replaced by exporter class). - Rewrite tests: new test_exporter.py, updated test_engine.py and test_models.py. 172/172 ultralytics + 44/44 backend tests pass.

Remove the NotImplementedError guard that blocked quantization of Ultralytics models. The full OVEngine → OVModel → NNCF pipeline is backend-agnostic and works on the exported OpenVINO IR which already has correct metadata (optimization_config: '{}').

Two critical bugs fixed: 1. scale_values metadata: DataInputParams.std was (1,1,1), causing ModelAPI to skip /255 normalization at inference. Changed to (255,255,255) so exported IR correctly divides uint8 inputs by 255. 2. Adapter bbox rescaling: val/test subsets use resize_targets=False, so bboxes stay in original image coordinates while images are resized to 640x640. The adapter now detects canvas_size != tensor dims and rescales bboxes to tensor space before XYWH normalization. This fixes val mAP being always 0. Added unit test for the canvas_size != tensor_size rescaling path.

Address PR #6222 review feedback: - Remove standalone export_model_type, export_task_type, num_classes properties from UltralyticsModel base (inline in _export_parameters) - Add _pretrained_weights ClassVar with per-variant URLs (same pattern as Lightning's YOLOX, RFDETR, etc.) - Add _default_preprocessing_params property returning per-model DataInputParams dict (same pattern as Lightning) - _build_yolo() now auto-loads pretrained weights from URLs when pretrained=True - data_input_params resolves from _default_preprocessing_params with explicit imgsz override support - Remove all section divider comments from source and test files - Remove stale plugins/ directory (duplicate of trainers/xpu_mixin.py) - Fix ruff lint issues (SIM117, I001, formatting) - Add TestPretrainedWeights test class verifying the new pattern All 178 ultralytics unit tests pass.

Implement full IS export pipeline for Ultralytics backend: - Custom YOLO11Seg ModelAPI wrapper that decodes mask prototypes, applies NMS, crops+resizes masks to original image resolution, and returns InstanceSegmentationResult with 1-indexed labels (MaskRCNN convention). - Register wrapper import in OV IS model so ModelAPI discovers it via model_type='YOLO11-seg' in IR metadata. - Remove NotImplementedError guard from engine.export() for segmentation. - Fix val/test mask resolution mismatch in adapter: resize masks from original image dims to tensor dims using bilinear interpolation when the DataModule delivers unresized masks for evaluation subsets.

YOLO26-seg models have an auxiliary semantic segmentation loss head (BCEDiceLoss on per-pixel class labels) that requires a 'sem_masks' tensor in the training batch. Generate it from instance masks + class labels in the adapter, and stack it in the collate function. The sem_masks tensor is produced at the same spatial resolution as instance masks to match the loss function's boolean indexing for background zeroing. Also add a unit test verifying mask decode correctness for non-square (letterboxed) images.

Resolve conflicts after #6319 (remove _FP32OpenvinoAdapter) merged: - Accept develop's removal of _FP32OpenvinoAdapter and _patch_pad_constant_type - Keep _resolve_model_type (needed for YOLO11/YOLO11-seg dynamic dispatch) - Fix type annotation: _FP32OpenvinoAdapter -> OpenvinoAdapter - Remove TestFP32OpenvinoAdapter tests (class no longer exists) - Add intensity_config to DataInputParams in Lightning trainer path

….744 / 0.463 and IS, 0.773 / 0.377.

Copilot

Pull request overview

Integrates an optional Ultralytics-based backend into the library/ (getitune) training stack for object detection and instance segmentation, and wires it through the application/backend/ training + model artifact workflows (including export and INT8 PTQ paths).

Changes:

Added Ultralytics engine/model wrappers plus a DataModule bridge (adapter/collate/geometry), custom trainers/validators, and an Ultralytics exporter (OpenVINO/ONNX + metadata).
Updated recipes/manifests and backend plumbing to support the new Ultralytics variants, plus XPU-specific training support.
Adjusted checkpoint/artifact naming conventions toward .pt, and added a Datumaro COCO multi-polygon bbox workaround + related tests.

Reviewed changes

Copilot reviewed 87 out of 90 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
library/tests/unit/data/test_coco_bbox_fix.py	Unit tests for the Datumaro COCO multi-polygon bbox monkey-patch.
library/tests/unit/backend/ultralytics/test_trainers.py	Regression tests for Ultralytics trainer customizations (preprocess, memory clear, progress, XPU transfer).
library/tests/unit/backend/ultralytics/test_models.py	Unit tests for Ultralytics model wrapper behaviors (checkpoint loading, export params, preprocessing params).
library/tests/unit/backend/ultralytics/test_data_bridge.py	Unit tests for dataset adapter + collate contract (bbox normalization, letterbox padding, ratio_pad).
library/tests/unit/backend/ultralytics/conftest.py	Skips Ultralytics unit tests when `ultralytics` isn’t installed.
library/tests/unit/backend/ultralytics/init.py	Package init for Ultralytics unit tests.
library/tests/unit/backend/openvino/models/test_base.py	Adds tests for resolving `model_type` from IR/ONNX metadata.
library/tests/regression/test_regression.py	Updates expected best-checkpoint filename to `.pt`.
library/tests/perf_v2/benchmark.py	Updates expected best-checkpoint filename to `.pt`.
library/tests/integration/api/test_ultralytics_engine.py	End-to-end integration test for Ultralytics engine workflow (train/test/predict/export/OV/INT8).
library/src/getitune/types/types.py	Extends the `MODEL` type union to include optional Ultralytics model type.
library/src/getitune/types/export.py	Ensures label IDs are stringified when building metadata.
library/src/getitune/tools/auto_configurator.py	Extends OV subset pipeline updater to support centered letterbox padding + pad value propagation.
library/src/getitune/recipe/instance_segmentation/yolo26_s_seg.yaml	New Ultralytics inst-seg recipe variant (S).
library/src/getitune/recipe/instance_segmentation/yolo26_n_seg.yaml	New Ultralytics inst-seg recipe variant (N).
library/src/getitune/recipe/instance_segmentation/yolo26_m_seg.yaml	New Ultralytics inst-seg recipe variant (M).
library/src/getitune/recipe/detection/yolo26_s.yaml	New Ultralytics detection recipe variant (S).
library/src/getitune/recipe/detection/yolo26_n.yaml	New Ultralytics detection recipe variant (N).
library/src/getitune/recipe/detection/yolo26_m.yaml	New Ultralytics detection recipe variant (M).
library/src/getitune/models/init.py	Removes `RTMDet` from exports list (aligns exposed model set).
library/src/getitune/engine/engine.py	Adds a cross-backend `best_checkpoint` property hook to the Engine API.
library/src/getitune/engine/init.py	Adds optional UltralyticsEngine to `create_engine()` discovery.
library/src/getitune/data/module.py	Applies Datumaro COCO bbox fix and adjusts subset mapping logic.
library/src/getitune/data/dataset/base.py	Fixes datumaro parquet middle-channel layout permutation.
library/src/getitune/data/augmentation/transforms.py	Adds centered letterbox padding to Resize, adds bbox filtering transform, adds CachedMosaic `area_thr`.
library/src/getitune/data/_coco_bbox_fix.py	Implements Datumaro COCO multi-polygon bbox union fix + dedup logic via monkey-patch.
library/src/getitune/backend/ultralytics/validators/base.py	Standalone evaluation logic for validators using DataModule (no YAML dataset file).
library/src/getitune/backend/ultralytics/validators/detection.py	Detection validator subclass wired to GetiTuneValidatorMixin.
library/src/getitune/backend/ultralytics/validators/instance_segmentation.py	Instance-seg validator subclass wired to GetiTuneValidatorMixin.
library/src/getitune/backend/ultralytics/validators/init.py	Exports custom Ultralytics validators.
library/src/getitune/backend/ultralytics/trainers/base.py	DataModule bridge mixin: dataloader, warmup, progress callback, mem clear, grad clip, augmentation disabling.
library/src/getitune/backend/ultralytics/trainers/detection.py	Detection trainer bridge (skip /255 preprocessing when using DataModule).
library/src/getitune/backend/ultralytics/trainers/instance_segmentation.py	Segmentation trainer bridge (mask downsample to proto resolution; overlap masks).
library/src/getitune/backend/ultralytics/trainers/init.py	Exports custom Ultralytics trainers.
library/src/getitune/backend/ultralytics/plugins/xpu_mixin.py	Adds XPU bf16 training support for Ultralytics trainers.
library/src/getitune/backend/ultralytics/plugins/init.py	Exports Ultralytics plugin mixins.
library/src/getitune/backend/ultralytics/models/base.py	Base Ultralytics model wrapper (YOLO build/load/export, preprocessing params, export metadata).
library/src/getitune/backend/ultralytics/models/detection.py	Ultralytics YOLO detection model wrapper (variants + metrics mapping).
library/src/getitune/backend/ultralytics/models/instance_segmentation.py	Ultralytics YOLO instance-seg model wrapper (variants + export params + metrics mapping).
library/src/getitune/backend/ultralytics/models/init.py	Exports Ultralytics model wrappers.
library/src/getitune/backend/ultralytics/data/adapter.py	Converts VisionDataset samples to Ultralytics sample dicts (bboxes/ratio_pad/masks).
library/src/getitune/backend/ultralytics/data/collate.py	Collates Ultralytics batches; builds overlap masks and semantic maps.
library/src/getitune/backend/ultralytics/data/geometry.py	Bbox conversion and ratio_pad helpers.
library/src/getitune/backend/ultralytics/data/init.py	Exports Ultralytics data bridge utilities.
library/src/getitune/backend/ultralytics/exporter/exporter.py	Ultralytics exporter to OpenVINO/ONNX with metadata embedding + FP16 compression.
library/src/getitune/backend/ultralytics/exporter/init.py	Exports UltralyticsModelExporter.
library/src/getitune/backend/ultralytics/tools/utils.py	Recipe loading/override utilities for Ultralytics via OmegaConf.
library/src/getitune/backend/ultralytics/tools/init.py	Ultralytics tools package init.
library/src/getitune/backend/ultralytics/init.py	Ultralytics backend package exports.
library/src/getitune/backend/openvino/models/base.py	Adds `center_padding` + `model_type` resolution from metadata; type-hint refinements.
library/src/getitune/backend/openvino/models/detection.py	Adjusts ONNX metadata parsing for confidence threshold.
library/src/getitune/backend/openvino/models/instance_segmentation.py	Adjusts ONNX metadata parsing for confidence threshold; minor cleanup.
library/src/getitune/backend/openvino/engine.py	Passes `center_padding` into OV subset pipeline update calls.
library/src/getitune/backend/lightning/engine.py	Supports loading “plain pretrained weights” vs getitune checkpoints; creates `best_checkpoint.pt` symlink; implements `best_checkpoint` property.
library/pyproject.toml	Adds `ultralytics` extra; updates OpenVINO Model API pin; adds onnxslim dependency.
library/Justfile	Ensures venv sync includes the `ultralytics` extra.
application/backend/tests/unit/services/data_collect/test_prediction_converter.py	Adds test for underscore-vs-space label name matching in predictions.
application/backend/tests/unit/execution/training/test_getitune_trainer.py	Updates tests for `.pt` naming and unified `create_engine` path; adds Ultralytics training/eval tests.
application/backend/tests/unit/execution/quantization/test_getitune_quantizer.py	Extends tests for INT8 artifact storage + metadata.yaml propagation/update.
application/backend/tests/unit/execution/common/test_getitune_converters.py	Adds test for preserving per-class metric suffix when stripping phase prefix.
application/backend/tests/unit/execution/common/test_geti_config_converter.py	Adds tests for Ultralytics recipe conversion + transforms updater behavior.
application/backend/tests/unit/api/schemas/test_training_configuration_view.py	Updates mosaic default values and descriptions (scale/translate).
application/backend/tests/unit/api/routers/test_models.py	Updates PyTorch model artifact filename to `model.pt` in download tests.
application/backend/tests/unit/api/routers/test_model_architectures.py	Updates expected model counts for newly added Ultralytics variants.
application/backend/tests/integration/services/test_model_service.py	Updates expected PyTorch artifact naming to `model.pt`.
application/backend/pyproject.toml	Pulls `getitune[...,ultralytics]` in cpu/xpu/cuda dependency groups.
application/backend/app/supported_models/manifests/detection/yolo26_n.yaml	Adds new detection model manifest (N).
application/backend/app/supported_models/manifests/detection/yolo26_s.yaml	Adds new detection model manifest (S).
application/backend/app/supported_models/manifests/detection/yolo26_m.yaml	Adds new detection model manifest (M).
application/backend/app/supported_models/manifests/instance_segmentation/yolo26_n.yaml	Adds new instance-seg model manifest (N).
application/backend/app/supported_models/manifests/instance_segmentation/yolo26_s.yaml	Adds new instance-seg model manifest (S).
application/backend/app/supported_models/manifests/instance_segmentation/yolo26_m.yaml	Adds new instance-seg model manifest (M).
application/backend/app/services/model_service.py	Adds new metric display keys; includes metadata.yaml in OpenVINO/ONNX bundles; switches to model.pt.
application/backend/app/services/media_prediction_service.py	Fixes cv2.imread call by casting Path to str.
application/backend/app/services/inference/inference_server.py	Adapts to variable-length OpenVINO variant file lists (xml/bin/metadata).
application/backend/app/services/data_collect/prediction_converter.py	Adjusts label resolution docstring/comment formatting and underscore handling behavior.
application/backend/app/models/training_configuration/augmentation.py	Updates Mosaic parameter docs/ranges (translate max + descriptions).
application/backend/app/execution/training/getitune_trainer.py	Uses library `create_engine` for unified backend support; stores PyTorch model as `model.pt`; handles metrics.csv fallback.
application/backend/app/execution/quantization/getitune_quantizer.py	Copies/updates metadata.yaml into INT8 variant artifacts.
application/backend/app/execution/common/getitune_converters.py	Fixes metric key stripping to preserve class-name suffix (`split('/', 1)`).

leoll2 · 2026-06-08T11:36:26Z

        pytorch_variant_dir = variants_dir / str(created_variants[ModelFormat.PYTORCH])
        pytorch_variant_dir.mkdir(parents=True, exist_ok=True)
-        shutil.copyfile(trained_model_path, pytorch_variant_dir / "model.ckpt")
+        shutil.copyfile(trained_model_path, pytorch_variant_dir / "model.pt")


Does the library now save checkpoints with .pt extension instead of .ckpt for all model architectures? It looks like a minor thing but it could actually become a breaking change if this code will be released in 3.1+ instead of 3.0. Something to keep an eye on.

Yes, all models are now "pt" as a more standard checkpoint name in industry + it is easier to change sym link in lightning backend than change the ultralytics behavior

… backends

leoll2 · 2026-06-08T14:00:20Z

+            try:
+                image = cv2.imread(file)
+            except FileNotFoundError:
+                # Ultralytics patches cv2.imread with np.fromfile which raises
+                # FileNotFoundError on missing/corrupt files instead of returning None.


Can you share a reference to this patched code? If it's true, I'm shocked that Ultralytics developers think it's a good idea to casually hijack the methods of another library.

Yes, @warrkan already fixed one problem caused by their patch.
https://github.com/ultralytics/ultralytics/blob/29d792b3b00b6aea742a2ff9367d4d1e9e22f372/ultralytics/utils/__init__.py#L1495
https://github.com/ultralytics/ultralytics/blob/main/ultralytics/utils/patches.py

It patches OpenCV only on Windows for some reasons. That's why tests were failed on Windows and passed on Linux

kprokofi added 30 commits April 22, 2026 20:43

Add export pipeline with checkpoint resolution and unit tests (Phase 4)

b106472

Finalize Phase 3/4 trainer fallback and export checkpoint handling

7067515

Tighten Ultralytics configurator boundaries and engine defaults

8cd1de6

Integrate Ultralytics backend into training service

131cb7d

Merge remote-tracking branch 'origin/develop' into kprokofi/ultralyti…

6d7822f

…cs-backend

Fix stale OTX import aliases after merging develop (#6135 rename)

624da49

leoll2 reviewed Jun 8, 2026

View reviewed changes

change default values in manifests

df32508

AlbertvanHouten requested a review from Copilot June 8, 2026 11:31

Copilot started reviewing on behalf of AlbertvanHouten June 8, 2026 11:31 View session

simplify mask handling. Reply comments. WGISD training metrics: OD, 0…

fe5663c

….744 / 0.463 and IS, 0.773 / 0.377.

Copilot AI reviewed Jun 8, 2026

View reviewed changes

leoll2 reviewed Jun 8, 2026

View reviewed changes

reply some comments, resolve metadata validation, unify metrics for 2…

16e23c3

… backends

leoll2 reviewed Jun 8, 2026

View reviewed changes

AlbertvanHouten reviewed Jun 8, 2026

View reviewed changes

kprokofi added 11 commits June 8, 2026 23:40

update model api, remove hardcoded train / val/ test subsets name

0bd6106

reply comments

fcdb173

return scale masks to orig size back

7887a87

remove datumaro workaround

bf3dbff

fix to susbsets

428678c

add checkpoint method to OVEngine

b14d1b0

change modelapi version for demo

02d5c1b

restore workarount

776536f

resolve checkpoint loading

c7868d5

cap threshold to avoid high values:

49b53e1

minor fix

faa043e

kprokofi requested review from AlbertvanHouten and leoll2 June 9, 2026 16:57

AlbertvanHouten approved these changes Jun 10, 2026

View reviewed changes

leoll2 approved these changes Jun 10, 2026

View reviewed changes

kprokofi added this pull request to the merge queue Jun 10, 2026

Merged via the queue into develop with commit 42dc6b0 Jun 10, 2026
56 checks passed

kprokofi deleted the kprokofi/ultralytics-production branch June 10, 2026 10:47

leoll2 linked an issue Jul 14, 2026 that may be closed by this pull request

Introduce Ultralytics Engine #4454

Closed

Uh oh!

Conversation

kprokofi commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How to test

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

leoll2 Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

kprokofi Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

leoll2 Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

kprokofi Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

kprokofi Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

kprokofi commented Jun 2, 2026 •

edited

Loading