Update examples for native SegMask and SegParser#850
Conversation
|
Re-checked generic-example YOLOv8 instance segmentation with depthai==3.8.0 + depthai-nodes main. Instance masks are visible natively without --overlay_mode, and the no-overlay pipeline does not include ApplyColormap / ImgFrameOverlay. I updated generic-example README/help so the instance segmentation example no longer uses --overlay_mode. I’ll continue checking the remaining instance-segmentation/custom annotation candidates to see if anything else can be safely simplified. |
|
Checked apps/object-volume-measurement-3d as another instance segmentation/custom annotation candidate. I don't think its AnnotationNode should be removed in this PR. It is not only a visual overlay path: it handles click selection, uses getCvSegmentationMask(), generates segmented RGB/depth for RGBD pointcloud creation, communicates selection modes to MeasurementNode, and draws measurement overlays. So for now I’ll keep it unchanged. If we want to adjust its background/mask handling, I’d treat that as a separate runtime-validated follow-up because it is part of the measurement pipeline. |
|
Let's run an LLM agent from the repo origin and instruct it to check if there are any example specific AGENTS.md file updates needed or any INDEX.md updates needed. And also say that it should ignore the temporary depthai-nodes dependency not using released version, this will go away before this branch is merged so it shouldn't be included in any of the AGENTS.md file changes (or anything else that LLMs might try to do) |
Checked this as well. Looks like no extra AGENTS.md / INDEX.md updates are needed here. The docs/index checks pass and regenerating the index didn’t produce any diff. Also kept the temporary depthai-nodes main dependency out of the docs as discussed. |
|
There are currently issues with HIL setup for RVC2 tests. Will ignore those tests for now and merge anyway |
Purpose
Update
oak-examplesfor the new native segmentation flow in DepthAI /depthai-nodes.The relevant
depthai-nodeschanges are now merged inmainvia luxonis/depthai-nodes#304. This PR updates examples that useSegmentationMask/SegmentationParserso they work with nativedai.SegmentationMask, nativedai.node.SegmentationParser, background value255, and DAI 3.8 Visualizer support.This PR is opened now so the current state can be reviewed while I continue checking whether any instance segmentation examples can be simplified further with DAI 3.8 native
ImgDetectionsmask visualization.Specification
Updated DeepLab-based examples:
neural-networks/segmentation/blur-backgroundneural-networks/segmentation/depth-cropdai.SegmentationMaskgetCvMask()setBackgroundClass(False)where class ids need to be preserved for manual example logicUpdated
apps/dino-trackingFastSAM mask handling:dai.SegmentationMaskgetCvMask()255as background during click-based object selectionUpdated YOLO-P:
AnnotationNodeVideoDetectionsRoad SegmentationLane SegmentationAGENTS.mdaccordinglyUpdated
generic-exampledocs/help:--overlay_mode--overlay_moderemains present for now while instance segmentation visualization is re-checked with DAI 3.8Temporarily updated touched example requirements to target:
depthai==3.8.0depthai-nodesfrommaindepthai-nodesversion once a release containing Use native segmentation parser and mask depthai-nodes#304 is availableDependencies & Potential Impact
Temporarily depends on
depthai-nodesfrommainbecause luxonis/depthai-nodes#304 is merged but a releaseddepthai-nodespackage containing those changes is not available yet.Potential impact:
depthai-nodesversion before final merge/release, or once the expected package is availableTesting & Validation
Validation performed so far:
compileallon touched Python directories passedgit diff --checkpassedvalidate_agents_docs.pypassed in a clean worktreegenerate_agents_index.py --checkpassed in a clean worktreedepthai==3.8.0depthai-nodesmain containing Use native segmentation parser and mask depthai-nodes#304Runtime/manual validation performed:
generic-example--overlay_modeImgDetectionsmask visualization is still in progressFollow-up validation in this PR:
ImgDetectionsmask visualizationDeployment Plan
No deployment required.
Rollout:
depthai==3.8.0anddepthai-nodesmaindepthai-nodesmain dependency with the released version once availableRollback:
Monitoring:
AI Usage
Assisted-by: ChatGPT