Skip to content

Update examples for native SegMask and SegParser#850

Merged
klemen1999 merged 9 commits into
mainfrom
feat/86cajeq9j-native-segmask-segparser-examples
Jul 17, 2026
Merged

Update examples for native SegMask and SegParser#850
klemen1999 merged 9 commits into
mainfrom
feat/86cajeq9j-native-segmask-segparser-examples

Conversation

@rolandocortez

Copy link
Copy Markdown
Contributor

Purpose

Update oak-examples for the new native segmentation flow in DepthAI / depthai-nodes.

The relevant depthai-nodes changes are now merged in main via luxonis/depthai-nodes#304. This PR updates examples that use SegmentationMask / SegmentationParser so they work with native dai.SegmentationMask, native dai.node.SegmentationParser, background value 255, and DAI 3.8 Visualizer support.

This PR is opened now so the current state can be reviewed while I continue checking whether any instance segmentation examples can be simplified further with DAI 3.8 native ImgDetections mask visualization.

Specification

  • Updated DeepLab-based examples:

    • neural-networks/segmentation/blur-background
    • neural-networks/segmentation/depth-crop
    • migrated to native dai.SegmentationMask
    • replaced old mask access with getCvMask()
    • added setBackgroundClass(False) where class ids need to be preserved for manual example logic
  • Updated apps/dino-tracking FastSAM mask handling:

    • migrated to native dai.SegmentationMask
    • replaced old mask access with getCvMask()
    • handles 255 as background during click-based object selection
  • Updated YOLO-P:

    • removed the custom AnnotationNode
    • sends outputs directly to Visualizer topics:
      • Video
      • Detections
      • Road Segmentation
      • Lane Segmentation
    • updated AGENTS.md accordingly
  • Updated generic-example docs/help:

    • semantic segmentation example no longer uses --overlay_mode
    • --overlay_mode remains present for now while instance segmentation visualization is re-checked with DAI 3.8
  • Temporarily updated touched example requirements to target:

Dependencies & Potential Impact

Temporarily depends on depthai-nodes from main because luxonis/depthai-nodes#304 is merged but a released depthai-nodes package containing those changes is not available yet.

Potential impact:

  • touched examples now depend on DAI 3.8 behavior
  • temporary Git dependency should be replaced with the final released depthai-nodes version before final merge/release, or once the expected package is available
  • examples using manual/custom visualization were updated only where validation showed the native path was appropriate

Testing & Validation

Validation performed so far:

Runtime/manual validation performed:

  • generic-example
    • default detection run checked
    • semantic segmentation run checked without --overlay_mode
    • instance segmentation behavior checked; follow-up re-check with DAI 3.8 native ImgDetections mask visualization is still in progress
  • YOLO-P
    • checked with native Visualizer topics after removing the custom annotation node

Follow-up validation in this PR:

  • re-check instance segmentation examples with DAI 3.8 native ImgDetections mask visualization
  • inspect whether any remaining custom overlay / annotation paths can be simplified or removed

Deployment Plan

No deployment required.

Rollout:

  • review and test this PR against depthai==3.8.0 and depthai-nodes main
  • replace the temporary depthai-nodes main dependency with the released version once available
  • merge after review and CI validation

Rollback:

  • revert this PR if the new native segmentation behavior causes regressions in the updated examples

Monitoring:

  • CI results
  • manual/runtime validation of affected examples
  • reviewer feedback on instance segmentation visualization behavior

AI Usage

Assisted-by: ChatGPT

@rolandocortez
rolandocortez requested a review from klemen1999 July 14, 2026 09:26
@rolandocortez rolandocortez added the testable PR is ready to be tested label Jul 14, 2026
@rolandocortez

Copy link
Copy Markdown
Contributor Author

Re-checked generic-example YOLOv8 instance segmentation with depthai==3.8.0 + depthai-nodes main.

Instance masks are visible natively without --overlay_mode, and the no-overlay pipeline does not include ApplyColormap / ImgFrameOverlay. I updated generic-example README/help so the instance segmentation example no longer uses --overlay_mode.

I’ll continue checking the remaining instance-segmentation/custom annotation candidates to see if anything else can be safely simplified.

@rolandocortez

Copy link
Copy Markdown
Contributor Author

Checked apps/object-volume-measurement-3d as another instance segmentation/custom annotation candidate.

I don't think its AnnotationNode should be removed in this PR. It is not only a visual overlay path: it handles click selection, uses getCvSegmentationMask(), generates segmented RGB/depth for RGBD pointcloud creation, communicates selection modes to MeasurementNode, and draws measurement overlays.

So for now I’ll keep it unchanged. If we want to adjust its background/mask handling, I’d treat that as a separate runtime-validated follow-up because it is part of the measurement pipeline.

@PetrNovota PetrNovota left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Comment thread apps/dino-tracking/backend/src/object_selection/mask_selection_node.py Outdated
@klemen1999

Copy link
Copy Markdown
Collaborator

Let's run an LLM agent from the repo origin and instruct it to check if there are any example specific AGENTS.md file updates needed or any INDEX.md updates needed. And also say that it should ignore the temporary depthai-nodes dependency not using released version, this will go away before this branch is merged so it shouldn't be included in any of the AGENTS.md file changes (or anything else that LLMs might try to do)

@klemen1999 klemen1999 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM

Comment thread neural-networks/generic-example/README.md
@rolandocortez

Copy link
Copy Markdown
Contributor Author

Let's run an LLM agent from the repo origin and instruct it to check if there are any example specific AGENTS.md file updates needed or any INDEX.md updates needed. And also say that it should ignore the temporary depthai-nodes dependency not using released version, this will go away before this branch is merged so it shouldn't be included in any of the AGENTS.md file changes (or anything else that LLMs might try to do)

Checked this as well. Looks like no extra AGENTS.md / INDEX.md updates are needed here. The docs/index checks pass and regenerating the index didn’t produce any diff. Also kept the temporary depthai-nodes main dependency out of the docs as discussed.

@klemen1999

Copy link
Copy Markdown
Collaborator

There are currently issues with HIL setup for RVC2 tests. Will ignore those tests for now and merge anyway

@klemen1999
klemen1999 merged commit 88c9a0f into main Jul 17, 2026
7 of 9 checks passed
@klemen1999
klemen1999 deleted the feat/86cajeq9j-native-segmask-segparser-examples branch July 17, 2026 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testable PR is ready to be tested

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants