Skip to content

Commit 7b92beb

Browse files
Joeychosjh26
Joeycho
authored and
sjh26
committed
update - joey
1 parent bfe2979 commit 7b92beb

2 files changed

Lines changed: 56 additions & 104 deletions

File tree

‎PW45_2026_Boston/Projects/RobustSegmentationExperienceInOHIFViewer/README.md‎

Lines changed: 56 additions & 104 deletions
Original file line numberDiff line numberDiff line change
@@ -20,33 +20,27 @@ OHIF Viewer is a widely used web-based medical image viewer built on Cornerstone
2020

2121
During Project Week 45, I would like to focus on improving the stability and user experience of segmentation in OHIF/Cornerstone. I am looking forward to collaborating with others interested in segmentation, digital pathology, and web-based imaging viewers.
2222

23-
This project targets three complementary areas:
23+
This project targets two complementary areas:
2424

25-
- **Workflow reliability** — Reduce errors and fragile behavior in common segmentation tasks (loading, editing, saving, and display).
26-
- **Representation interoperability** — Enable better interchangeability between contour-based and labelmap-based segmentations inside OHIF/Cornerstone3D.
27-
- **Microscopy segmentation** — Explore segmentation support for whole-slide and microscopy images in OHIF, as in the [SLIM](https://github.com/MGHComputationalPathology/slim) viewer.
25+
- **DICOM-SEG landscape** — Understand the development and history of DICOM Segmentation, current standard direction (including label-map encodings), and recommended usage for save/load and interchange with research formats.
26+
- **Overlapping segments in OHIF** — Ensure overlapping segmentations work reliably across **Stack Viewport + MPR**, **Volume Viewport**, and downstream tools — **segmentation statistics** and **bidirectional measurements**.
2827

29-
Several related gaps remain in the broader stack: the WSI microscopy viewer (`wsi-microscopy-viewer`) does not yet support segmentation display, overlapping segmentations are not always handled reliably, and contour ↔ labelmap conversion is not yet exposed end-to-end in OHIF. This project aims to advance solutions within Cornerstone3D and OHIF toward a more dependable segmentation experience across radiology and pathology use cases.
28+
Several related gaps remain in the broader stack: overlapping segmentations are not always handled reliably across viewport types, DICOM persistence paths are still evolving ([PR #5806](https://github.com/OHIF/Viewers/pull/5806), [#4875](https://github.com/OHIF/Viewers/issues/4875)), and contour ↔ labelmap conversion is not yet exposed end-to-end in OHIF. This project aims to advance solutions within Cornerstone3D and OHIF toward a more dependable segmentation experience.
3029

3130
For DICOM persistence, standards context, and recommended interchange with research formats (NIfTI), see [Background and References](#background-and-references).
3231

3332
## Objective
3433

35-
1. **Improve segmentation reliability** — Identify and address high-impact failure modes in OHIF/Cornerstone segmentation workflows (load, render, edit, persist) to reduce user-facing errors.
36-
2. **Contour ↔ labelmap interoperability** — Allow users and tools to convert between contour-based and labelmap-based segmentation representations in OHIF/Cornerstone3D, building on work such as [Cornerstone3D PR #2170](https://github.com/cornerstonejs/cornerstone3D/pull/2170).
37-
3. **Microscopy segmentation support** — Enable segmentation rendering (labelmap and/or contour overlay) in the OHIF WSI microscopy viewer, progressing toward capabilities comparable to SLIM.
38-
4. **DICOM persistence** — Clarify recommended save/load paths for segmentations (DICOM SEG and emerging label-map encodings) and validate practical DICOM ↔ NIfTI interchange (see [DICOM-SEG and format interchange](#dicom-seg-and-format-interchange)).
34+
1. **Understand DICOM-SEG** — Review the development and history of DICOM Segmentation (classic binary/fractional SEG, emerging label-map encodings, tooling ecosystem) and document **recommended usage** for saving, loading, and sharing segmentations in clinical and research workflows (see [DICOM-SEG and format interchange](#dicom-seg-and-format-interchange)).
35+
36+
2. **Overlapping segments across OHIF viewports and measurements** — In the context of OHIF, ensure **overlapping segments** work correctly for **Stack Viewport + MPR** and **Volume Viewport**, and that this behavior is connected to working **segmentation statistics** and **bidirectional measurements** on those segmentations (see [Illustrations](#illustrations)).
3937

4038
## Approach and Plan
4139

42-
1. **Audit current state** — Review segmentation pain points in OHIF/Viewers and Cornerstone3D; compare microscopy segmentation support in `wsi-microscopy-viewer` vs. SLIM and document pipeline gaps.
43-
2. **Reliability fixes** — Triage and reproduce issues from community reports (e.g. [#5453](https://github.com/OHIF/Viewers/issues/5453), [#5849](https://github.com/OHIF/Viewers/issues/5849)); prototype fixes or workarounds for the most impactful cases.
44-
3. **Contour ↔ labelmap conversion** — Review and test upstream conversion work; expose it in the OHIF UI so users need not convert representations manually.
45-
4. **Microscopy segmentation MVP** — Integrate Cornerstone3D segmentation rendering into the OHIF WSI microscopy viewport; target labelmap overlay as a first milestone.
46-
5. **Overlapping segments** — Investigate segment blending/ordering in Cornerstone3D so multiple overlapping segments render and interact correctly.
47-
6. **DICOM persistence audit** — Map save/load paths (classic SEG vs label-map objects); test against [OHIF/Viewers PR #5806](https://github.com/OHIF/Viewers/pull/5806); document DICOM ↔ NIfTI assumptions ([background notes](#dicom-seg-and-format-interchange)).
48-
7. **Integration testing** — Validate changes with real DICOM SEG and RT-STRUCT datasets (radiology and pathology); record short demo screencasts.
49-
8. **Documentation and follow-up** — Open or update GitHub issues/PRs in `cornerstonejs/cornerstone3D` and `OHIF/Viewers` for work continuing beyond the week.
40+
1. **DICOM-SEG review** — Summarize standard history (binary SEG → label-map Sup 243), tooling roles (dcmjs, highdicom, dcmqi, etc.), and recommended save/load paths; validate against [PR #5806](https://github.com/OHIF/Viewers/pull/5806) ([background notes](#dicom-seg-and-format-interchange)).
41+
2. **Reproduce overlapping-segment issues** — Triage reports (e.g. [#5453](https://github.com/OHIF/Viewers/issues/5453), [Cornerstone3D PR #2170](https://github.com/cornerstonejs/cornerstone3D/pull/2170)) across Stack + MPR and Volume viewports.
42+
3. **Fix overlapping segment rendering** — Investigate segment blending/ordering in Cornerstone3D so multiple overlapping segments render and interact correctly in all target viewport types.
43+
4. **Connect stats and bidirectional measurements** — Verify segmentation statistics (volume, HU stats, voxel count) and bidirectional tool output remain correct when segments overlap and when switching between Stack/MPR and Volume layouts.
5044

5145
## Progress and Next Steps
5246

@@ -58,7 +52,9 @@ For DICOM persistence, standards context, and recommended interchange with resea
5852

5953
# Illustrations
6054

61-
<!-- Add screenshots, diagrams, or screen recordings here once available. -->
55+
OHIF MPR layout (Stack + MPR viewports) with overlapping segmentations (red and green), segmentation statistics, and bidirectional measurements:
56+
57+
![OHIF overlapping segmentations with statistics and bidirectional measurements in MPR view](ohif-segmentation-stats-mpr.png)
6258

6359
# Background and References
6460

@@ -84,7 +80,7 @@ For DICOM persistence, standards context, and recommended interchange with resea
8480

8581
## DICOM-SEG and format interchange
8682

87-
Reference material for persistence, standards, tooling, and conversion with NIfTI. Intended as background for the [DICOM persistence](#dicom-persistence) objective and [approach step 6](#approach-and-plan).
83+
Reference material for persistence, standards, tooling, and conversion with NIfTI. Intended as background for [objective 1](#objective) and [approach step 1](#approach-and-plan).
8884

8985
### Recommended practice
9086

@@ -145,95 +141,51 @@ NIfTI does not carry DICOM identity or segment ontology. Required inputs:
145141

146142
### Package Overview
147143

148-
```mermaid
149-
flowchart TB
150-
subgraph files [Files on disk]
151-
DCM[".dcm DICOM Part 10"]
152-
NII[".nii.gz NIfTI"]
153-
JSON["gt_rebuilt_metadata.json"]
154-
end
155-
156-
subgraph python [Your Python pipeline - test_medimage]
157-
PY_PYD["pydicom"]
158-
PY_PYS["pydicom-seg"]
159-
PY_SITK["SimpleITK"]
160-
PY_NP["NumPy"]
161-
PY_SUB["subprocess / stdlib"]
162-
end
163-
164-
subgraph cpp_cli [C++ CLI tools]
165-
DCMQI["dcmqi binaries<br/>segimage2itkimage<br/>itkimage2segimage"]
166-
DCMTK["DCMTK<br/>dcmodify"]
167-
GDCM_PY["python-gdcm<br/>optional fix scripts"]
168-
end
169-
170-
subgraph sitk_backends [Behind SimpleITK]
171-
ITK["ITK"]
172-
GDCM_IO["GDCM via GDCMImageIO<br/>GetGDCMSeriesFileNames"]
173-
end
174-
175-
subgraph browser [OHIF Viewer - browser only]
176-
OHIF["OHIF Viewers"]
177-
DWC["dicomweb-client"]
178-
DCMJS["dcmjs"]
179-
CS_ADP["@cornerstonejs/adapters"]
180-
CS3D["@cornerstonejs/core + tools"]
181-
CIL["@cornerstonejs/dicom-image-loader"]
182-
DP["dicom-parser"]
183-
end
184-
185-
subgraph not_in_repo [Common but NOT in your repo]
186-
HD["highdicom - not used"]
187-
NB["nibabel - not used"]
188-
end
189-
190-
subgraph pacs [Archive / transport]
191-
PACS["PACS / Orthanc / DICOMweb<br/>QIDO - WADO - STOW"]
192-
end
193-
194-
DCM --> PY_PYD
195-
DCM --> PY_SITK
196-
DCM --> DCMQI
197-
DCM --> DCMTK
198-
DCM --> GDCM_PY
199-
NII --> PY_SITK
200-
PY_SITK --> ITK
201-
PY_SITK --> GDCM_IO
202-
DCMQI -.-> ITK
203-
DCMQI -.-> DCMTK
204-
205-
PY_PYD --> NII
206-
PY_PYS --> NII
207-
PY_SITK --> NII
208-
PY_NP --> NII
209-
NII --> DCMQI
210-
DCMQI --> DCM
211-
PY_PYD --> DCM
212-
DCMTK --> DCM
213-
GDCM_PY --> DCM
214-
PY_PYD --> JSON
215-
216-
PACS <-->|DICOMweb| DWC
217-
DWC --> OHIF
218-
DCMJS --> OHIF
219-
OHIF --> CS3D
220-
CS_ADP --> DCMJS
221-
CS_ADP --> CS3D
222-
CIL --> CS3D
223-
DP --> CIL
224-
DWC -->|retrieve bytes| CS_ADP
225-
DCMJS -->|STOW Part 10| DWC
226-
227-
HD -.->|alternative SEG writer| DCM
144+
Project Week pages do not render Mermaid; the diagrams below use plain text so they display on [projectweek.na-mic.org](https://projectweek.na-mic.org).
145+
146+
**End-to-end data flow**
147+
148+
```
149+
Files on disk Archive / transport
150+
┌──────────────┐ ┌─────────────────────────────┐
151+
│ .dcm (SEG) │◄───────────────►│ PACS / Orthanc / DICOMweb │
152+
│ .nii.gz │ STOW/QIDO │ QIDO · WADO-RS · STOW-RS │
153+
└──────┬───────┘ └──────────────┬──────────────┘
154+
│ │
155+
│ read/write │ retrieve / store
156+
▼ ▼
157+
┌──────────────────────────────────────────────────────────────────┐
158+
│ Conversion & I/O layers │
159+
├─────────────────────┬──────────────────────┬─────────────────────┤
160+
│ Python (offline) │ C++ CLI │ Browser (OHIF) │
161+
│ · pydicom │ · dcmqi │ · dicomweb-client │
162+
│ · pydicom-seg │ segimage2itkimage │ · dcmjs │
163+
│ · SimpleITK/NumPy │ itkimage2segimage │ · @cornerstonejs/ │
164+
│ · highdicom │ · DCMTK (dcmodify) │ adapters │
165+
│ │ · python-gdcm │ · cornerstone-dicom-│
166+
│ │ │ seg │
167+
└─────────┬───────────┴──────────┬───────────┴──────────┬──────────┘
168+
│ │ │
169+
└──────────┬───────────┴──────────────────────┘
170+
▼
171+
┌─────────────────────┐
172+
│ ITK / GDCM (C++) │ ← behind SimpleITK & dcmqi
173+
└─────────────────────┘
174+
│
175+
▼
176+
┌─────────────────────┐
177+
│ OHIF + Cornerstone3D│ labelmap / contour in viewport
178+
│ core + tools │ → stats, bidirectional, export
179+
└─────────────────────┘
228180
```
229181

230-
### DICOM tooling map
182+
**Layer stack (who sits on whom)**
231183

232184
```
233185
┌─────────────────────────────────────────┐
234-
│ Application layer │
235-
│ OHIF │ 3D Slicer │ ML scripts │
236-
└────┬──────────┬──────────────┬──────────┘
186+
│ Application layer │
187+
│ OHIF │ 3D Slicer │ ML / Python │
188+
└────┬──────────┬──────────────┬───────────┘
237189
│ │ │
238190
┌─────────────┼──────────┼──────────────┼─────────────┐
239191
▼ ▼ ▼ ▼ ▼
@@ -242,7 +194,7 @@ flowchart TB
242194
│ Reporting │
243195
└─────────────┬──────────────┘
244196
▼
245-
pydicom (Python) │ dicomParser (JS, legacy)
197+
pydicom (Python) │ dicom-parser (JS, legacy)
246198
▼
247199
DCMTK / GDCM (C++) — dcmseg, DIMSE
248200
```

‎PW45_2026_Boston/Projects/RobustSegmentationExperienceInOHIFViewer/ohif-segmentation-stats-mpr.png‎

1.08 MB
Loading

0 commit comments

Comments
 (0)