Skip to content

DOC: Recreate lost Core/Common example visuals with new CIDs#455

Closed
hjmjohnson wants to merge 1 commit into
InsightSoftwareConsortium:mainfrom
hjmjohnson:fix/remove-missing-visual-cids
Closed

DOC: Recreate lost Core/Common example visuals with new CIDs#455
hjmjohnson wants to merge 1 commit into
InsightSoftwareConsortium:mainfrom
hjmjohnson:fix/remove-missing-visual-cids

Conversation

@hjmjohnson

@hjmjohnson hjmjohnson commented Jun 7, 2026

Copy link
Copy Markdown
Member

Recreate the 20 Core/Common example visuals whose binaries were never published by #440 and are unrecoverable from any public source. The .cid files now point at recreated diagrams (published via InsightSoftwareConsortium/ITKTestingData#63); also fixes the Operator.pngOperation.png reference mismatch. Blocked on ITKTestingData#63 merging first.

Why the original PNGs are unrecoverable — the hunt
Source Method Result
ITKTestingData gh-pages CID/<cid> direct fetch of all 20 CIDs absent (sibling MakePartOfImageTransparent.png from the same #440 IS present)
data.kitware.com (Girder) global exact-name search, all 16 unique basenames 3 exact-name hits, all CID-mismatched; one is the already-published file under its old example name
data.kitware.com ITK/ITKSphinxExamples/Nightly full collection walk 9 old example folders; none contain these diagrams
Old .sha512/.md5 hashes git log --all --diff-filter=A, any extension none — all 20 files were born as .cid in #440; the figures never existed before
GitHub user-attachments in #440 + #359 harvested all asset URLs one unrelated mockup only
jph6366 fork recursive git-tree scan, both branches zero PNG blobs
IPFS (7 gateways) direct CID fetch never pinned durably

The binaries only ever existed on the #440 contributor's machine. If the originals resurface they can replace the recreations.

What changed
  • 20 .cid files updated to the CIDs of the recreated diagrams (matplotlib renderings matching each figure's caption/context; review gallery in the comments below and in ITKTestingData#63).
  • ApplyCustomOperationToEachPixelInImage/Documentation.rst: figure:: Operator.png corrected to Operation.png — the figure block referenced a name with no corresponding .cid file while Operation.png.cid sat orphaned.
  • No figure blocks removed; all 13 Documentation.rst figure references stay live.
Local validation

CID computation (CIDv1/raw/sha2-256) validated by roundtrip against the published MakePartOfImageTransparent.png blob. Each new blob in ITKTestingData#63 is byte-identical to the file hashed into its .cid reference here.

@dzenanz

dzenanz commented Jun 8, 2026

Copy link
Copy Markdown
Member

data.kitware.com/api/v1/file/hashsum/CID

Girder does not support IPFS CIDs. I think it only supports MD5 and SHA512. Searching by filename usually gives results.

@dzenanz

dzenanz commented Jun 8, 2026

Copy link
Copy Markdown
Member

As examples used to build without errors, the missing files must be somewhere. data.kitware.com used to be the default for many years, so that should be the first place to look. If search by filename does not yield the (desired) result, trying to download by the old sha512/md5 hash is another avenue.

@hjmjohnson

Copy link
Copy Markdown
Member Author

@dzenanz Both avenues checked — these 20 figures never existed before #440, so there are no old sha512/md5 hashes to try, and a full data.kitware.com filename search (all 16 unique basenames) found no matching content.

Evidence

No prior hashes exist. git log --all --diff-filter=A over every basename (any extension: .png, .png.sha512, .png.md5, .png.cid) shows all 20 files were born as .cid in ed7c181b (#440, 2025-03-11) and never existed in any branch before. The examples built without errors before #440 because these figure references didn't exist yet — the 20 fetch failures date exactly to #440's merge.

Girder filename search (per your suggestion). resource/search on data.kitware.com for all 16 unique basenames:

Query Exact-name hits Verdict
Transparency.png ITK/ITKSphinxExamples/Nightly/src/Core/Common/Transparency (544 B, 2019) CID bafkreicud23… = byte-identical to the already-published MakePartOfImageTransparent.png — the old name for the one figure that is not lost
Transparency.png mseng10/Public (240 KB, 2019) Actually a TIFF (II* magic) misnamed .png; CID mismatch
MiniPipeline.png mseng10/Public (330 B, 2019) CID mismatch
other 13 basenames no exact-name items

CID comparison method validated by roundtrip: downloading the published sibling MakePartOfImageTransparent.png from ITKTestingData CID/ and recomputing yields its exact .cid value.

The binaries only ever existed on the #440 contributor's machine; the CIDs were computed locally and the uploads never happened. Happy to restore any figure together with its binary if it ever resurfaces.

@hjmjohnson

Copy link
Copy Markdown
Member Author

Status: the 20 visuals below are confirmed unrecoverable from any public source — the binaries only ever existed on the #440 contributor's machine (CIDs were computed locally; the uploads never happened). Plan: recreate the diagrams, publish the new binaries via an ITKTestingData PR, and convert this PR from reference-removal to .cid-update.

The 20 truly missing files
ApplyAFilterOnlyToASpecifiedRegionOfAnImage/Relationship.png
ApplyCustomOperationToEachPixelInImage/Operation.png
ApplyCustomOperationToEachPixelInImage/Relationship.png
BoundingBoxOfAPointSet/BoundingBox.png
BresenhamLine/Operation.png
BresenhamLine/Segmentation.png
ConvertArrayToImage/Relationship.png
ConvertArrayToImage/Sphere.png
ConvertArrayToImage/Sphere3D.png
CreateDerivativeKernel/DerivativeKernel.png
CreateDerivativeKernel/DerivativeOperator.png
CreateGaussianKernel/GaussianBlur.png
CreateGaussianKernel/GaussianKernel.png
CreateVectorImage/Operation.png
CropImageBySpecifyingRegion/3DVolume.png
CropImageBySpecifyingRegion/ExtractedSlice.png
ImageRegionIntersection/ImageRegionIntersection.png
MakePartOfImageTransparent/Transparency.png
MiniPipeline/MiniPipeline.png
ReadAPointSet/tetrahedron.png

(all under src/Core/Common/)

The hunt — every source checked
Source Method Result
ITKTestingData gh-pages CID/<cid> direct fetch of all 20 CIDs absent (sibling MakePartOfImageTransparent.png from the same #440 IS present, confirming path format)
data.kitware.com (Girder) global exact-name resource/search for all 16 unique basenames 3 exact-name hits, all CID-mismatched: Transparency.png in ITK/ITKSphinxExamples/Nightly/.../Transparency is byte-identical to the already-published MakePartOfImageTransparent.png (old example name); 2 unrelated 2019 mseng10/Public files (one a TIFF misnamed .png)
data.kitware.com ITK/ITKSphinxExamples/Nightly collection full folder walk src/Core/Common/ holds 9 old example folders; none contain these diagrams
Old .sha512/.md5 ExternalData hashes git log --all --diff-filter=A per basename, any extension none exist — all 20 files were born as .cid in #440 (ed7c181b); the figures never existed before, so were never uploaded under any hash scheme
GitHub user-attachments harvested every asset URL in #440 + #359 bodies/comments/review comments one image only — an unrelated sequence-diagram mockup
jph6366 fork recursive git-tree scan of both branches zero PNG blobs under Core/Common
IPFS ipfs.io, dweb.link, w3s.link, nftstorage.link, pinata, filebase, cf-ipfs never pinned durably
examples.itk.org (published site) page inspection only generated output PNGs appear; these visuals never built into the site

CID verification method (CIDv1/raw/sha2-256) validated by roundtrip on the published sibling file.

@hjmjohnson

Copy link
Copy Markdown
Member Author

The 20 visuals have been recreated as clean conceptual diagrams; binaries are published in InsightSoftwareConsortium/ITKTestingData#63. This PR will be reworked from reference-removal to .cid-update once that lands.

Review gallery — all 20 recreated figures
ApplyAFilterOnlyToASpecifiedRegionOfAnImage/Relationship.png
ApplyCustomOperationToEachPixelInImage/Operation.png
ApplyCustomOperationToEachPixelInImage/Relationship.png
BoundingBoxOfAPointSet/BoundingBox.png
BresenhamLine/Operation.png
BresenhamLine/Segmentation.png
ConvertArrayToImage/Relationship.png
ConvertArrayToImage/Sphere.png
ConvertArrayToImage/Sphere3D.png
CreateDerivativeKernel/DerivativeKernel.png
CreateDerivativeKernel/DerivativeOperator.png
CreateGaussianKernel/GaussianBlur.png
CreateGaussianKernel/GaussianKernel.png
CreateVectorImage/Operation.png
CropImageBySpecifyingRegion/3DVolume.png
CropImageBySpecifyingRegion/ExtractedSlice.png
ImageRegionIntersection/ImageRegionIntersection.png
MakePartOfImageTransparent/Transparency.png
MiniPipeline/MiniPipeline.png
ReadAPointSet/tetrahedron.png

The 20 PNGs referenced since InsightSoftwareConsortium#440 were never published to
ITKTestingData and are unrecoverable from any public source; every
build of Module_SphinxExamples fails 20 ExternalData fetches. Point
the .cid files at recreated diagrams (ITKTestingData#63) and fix the
Operator.png/Operation.png reference mismatch.
@hjmjohnson hjmjohnson force-pushed the fix/remove-missing-visual-cids branch from 5d1bd8e to 36f1ee8 Compare June 10, 2026 23:33
@github-actions github-actions Bot added area:Core Issues affecting the Core module area:Documentation Issues affecting the Documentation module type:Data Changes to example data (usually displayed images) type:Documentation Documentation improvement or change labels Jun 10, 2026
@hjmjohnson hjmjohnson changed the title DOC: Remove dead .cid references to lost visual PNGs DOC: Recreate lost Core/Common example visuals with new CIDs Jun 10, 2026
@hjmjohnson hjmjohnson marked this pull request as ready for review June 10, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Core Issues affecting the Core module area:Documentation Issues affecting the Documentation module type:Data Changes to example data (usually displayed images) type:Documentation Documentation improvement or change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants