ENH: Pin vendored Eigen3 fork to tagged release 5.0.1 (closes #6239, supersedes #6247)#6249
Conversation
This comment was marked as resolved.
This comment was marked as resolved.
SimpleITK has three distinct ways it can consume ITK. The new root build with content fetch, the default superbuild from an installed directory, and superbuild with "ITK_USE_BUILD_DIR". It is important to test using ITK from a build and installation directory. Additionally, there were some oddities/additional requirements with building a wrapped ITK remote module against an ITK build directory. This took me a lot of time to get the updated cmake instructor working in these difference configuration and figuring out some issue with ThirdParty libraries export specification. unfortunately, the ThirdParty library would done several different ways and there was not way to document on the intricate details, and there may be leftover incorrect comments in places. |
Greptile P1 finding on PR InsightSoftwareConsortium/ITK#6249: the 8-line comment block at CMakeLists.txt:971-981 and the 3-line block in Eigen3Config.cmake.in:7-9 violate ITK's prose-budget cap of 1 short line per in-source comment. Both reduce to a single load-bearing line carrying the issue reference; the rest of the rationale lives in the commit message and PR body, per AGENTS.md → prose-budget.md. Issue: InsightSoftwareConsortium/ITK#6239 Reviewer: greptile P1 on InsightSoftwareConsortium/ITK#6249
f4f5880 to
02a74a0
Compare
|
Thanks @blowekamp. Acknowledging the three SimpleITK consumption modes and the wrapped-remote-module-against-build-tree gotcha. Kicking off the matrix locally now; will report results back here. Test matrix (3 SimpleITK modes × 2 ITK Eigen modes + 1 wrapped-module test)
Acknowledging your note that the ThirdParty libraries export specification has multiple historical paths and there may be leftover incorrect comments. The current PR contains a follow-up commit (02a74a0) that already trimmed two over-budget comment blocks per greptile's prose-budget finding; if the matrix surfaces additional incorrect-comment hotspots in the install/build/use paths, I'll fix those as fixup commits and call them out explicitly. Cleanup of the dual install ruleThe current Will post matrix results once builds complete. |
Build matrix results — three SimpleITK consumption modes against this PR's tipAll three modes Brad called out pass cleanly with the new fork tag content. eigen_internal is correctly registered in
Mechanical detail for M3 (the build-dir consumer Brad noted as the tricky one)
In the build-tree Pending: M4-M6
Reproducer state preserved
|
M6 result — ITKTotalVariation against ITK build directory ✅Reproduced the canonical "wrapped remote module against ITK build directory" case using ITKTotalVariation PR #57 (@blowekamp's "Update for ITK Interface Modules and use ITK targets and installation"). This is also the canonical proxTV cmake -G Ninja -S ~/src/ITKTotalVariation -B /tmp/m6-itktotalvariation-bld \
-DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON \
-DITK_DIR=~/src/ITK-eigen-fork-verify-bld # ← ITK BUILD tree, not install
cmake --build /tmp/m6-itktotalvariation-bld -j8
Net build matrix for PR #6249
All four consumer modes pass with the new fork tag. eigen_internal lands in Reproducer state preserved at |
|
Why are we using an arbitrary Eigen hash and not a release version? |
|
This got co-mingled with other work testing with latest eigen and forgot to recheck the pinning. That is coming shortly. |
Code extracted from:
https://github.com/InsightSoftwareConsortium/eigen
at commit 6c0093b842e91dd7e826a48127f5896ef2015c4b (for/itk-eigen-5.0.1).
# By Eigen Upstream * upstream-Eigen3: Eigen3 2026-05-11 (for/itk-eigen-5.0.1)
Repoint Modules/ThirdParty/Eigen3/UpdateFromUpstream.sh from the
moving master-snapshot tag
for/itk-20260509-599d71ab-p2-prose (libeigen/master HEAD)
to the SemVer-stable release tag
for/itk-eigen-5.0.1 (libeigen 5.0.1 + 6 ITK overlays)
The fork-side tag for/itk-eigen-5.0.1 is built by branching off
libeigen/eigen 5.0.1 (bc3b39870) and cherry-picking the same 6 ITK
overlay patches that the master-snapshot fork tag carried:
- .gitattributes: relax content checks for lapacke.h
- CMakeLists.txt: stub for vendored INTERFACE-only use
- README.kitware.md: fork purpose + local-patches documentation
- CMakeLists.txt: register eigen_internal in ITK targets +
parametrize Eigen3Config.cmake.in
- CMakeLists.txt: post-import greptile fixups
- CMakeLists.txt: trim eigen_internal install comments to fit
ITK prose budget
Pinning to a release tag (rather than a master snapshot) gives ITK
stable, reproducible upstream versioning and avoids inheriting any
post-5.0.1 master-branch development changes.
02a74a0 to
5a59ac8
Compare
|
I ran a diff between this branch and 7aa8ae2 on the itkeigen/CMakeLists.txt filter where things were known to work. I see the following: This is not correct. The eigen_internal should support the native eigen includes. This is wrong here too: ITK/Modules/ThirdParty/Eigen3/CMakeLists.txt Lines 65 to 69 in 59176b1 The ITK::ITKEigen3Module should add support for the itkeigen path. This way direct usage of eigen_internal is equivalent to a normal Eigen Library. Additionally, the generation of "ITKInternalEigen3Targets.cmake" and ITKInternalEigen3Config.cmake was intentionally removed. There usage was very error prone and problematic in certain cases. Consumers can just use the ITK Eigen targets that are not properly exported in their application. TVProx was updated to do this greatly simplifying the CMake code all around. |
|
Force-pushed to repin the vendored Eigen3 fork from the master-snapshot tag to the SemVer-stable release tag Title + body updated to reflect the new pinning. CI re-running on the new HEAD. |
|
@blowekamp I'm at the end of my skill base and knowledge base for all the scenarios needed for Eigen. I naively thought this was going to be some minor refinements on an UpdateFromUpstream.sh standard vendored tool update. I hoped that the many modernizations of CMake for Eigen in the 5.0 branch were going to limit the amount of custom CMake variants that are necessary. I now realize that I should not have attempted to update Eigen. This is not a typical update vendored tool pattern, but one that has many, many patches and potholes and subtle build implementations. Testing the build and testing against BRAINSTools, Slicer, Slicer Extensions, and ANTs is insufficient evidence that the build is correct. I propose reverting back to Pre Eigen 5.0 (Pre PR #6176) (i.e., the eigen version from a month ago) and move on to the more pressing issues of remote module ingestion. An update to Eigen 5.0 can happen when more knowledgeable developers have the desire to do the work. I am sorry that I have taken up so much of your time on this issue. I don't know what to do next. |
|
We are this far we might as well keep going. Is there documentation for the process done to update the ITK eigen3 fork? I have not been looking at that closely. At this point I have reviewed the Eigen CMake code close enough I should be able to restore it to what I had operational before. A problem with ITK's Eigen is that too many usages were made requirements. For example being able to include two different Eigen libraries in one compilation unit does not seem reasonable. I neglected to remove this documentation here: https://github.com/InsightSoftwareConsortium/ITK/blob/main/Modules/ThirdParty/Eigen3/src/itk_eigen.h.in#L32-L57 I propose we finish this update to 5.0.1, which works for most cases. Then I'll clean up the include paths, and such in ITK. I'll make a PR in ITK and testing can be done to verify. When checked, we can some hope put the change into ITK Eigen's fork so it'll stick this time. |
|
@blowekamp Thank you. I do not want to burden you more, but you might be one of the few people who knows what needs to be done to get this right. Here is what I propose to keep moving forward (ignore any of this that you want, your knowlegebase on this issue far exceeds mine):
# I think this is what should have happened as a starting point
# Inputs
WORK_TREE=~/src
REPO_DIR=$WORK_TREE/eigen
ITK_CURRENT_TAG=for/itk-20260305-4c99fca # prior fork tag
ITK_EIGEN_TARGET_TAG=5.0.1 # new libeigen release
NEW_BRANCH=branch_itk_$ITK_EIGEN_TARGET_TAG
ITK_NEW_TAG=for/itk-eigen-$ITK_EIGEN_TARGET_TAG
# 1. Clone (or reuse) the fork and ensure the tree is clean:
git clone git@github.com:InsightSoftwareConsortium/eigen.git $REPO_DIR
cd $REPO_DIR
# 2. Wire remotes:
git remote add isceigen git@github.com:InsightSoftwareConsortium/eigen.git # or set-url
git remote add libeigen git@gitlab.com:libeigen/eigen.git
# 3. Fetch (no --tags on libeigen — historical tag collisions):
git fetch isceigen
git fetch libeigen master
# 4. Mirror libeigen/master → isceigen/master:
git push isceigen refs/remotes/libeigen/master:refs/heads/master
# 5. Mirror the chosen release tag (refuse to move if already on isceigen at a different SHA):
git fetch libeigen tag $ITK_EIGEN_TARGET_TAG
git push isceigen tag $ITK_EIGEN_TARGET_TAG
# 6. Fetch the prior ITK fork tag:
git fetch isceigen refs/tags/$ITK_CURRENT_TAG:refs/tags/$ITK_CURRENT_TAG
# 7. Inspect overlay commits to replay:
git log --oneline libeigen/master..refs/tags/$ITK_CURRENT_TAG
# 8. Branch from the old fork tag:
git checkout -b $NEW_BRANCH refs/tags/$ITK_CURRENT_TAG
# 9. Rebase overlays onto the new release tag (resolve conflicts as needed):
git rebase --onto refs/tags/$ITK_EIGEN_TARGET_TAG libeigen/master $NEW_BRANCH
# 10. Verify:
git log --oneline refs/tags/$ITK_EIGEN_TARGET_TAG..HEAD
git diff --stat refs/tags/$ITK_EIGEN_TARGET_TAG..HEAD
# 11. Tag the rebased tip and publish (ITK's UpdateFromUpstream.sh consumes this tag):
git tag -a $ITK_NEW_TAG -m "ITK overlay replayed onto libeigen $ITK_EIGEN_TARGET_TAG (was $ITK_CURRENT_TAG)"
git push isceigen refs/tags/$ITK_NEW_TAG
git push isceigen $NEW_BRANCH:$NEW_BRANCH
# 12. In ITK, set in Modules/ThirdParty/Eigen3/UpdateFromUpstream.sh:
readonly tag="for/itk-eigen-5.0.1"
============= |
|
Thanks you for the details on how the update should happen. I am looking at non-merge commit in The workflow for working on a forked thirparty I don't think are specified clearly and commits can more easily be managed by reapplying in the fork than as part of conflict resolution during ITK sub-tree merge. Perhaps we could just switch to using FetchContent from the forked repo instead? This would nearly force the changes into the fork. Also note that DCMTK has a fork, but no subtree merge. Instead it uses and External Project to install it for usage. I am currently inspecting histories to better understand.... |
|
@blowekamp Take the proposed plan as a suggested starting point, not as some agreed-upon plan. I am certain the way I originally approached the problem was wrong. When retracking my steps, I tried to identify a better approach. Please use the plan I laid out as the alpha draft for a procedure to normalize how Eigen should be updated in the future. (PS. It looks like Eigen is under heavy development right now to remove old coding paradigms, use new C++ features to overcome and simplify the codebase, and hopefully provide performance improvements. I'd guess that we will desire to update Eigen again in a few months to take advantage of those changes. |
|
See PR #6259 for continued work. |
|
Superceeded. |
Pin ITK's vendored Eigen3 fork to the tagged Eigen 5.0.1 release (
bc3b39870) plus 6 ITK overlay patches, instead of a moving libeigen/master snapshot. Closes #6239 durably; supersedes #6247.What changed vs the prior round of this PR
Earlier revisions of this PR pinned to
for/itk-20260509-599d71ab-p2-prose— a fork tag based on libeigen/master HEAD599d71ab(2026-05-09). That's a moving point on master, not a release tag. This revision repointsModules/ThirdParty/Eigen3/UpdateFromUpstream.shto:which is built as:
libeigen/eigen 5.0.1(bc3b39870) + 6 ITK overlay commits cherry-picked on top (same overlay set as the previous master-snapshot tag).Why a tagged release instead of a master snapshot
SemVer-stable release tags give ITK reproducible upstream versioning. Pinning to a master HEAD inherits any post-5.0.1 development churn (CI lint changes, GPU work, REUSE updates, etc.) that isn't release-blessed. The 5.0.1 baseline is the most recent Eigen point-release and is what downstream consumers will normally pair with ITK builds.
Overlay stack on top of 5.0.1
.gitattributeschangeCMakeLists.txtstubREADME.kitware.mdCMakeLists.txt(eigen_internal + Config)CMakeLists.txt(greptile fixups)CMakeLists.txt(prose budget)Tag
for/itk-eigen-5.0.1is published onInsightSoftwareConsortium/eigen.