[open3d] add Open3D port

[aminya]

This adds Open3D.

It's the successor of #17423 and #12199.
Fixes isl-org/Open3D#5570
Fixes #27032
Fixes #15980
Fixes #16104
Fixes #14751
Fixes #12236
Fixes: #11950
Fixes: #6158
Fixes: #9172

• Changes comply with the maintainer guide.
• The name of the port matches an existing name for this component on https://repology.org/ if possible, and/or is strongly associated with that component on search engines.
• Optional dependencies are resolved in exactly one way. For example, if the component is built with CMake, all find_package calls are REQUIRED, are satisfied by vcpkg.json's declared dependencies, or disabled with CMAKE_DISABLE_FIND_PACKAGE_Xxx.
• The versioning scheme in vcpkg.json matches what upstream says.
• The license declaration in vcpkg.json matches what upstream says.
• The installed as the "copyright" file matches what upstream says.
• The source code of the component installed comes from an authoritative source.
• The generated "usage text" is accurate. See adding-usage for context.
• The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
• Only one version is in the new port's versions file.
• Only one version is added to each modified port's versions file.

[dg0yt]

Can you set this PR to draft until it is ready for review (or until you actually want review comments).

[aminya]

Can you set this PR to draft until it is ready for review (or until you actually want review comments).

I am ready to get review comments. I successfully built this on Ubuntu locally, and the rest is fixing the issues of the CI environment.

[aminya]

Uvatlas is failing to be used on Windows with an error regarding missing DirectX headers. Have you tested UvAtlas on Windows recently inside CI?

-- Configuring done (21.5s)
CMake Error at D:/installed/x64-windows/share/uvatlas/UVAtlas-targets.cmake:60 (set_target_properties):
  The link interface of target "Microsoft::UVAtlas" contains:

    Microsoft::DirectX-Headers

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

Call Stack (most recent call first):
  D:/installed/x64-windows/share/uvatlas/uvatlas-config.cmake:27 (include)
  D:/a/_work/1/s/scripts/buildsystems/vcpkg.cmake:859 (_find_package)
  3rdparty/find_dependencies.cmake:293 (find_package)
  3rdparty/find_dependencies.cmake:1537 (open3d_find_package_3rdparty_library)
  CMakeLists.txt:503 (include)

[dg0yt]

Have you tested UvAtlas on Windows recently inside CI?

If there is no port using it, then CI doesn't test for such issues.
(That's why I add extra test ports more often today.)

[walbourn]

So it seems there are two possible fixes:

1. Modify upstream CMakeLists.txt

if(directxmath_FOUND)
    message(STATUS "Using DirectXMath package")
    target_link_libraries(${PROJECT_NAME} PUBLIC Microsoft::DirectXMath)
endif()

if(directx-headers_FOUND)
    message(STATUS "Using DirectX-Headers package")
    target_link_libraries(${PROJECT_NAME} PUBLIC Microsoft::DirectX-Headers)
    target_compile_definitions(${PROJECT_NAME} PUBLIC USING_DIRECTX_HEADERS)
endif()

Use PRIVATE instead of PUBLIC

2. Modify this port to include directx-headers as another dependency.

[walbourn]

Since both of these libraries are header only, I think making these PRIVATE is probably the simplest fix.

[walbourn]

I applied this patch in #41878

[dg0yt]

Use PRIVATE instead of PUBLIC

Doesn't help for static linkage.

Since both of these libraries are header only, I think making these PRIVATE is probably the simplest fix.

PRIVATE makes no sense for header-only libs. Everything is INTERFACE

Modify this port to include directx-headers as another dependency.

No. It is the responsibilit of uvatlas's CMake config to call find_dependency() as needed to resolve imported targets.

[aminya]

Filament that's used internally for the GUI of Open3D needs libc++. Can we get these libraries included?

[dg0yt]

AFAICT there can be only one C++ standard lib in the whole triplet, and it already comes with the C++ compiler.

[dg0yt]

Can you explain how it depends on a particular libc++?

[aminya]

Open3D relies on Filament which is provided by Google. Filament apparently relies on libc++. However, I suspect if we're not building from the source, it should work without a libc++. I haven't tested on a system that lacks libc++ yet.

https://github.com/isl-org/Open3D/blob/0f06a149c4fb9406fd3e432a5cb0c024f38e2f0e/3rdparty/find_dependencies.cmake#L1363-L1376

[dg0yt]

Oh, filament. Shouldn't it be packaged separately? With its own set of patches? Does it build at all with gcc (x64-linux CI)?

Past attempt, related to past open3d attempts: #17369.

[aminya]

Yeah, in a perfect world, Filament can be potentially ported as well, but I'd like to be realistic and pragmatic. Open3D has been blocked for years. Improvements can be done step by step.

Right now, the official binaries are used. I will test if they need libc++ during runtime

[dg0yt]

Hm, by not porting Filament, the port might just run into the ABI issues that de-vendoring and building from source tries to avoid...

[aminya]

Filament works if the libc++ is linked. There are no abi issues. Also, this is private to Open3D, so no worries regarding that.

I will work on porting filament, but that will take a lot of time, and I don't want that to delay Open3D more

[dg0yt]

Note that changes to managed-image.json need an image update. This is not part of the PR build.

[aminya]

Started the Filament port here
#41916

[dg0yt]

Do you intent do upstream the patches? Some review comments will be about minimizing the patches for vcpkg, but that's not the right thing for upstream.

[aminya]

Do you intent do upstream the patches? Some review comments will be about minimizing the patches for vcpkg, but that's not the right thing for upstream.

Yes, that's my goal. I've tried to keep the patches generic so that they can be upstreamed. I'd like to get this merged here, then focus on upstreaming. Some of the patches for fmt 10/11 are already merged and will be available in the next release (whenever it might be). I have marked them explicitly in the code so we know later.

[dg0yt]

I see the barrier of making filament a proper port.
I just warn that not making it a proper port could be a blocker for open3d.
https://github.com/google/filament/tree/main/third_party

Ant this in addition to open3d itself:
https://github.com/isl-org/Open3D/tree/main/3rdparty

(That's almost the same pain as with other googlish ports (chromium, skia, tensorflow), but this time using CMake.)

[aminya]

Started the Filament port here
#41916

[jordanlui]

Hi @aminya,
I've cloned your open3d fork and attempted to have vcpkg find open3d in a simple cmake test project, but vcpkg is failing to install seacas, which is a dependency of VTK.
Do you use a particular vcpkg baseline or configuration when you install this open3d port?

The seacas build errors complain about missing PYTHON_EXECUTABLE and CACHE variable _Python_EXECUTABLE_DEBUG, even though I have set these in Windows env vars and also tried setting them in CMakeLists.txt.

My system:
Win 11
VS: 2022/Community/VC/Tools/MSVC/14.33.31629

[aminya]

Hey, I haven't tried building on Windows yet. It should work on Linux. There are some linked PRs before I can finalize this.

[MonicaLiu0311]

Is there any new progress?

[aminya]

If the maintainers want everything to be built with vcpkg, it would take a long time to port all the dependencies, but if we're ready to compromise, this PR is ready to go

Right now, blocked by this PR:
#41916

[MonicaLiu0311]

If the maintainers want everything to be built with vcpkg, it would take a long time to port all the dependencies, but if we're ready to compromise, this PR is ready to go

Right now, blocked by this PR: #41916

Got it, I'll wait for your changes.

[patrikhuber]

@aminya Thanks a lot for your work on this.

Any chance to make VTK an optional feature flag? Many people don't need it and it makes the build slimmer & faster. It would actually be great to use a few more feature flags too: Open3D's CMake also has build options for Python (which I'm guessing most people building open3d in vcpkg don't need), and the whole GUI/rendering/filament/webrtc can be disabled too with Open3D's CMake variables. It would be good to have those as feature flags. Maybe we could even merge this PR without rendering/filament support, and then later add it as a feature flag? Otherwise I fear this may never be merged.

Also does this port build open3d with CUDA?

[patrikhuber]

I just read through #41916 as well. I don't think we'll realistically get a Filament port in vcpkg any time soon, probably not in this decade. Any chance we could "solve" this by just merging a open3d port into vcpkg for now that does not support filament/rendering? It can be disabled using Open3D's build using Open3D's CMake cache variables.

If filament is ported to vcpkg in the future, the right path forward would anyway be to add it as a feature flag to the open3d port.

Having a minimal port is the preference of many C++ devs anyway, especially for huge libraries like Open3D. And then the remaining stuff can be added with feature flags whenever it is ready & works (if ever).

[wawatt]

please update open3d-0.19.0, thanks!

[BillyONeal]

I just read through #41916 as well. I don't think we'll realistically get a Filament port in vcpkg any time soon, probably not in this decade. Any chance we could "solve" this by just merging a open3d port into vcpkg for now that does not support filament/rendering? It can be disabled using Open3D's build using Open3D's CMake cache variables.

There is no requirement that the vcpkg port support every knob and feature in upstream as long as the unsupported ones are explicitly turned off and don't start digging around for other stuff on the system.

[vicroms]

We have closed this PR due to inactivity. Please create a new PR if you're still interested in getting these changes merged.