Skip to content

Commit 32dcb29

Browse files
danrbaileydanrbailey
authored
Merge pull request #2191 from danrbailey/notes_0804
Meeting Notes
2 parents f4d8702 + 31b11b0 commit 32dcb29

2 files changed

Lines changed: 346 additions & 0 deletions

File tree

‎tsc/meetings/2026-03-25.md‎

Lines changed: 189 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,189 @@
1+
Minutes from OpenVDB TSC meeting, March 25th, 2026, (EDT)
2+
3+
Attendees: *Jeff* L., *Nick* A., *Dan* B., *Andre* P.
4+
5+
Additional Attendees: N/A
6+
7+
Regrets: *Ken* M.
8+
9+
Agenda:
10+
11+
1) Confirm quorum
12+
2) Secretary
13+
3) Website & File Caches
14+
4) DCO/CLA PR Policy
15+
5) I/O Rewrite: PR Walkthrough
16+
6) Next meeting
17+
18+
--------------------
19+
20+
1) Confirm quorum
21+
22+
Quorum is present (four of seven TSC members).
23+
24+
2) Secretary
25+
26+
Secretary was Dan Bailey.
27+
28+
3) Website & File Caches
29+
30+
Dan confirmed that old VDB sample file caches have been migrated to
31+
GitLab LFS under the website repository; all links updated, files are
32+
at format version 223. Nick to close related open issues.
33+
34+
A known issue remains: two Doxygen HTML outputs share the same filename
35+
under different capitalizations (NanoVDB IO.h vs OpenVDB IO.h), causing
36+
conflicts on case-insensitive filesystems (Windows, and to a lesser
37+
extent macOS). Website repository changes currently require a Linux
38+
machine to avoid this problem.
39+
40+
4) DCO/CLA PR Policy
41+
42+
Several open PRs are missing DCO sign-off or a signed CLA, making them
43+
un-mergeable. Agreed approach:
44+
45+
- Dan to post comments on all affected PRs giving contributors one week
46+
to resolve sign-off; PRs that remain unaddressed will be closed with
47+
an invitation to reopen once the requirements are met.
48+
- Dan to add an explicit policy statement to the contribution
49+
guidelines so contributors are aware of this policy up front.
50+
51+
5) I/O Rewrite: PR Walkthrough
52+
53+
Dan shared his screen and walked through a series of PRs on the
54+
io_point_codecs feature branch. Dan noted these have been broken into
55+
small reviewable pieces; feedback on each is welcome before they are
56+
merged.
57+
58+
a) Test CMakeLists: wildcard removal
59+
60+
Removed the implicit wildcard appended to test-name filters in CMake so
61+
that passing "tree" no longer also runs unrelated tests prefixed with
62+
"tree". Callers can restore the old behavior with an explicit glob
63+
pattern (e.g., "tree*"). Jeff and Nick approved.
64+
65+
b) Unit test: stderr suppression
66+
67+
Redirected stderr around a section of unit tests that intentionally
68+
invokes an API path expected to emit a warning, to reduce noise when
69+
running with GTEST_BRIEF. Jeff raised the concern that silencing output
70+
may mask future unexpected warnings. Nick noted that in this specific
71+
case the warning is the correct behavior of the API under test. Jeff
72+
approved.
73+
74+
c) TreeBase: readTopology / writeTopology made pure virtual
75+
76+
Moved legacy buffer-count read/write (a pre-open-source 32-bit integer
77+
always valued 1) from TreeBase into the Tree derived class, and made the
78+
virtual methods pure virtual on TreeBase. ABI is unchanged (vtable slot
79+
preserved). This is technically a breaking change for anyone subclassing
80+
TreeBase directly; judged to be extremely rare. Nick to verify there is
81+
no impact on a streaming VDB implementation at Weta.
82+
83+
d) PointDataIO.H — new header, multi-pass restriction,
84+
out-of-core atomic removal
85+
86+
Three changes bundled together:
87+
88+
- New PointDataIO.H header isolates readCompressedValues for
89+
PointDataIndex32, removing the dependency on the full PointDataGrid
90+
header for I/O code.
91+
- Multi-pass I/O restricted to PointDataLeaf only via a static_assert
92+
type-trait; the generic multi-pass unit test is removed. This
93+
simplifies codec implementation for all non-point tree types.
94+
Nick suggested renaming the multi-pass tag class to be more
95+
explicitly PointData-specific, or moving it into PointDataIO.H, so
96+
that the restriction is obvious at the point of use; Dan agreed to
97+
investigate.
98+
- Out-of-core atomic removed from AttributeArray read/write paths in
99+
this branch (delayed loading not supported); removal guarded by an
100+
ABI_14 macro for eventual full removal in the next ABI version.
101+
102+
e) Large I/O refactoring (Archive / File / Stream)
103+
104+
A significant refactoring pass to consolidate grid I/O:
105+
106+
- All grid-level I/O now flows through two methods — archiveReadGrid
107+
and archiveWriteGrid — eliminating several scattered grid creation
108+
sites; grid objects are now created in a single location inside
109+
readGrid.
110+
- Pimpl pattern removed from File and Stream; data members inlined.
111+
Jeff flagged a potential header-bloat concern; Dan confirmed no new
112+
includes were required. The pimpl was likely introduced for
113+
delayed-loading and Boost isolation reasons; no longer warranted.
114+
Nick noted symbol visibility as an additional historical motivation
115+
but agreed removal is acceptable here.
116+
- GridDescriptor::read() deprecated (not removed); it created a grid
117+
object that callers never used. readHeader and readStreamPosition
118+
introduced as explicit replacements. Nick confirmed that Weta uses
119+
the stream-level API, not the GridDescriptor API directly.
120+
- readGridPartial read-topology flag removed (always called with
121+
false in practice); partial-read folded into readGrid as a boolean
122+
argument.
123+
- World-space bounding box conversion to index-space deferred until
124+
the last moment inside readGrid, eliminating the intermediate local
125+
readBuffers struct and simplifying the call path.
126+
127+
Nick and Jeff approved with the suggestions above incorporated.
128+
129+
f) ReadOptions / WriteOptions API
130+
131+
ReadOptions and WriteOptions objects are now passed uniformly to all I/O
132+
API entry points.
133+
134+
- ReadOptions currently holds: world-space clip bounding box and
135+
topology-only read mode (matching the two options previously passed
136+
as separate arguments).
137+
- WriteOptions currently empty; reserved for future extension.
138+
- All public API methods default-construct these objects; internal
139+
methods do not.
140+
141+
Feedback:
142+
143+
- Jeff: C++20 aggregate (named) brace initialization would give
144+
keyword-argument ergonomics; Dan to verify and document once VDB
145+
requires C++20.
146+
- Nick: virtual destructor on the structs should be removed if nothing
147+
derives from them; Dan to confirm ReadOptions is not subclassed
148+
(per-codec options use a separate typed-options map).
149+
- Nick / Andre: shared_ptr typedefs should be removed if unused; prefer
150+
unique_ptr or no typedef. Shared-ptr usage likely a legacy of the
151+
pimpl pattern now removed; agreed to clean up.
152+
153+
g) Codec framework — IOCodec, TopologyCodec, BoolCodec, registry
154+
155+
New framework for pluggable grid I/O codecs:
156+
157+
- IOCodec base class exposes: readTopology, writeTopology, readBuffers,
158+
writeBuffers, and createData (pure virtual).
159+
- TopologyCodec intermediate class implements standard depth-first tree
160+
topology read/write; multiple variants are planned for different
161+
compression schemes. Jeff clarified the hierarchy: codecs that need
162+
a specific topology compression derive from the appropriate
163+
TopologyCodec variant and only implement buffer methods; fully custom
164+
codecs (e.g., a neural codec) derive directly from IOCodec.
165+
- BoolCodec is the first concrete codec, deriving from TopologyCodec
166+
and overriding the buffer methods.
167+
- Codec registry mirrors the GridRegistry and MetaMap patterns; the
168+
codec is looked up by grid-type string inside archiveReadGrid and
169+
falls back to existing behavior when no codec is registered.
170+
- createData hook on the codec allows it to return a different grid type
171+
than the one it is registered under (e.g., reading a float grid as a
172+
half grid).
173+
- Per-codec options: ReadOptions contains a std::map<string,
174+
TypedOptions> for type-specific settings (e.g., filtering VDB Points
175+
attributes by name). Jeff suggested using each codec's globally-unique
176+
identifier string as a key prefix to avoid collisions; Dan agreed and
177+
will document the convention.
178+
- Error / warning reporting: currently throws on the first error. Jeff
179+
requested a mechanism for non-fatal warnings (e.g., a partial read
180+
silently loading the full grid because the codec does not support
181+
clipping); Dan to design a message list with string + severity-level
182+
fields, keeping hard errors as exceptions.
183+
184+
Dan noted the branch is still in progress; once these foundational PRs
185+
are merged, additional codecs (scalar, point-data, etc.) will follow.
186+
187+
6) Next meeting
188+
189+
April 8th, 2026, usual time.

‎tsc/meetings/2026-04-08.md‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
Minutes from OpenVDB TSC meeting, April 8th, 2026, (PDT)
2+
3+
Attendees: *Dan* B., *Nick* A., *Jeff* L., *Andre* P., *Jonathan* S.
4+
5+
Additional Attendees: N/A
6+
7+
Regrets: *Ken* M.
8+
9+
Agenda:
10+
11+
1) Confirm quorum
12+
2) Secretary
13+
3) SIMD Support via VCL (MR review)
14+
4) I/O Rewrite Updates
15+
5) Custom Grid Type Discussion
16+
6) Next meeting
17+
18+
--------------------
19+
20+
1) Confirm quorum
21+
22+
Quorum is present.
23+
24+
2) Secretary
25+
26+
Secretary is Dan Bailey.
27+
28+
3) SIMD Support via VCL (MR review)
29+
30+
Nick presented a new MR introducing SIMD support in OpenVDB using Agner Fog's
31+
Vector Class Library (VCL) as an optional, header-only dependency bundled in
32+
the `ext/` folder.
33+
34+
Background: VDB currently avoids explicit SIMD and relies on compiler built-ins
35+
(e.g., PopCount, count-leading-zeros) via x86 intrinsics or platform-specific
36+
workarounds, with some confusion around when software fallbacks are needed. This
37+
MR formalises x86 SIMD usage and lays groundwork for broader vectorisation.
38+
39+
Library choice:
40+
- VCL (Agner Fog): Apache 2, fully header-only in the configuration used.
41+
Supports SSE2 through AVX512 with emulation for sizes not natively supported by
42+
the targeted instruction set. Previously blocked by license incompatibility;
43+
now resolved since OpenVDB is itself Apache 2.
44+
- Google Highway was considered but rejected as it is not header-only (requires
45+
runtime linking) and introduces a heavier external dependency.
46+
47+
MR contents:
48+
- VCL added as an optional `ext/` subdirectory, installed with OpenVDB.
49+
- New `SIMD.h` header wrapping VCL into a custom OpenVDB VCL namespace to allow
50+
mixing other VCL versions downstream. Provides an `x86 instruction-set` macro
51+
(integer, following VCL values) written into `version.h` at pre-process time.
52+
- Generic API in `SIMD.h` covering VCL container types, `openvdb::math` tuple
53+
types, and plain arithmetic scalars, enabling type-agnostic algorithm
54+
implementations via a single templated function.
55+
- Type aliases (e.g., `Vec4F`, `Vec8S`) that resolve to either a VCL SIMD
56+
container or a math tuple depending on whether VCL is enabled.
57+
- A `SIMD_NATIVE_TYPE` macro returning the largest SIMD container type for a
58+
given element type and the targeted instruction set.
59+
- First vectorised tool: rasterised SDF spheres. Achieves a 2× throughput on
60+
doubles with no observable floating-point differences across architectures or
61+
optimisation levels. Uses `getBatchSize()` and a new `rasterizePoints` range
62+
overload to batch-process points.
63+
64+
CMake and detection:
65+
- New `USE_VCL` CMake option, off by default.
66+
- Detection is compile-time only, driven by compiler flags (`-mavx`, `-msse4.2`,
67+
etc.), not the host machine's capabilities. Compatible with CMake's native
68+
cross-compilation toolchain.
69+
- The two legacy defines (`OPENVDB_USE_AVX`, `OPENVDB_USE_SSE42`) are moved from
70+
build-time emission into `version.h`, so downstream header consumers now also
71+
see the configured instruction set. Nick noted these defines are inaccurate
72+
(they reflect CMake options, not whether the compiler flag was actually passed)
73+
and would like to remove them. Jeff suggested simply relying on VCL's
74+
detection from compiler flags, which package managers already set correctly.
75+
- Dan suggested emitting a CMake error if the old defines are set without the
76+
corresponding compiler flags, to guide migration; Nick acknowledged this is
77+
tricky to implement reliably.
78+
79+
Review feedback and agreed follow-ups:
80+
- Namespace/location: move the SIMD API from `openvdb::util::simd` to
81+
`openvdb::simd`, in a dedicated `simd/` folder. Jeff agreed this is
82+
appropriate given its fundamental nature; using `openvdb::simd::` also avoids
83+
extra `using` declarations in code already in `namespace openvdb`.
84+
- Readability: Jeff raised concern about verbose explicit function calls (e.g.,
85+
`simd::square(simd::sub(cx, px))`) vs operator syntax. Nick acknowledged the
86+
verbosity and noted explicit casting is intentional given that type promotions
87+
are now surfaced. He will investigate adding operator overloads in a follow-on
88+
PR; this is not a blocker for merging the current MR. Jeff noted binary
89+
operators make SIMD code significantly easier to read.
90+
- Template instantiation: Jeff suggested collapsing large type-trait
91+
instantiation lists using macros for clarity. Nick agreed to do this.
92+
- Testing: Dan requested a small test (potentially static assertions) that
93+
validates the correct instruction path is active for a given configuration, to
94+
catch accidental mis-wiring. Nick will add this.
95+
- CI: Nick will add a CI configuration with `USE_VCL` enabled.
96+
- Default on/off: consensus to keep `USE_VCL` off by default now. Plan to
97+
evaluate enabling it by default for VDB 14. Jeff suggested turning it on
98+
experimentally in CI around November.
99+
- Ken's email regarding NanoVDB SIMD was noted; Nick expects no conflict since
100+
NanoVDB already copies the OpenVDB math library and the new SIMD headers are
101+
independent.
102+
- Andre indicated support for the VCL approach.
103+
- C++ standard SIMD (`std::simd`, expected in C++26): noted as a long-term
104+
migration target; estimated 10–12 years before VFX platform reaches C++26.
105+
The current namespace wrapping is intended to ease that future transition.
106+
107+
4) I/O Rewrite Updates
108+
109+
Dan raised two items related to his ongoing I/O rewrite work.
110+
111+
a) File format backward-compatibility fix:
112+
An off-by-one error was introduced when removing legacy I/O modes. The current
113+
code supports files from node-mask compression (format version 223, VDB 3,
114+
~2013) onwards, but inadvertently dropped support for version 222. The
115+
intention was to support from VDB 1.1 (the first Houdini release) onwards.
116+
Dan will restore the missing branch to re-enable reading version 222 files.
117+
Jeff confirmed this was the agreed scope. A small number of regression-test
118+
sample files predated version 223.
119+
120+
b) ReadDiagnostics API design:
121+
Dan described the current implementation: `ReadDiagnostics` is a nullable
122+
pointer passed into codecs; codecs check for null before populating it.
123+
Callers opt in by calling `enableReadDiagnostics()` on the archive, which
124+
allocates the object.
125+
126+
Nick suggested an alternative: `ReadDiagnostics` always exists as a value
127+
object, with `addWarning()` being a no-op when not enabled, removing the
128+
risk of null-pointer mistakes in codec implementations. Jeff agreed and
129+
proposed adding an `enabled` flag to the object so that codecs can also
130+
gate expensive diagnostic-collection logic cheaply. Dan will adopt this
131+
approach: always-present object with an `enabled` flag, with `addWarning()`
132+
being a no-op when disabled.
133+
134+
c) Feature branch review process:
135+
Dan has ~4–5 stacked MRs targeting an I/O feature branch rather than master,
136+
with each MR targeting the previous branch for reviewable diffs. He asked for
137+
guidance on review cadence. Jeff noted that the team needs to merge these
138+
incrementally or the stack will become unwieldy. The refactoring MR (large API
139+
changes to unify read/write paths and simplify codec integration) is the
140+
critical one that unblocks subsequent PRs. Team agreed to prioritise reviewing
141+
these promptly.
142+
143+
5) Custom Grid Type Discussion
144+
145+
Dan asked about a Weta-internal streaming grid type that Nick had mentioned, to
146+
understand how a non-`TreeBase`-derived grid handles I/O — useful context for
147+
the I/O rewrite's generality.
148+
149+
Nick: the implementation is a full reimplementation of the tree hierarchy (grid,
150+
tree, root, internal, and leaf nodes, and leaf buffer), with a unique grid type
151+
deriving from `GridBase` (not `TreeBase`). It has custom K-I/O. Nick is working
152+
on open-sourcing it, but Dan should not wait on it and should proceed with his
153+
I/O changes independently.
154+
155+
6) Next meeting
156+
157+
April 22nd, 2026, usual time.

0 commit comments

Comments
 (0)