Massive improvements to vdb tool

[kmuseth]

New Features:

• Added new action "-files", a control-flow scope (closed by -end, like -for and -each) that iterates over the files in a directory, binding each matching path to the loop variable "file" so the body can reference it as "{$file}" (e.g. "-files path=$HOME/data ext=obj,stl -read '{$file}' -mesh2ls -print -clear -end"). The mandatory "path" option sets the directory where the search begins; "recursive=1" descends into sub-directories. Matches are filtered by "extension" (a comma-separated list, e.g. ext="obj,ply,vdb"), by "include"/"exclude" filename patterns, and by file size via "min_size"/"max_size" which accept plain byte counts or human-readable suffixes (1B, 1KB, 1MB, 1GB, 1TB). As with the other loops, "quiet=true" suppresses the per-iteration "Processing: ..." output for this scope only.
• Added new action "-mesh2udf" (alias "-soup2udf"), which converts a polygon mesh or polygon soup into an unsigned distance field, i.e. a VDB grid storing the non-negative distance to the input surface within a symmetric narrow band on both sides of it. Unlike a signed level set this does not require the input to be watertight or consistently oriented, so it works on open meshes and arbitrary triangle/quad soups. The output resolution is set by "dim" (largest bbox dimension in voxels, default 256) or by "voxel" (explicit world-space voxel size, which takes precedence), while "width" controls the half-width of the narrow band in voxel units (default 3). The transform can instead be inherited from an existing grid via "vdb" (its stack age); "geo" selects which geometry to process.
• Added new action "-quad2tri" (alias "-q2t"), which triangulates every quadrilateral face in a polygon mesh, replacing each quad with two triangles so the result contains triangles only. This is useful before exporting to formats or feeding downstream tools that accept triangles exclusively. The split is a naive fan from the quad's first vertex (the diagonal between vertices 0 and 2), which is correct as long as each quad is planar and convex; the action assumes this and does not test for it, so non-planar or concave quads may produce a skewed or self-overlapping pair of triangles. Existing triangles in the mesh are left untouched. The "geo" option selects which geometry on the stack to process (defaults to the most recently inserted) and "keep" toggles whether that input is preserved or deleted after the conversion.
• Added new action "-vol2mesh" (alias "-vdb2mesh"), which extracts a polygon mesh from any scalar VDB grid by meshing one of its iso-surfaces, producing a surface of triangles and quads at the chosen value. Unlike "-fog2mesh" (which defaults to the 0.5 iso-value of a normalized fog volume) it defaults to the zero iso-value, making it the general-purpose mesher for level sets and scalar fields. The "iso" option sets the iso-value, and "adapt" controls adaptive meshing on a normalized 0 -> 1 scale where 0 yields a uniform tessellation and higher values simplify flat regions into larger polygons. An optional grid given via "mask" restricts meshing to the region it covers, and "invert=true" meshes the complement of that mask. "vdb" selects the grid, "name" sets the output name, and "keep" toggles whether the input is preserved.
• Added new action "-fog2mesh", which extracts a polygon mesh from a fog-volume VDB grid by meshing one of its iso-surfaces, producing a watertight surface of triangles and quads at the chosen density value. The "iso" option sets the iso-value defining the surface (default 0.5, suitable for a normalized fog volume), and "adapt" controls adaptive meshing on a normalized 0 -> 1 scale where 0 yields a uniform tessellation and higher values progressively simplify flat regions into larger polygons. An optional grid given via "mask" restricts meshing to the region it covers, and "invert=true" instead meshes the complement of that mask. "vdb" selects which grid to mesh, "name" sets the output geometry's name, and "keep" toggles whether the input volume is preserved (the mask is never removed).
• Added new action "-slice", which generates one or more images of axis-aligned planar slices through a scalar VDB grid, writing each as a PPM image file. Slice positions are given as normalized coordinates in the range 0 -> 1 across the grid's active bounding box via the "X", "Y" and "Z" options (each accepts one or more values, defaulting to 0.5, i.e. the mid-plane), so a single invocation can emit many slices along any combination of axes. The "file" option sets the output base name, "scale" sets the image pixel size (e.g. 512 or 1920x1080, with the aspect ratio derived from the grid when only one dimension is given), and "force" recomputes the true min/max for color mapping instead of assuming the expected ranges for level-set and fog-volume grids. "vdb" selects which grid to slice.
• Added new action "-img2mpeg", which converts multiple image files to an mpeg movie file when vdb_tool is optionally built with ffmpeg.
• Added new action "-log", which mirrors the tool's diagnostic output (timing reports, warnings, non-fatal errors and -print output) to a log file for the remainder of the pipeline. It redirects all three standard text streams (std::clog, std::cerr and std::cout) so nothing normally printed to the terminal is lost, and the file is unit-buffered so "tail -f" or "watch -n 0.5 vdb_tool.log" reflect progress in real time. A self-describing header (date, version and the full command line) is written first so a saved log is readable later. The "file" option names the log (a timestamped default is used when omitted), "append" adds to an existing file instead of truncating it, and "tee" (default true) also echoes to the terminal so interactive feedback is preserved; set tee=false to route output exclusively to the file.
• Added new action "-sdf2udf", which converts a signed distance field (a level set, negative inside the surface and positive outside) into an unsigned distance field by taking the absolute value of every voxel, so the result stores the non-negative distance to the surface on both sides. Because the inside/outside sign is discarded, the grid class is changed from level set to UNKNOWN, signalling to downstream actions that it is no longer a valid signed level set (so sign-dependent operations such as CSG booleans or signed-flood-fill should not be applied to it). This is useful when only proximity to the surface matters. The "vdb" option selects which grid (or grids) on the stack to process, "name" sets the output grid's name, and "keep" toggles whether the input volume is preserved or deleted after the conversion.
• Added new action "-forAllValues/forOnValues,forOffValues", which applies a math kernel to all, active, or inactive values in a grid. The reserved variable "v" is bound to the current voxel value; every other identifier is looked up once in the Processor's string memory (the same namespace used by -eval / -calc) and bound as a per-voxel constant, throwing if any referenced name is undefined. This lets kernels mix the voxel value with scalars set earlier in the pipeline, e.g. -eval '{2:@scale}' -forOnValues 'scalev + 1'. The kernel option accepts both the explicit form ("-forOnValues kernel='sin(v)+1'") and a bare positional form ("-forOnValues 'sin(v)+1'"); the bare form is enabled by the new greedy-anonymous parser mode (see Action::Action) and does not interfere with other named options of the same action such as keep= or class=. Stencil kernels: writing "v(dx,dy,dz)" with integer-literal offsets reads a relative neighbor voxel through a per-thread ConstAccessor (e.g. "v(1,0,0)-v(-1,0,0)" computes a finite-difference x-derivative or "v(1,0,0)+v(-1,0,0)+v(0,1,0)+v(0,-1,0)+v(0,0,1)+v(0,0,-1)-6v" computes a discrete Laplacian); the output grid is internally deep-copied so reads come from a stable snapshot. Multi-grid kernels: use= and vdb= accept comma-separated lists so the kernel can read from multiple grids in one pass; the first entry is the output (iterated and written), the rest are read-only inputs (e.g. "-forOnValues 'a - b' use=a,b vdb=0,1" computes pointwise differences).
• Added new class Calculator, a compact bytecode interpreter for math expressions over arbitrarily named float inputs. Accepts RPN (e.g. kernel="$x:abs:1:+"), single infix (e.g. kernel="abs(x)+1"), and infix multi-statement programs with assignment (e.g. kernel="t = x*x; t + sin(t)"). Input variables are discovered automatically and exposed via Calculator::variables(); the forAllValues/forOnValues/forOffValues actions only bind "x" and throw with a clear message on any other identifier.
• Added new action "-calc", which runs a Calculator expression at command-line scope. Input variables are read from the Processor's string memory (the same namespace used by -eval); intermediate slot values and trailing-LHS names are written back so subsequent -eval / -calc / -for actions can read the results by name (e.g. "-calc 'a=1+2;b=a*3' -eval str='{$b}'"). Read-only inputs (e.g. a -for loop variable) are left untouched. Accepts both the explicit form ("-calc kernel='x=1+2'") and a bare positional form ("-calc 'x=1+2'"); the latter is enabled by the new greedy-anonymous parser mode (see Action::Action). Echoes the result only when the kernel ends in a plain expression; assignment-terminated kernels are silent since their outputs are already in memory.
• Added new actions "-switch" and "-case", a multi-way control-flow construct analogous to -if but selecting one of N sub-pipelines. "-switch on=" opens a scope closed by -end; each enclosed "-case key=" runs its body only if its key matches the selector, and "key=" (or "key=default") is a catch-all that fires when no earlier case matched. Numeric keys are compared by value (so key=1 matches a selector of 1.0) and otherwise strings are compared verbatim. The selector accepts {...} template expressions, so e.g. "-for c=0,4 -switch on={$c} -case key=0 -sphere -end -case key= -platonic -end -end -end" picks a primitive per iteration.
• Added triangulation of N-gons to Geometry::readOBJ and Geometry::readSTL, assuming they are convex and planar.
• Added the string-similarity helpers vdb_tool::levenshtein (the classic edit-distance between two strings) and vdb_tool::fuzzyMatch (which ranks a list of candidate names against a query by combining substring matching with edit distance) to power "Did you mean ...?" suggestions throughout the tool. A mistyped action, option, or expression function/constant no longer just fails: the error now lists the closest known names, e.g. "-sphre" suggests "-sphere", "-erode raidus=2" suggests "radius=", and a kernel calling "tanj(x)" suggests "tan" or "atan". The same routine backs action-name matching in the Parser, option-name matching within an Action, and identifier matching in the Calculator, so the suggestions stay consistent across all three.

Improvements:

• Action "-read" now has optional support for reading UsdGeomMesh and UsdGeomPoints prims from OpenUSD files (.usd, .usda, .usdc, .usdz). Walks every prim, bakes each one's world transform into the vertex positions, preserves triangles and quads as-is, and fan-triangulates n-gons (n>4). UsdGeomPoints contribute vertex positions only. Instancing, subdivision, animation, and per-point widths are not supported by this minimal reader. Enable with VDB_TOOL_USE_USD.
• Action "-read" now has optional support for reading glTF (.gltf) and binary glTF (.glb) files. Walks every mesh primitive in TRIANGLES mode (POINTS/LINES/STRIPS/FANS are skipped with a warning), and consumes both indexed and non-indexed primitives with UBYTE/USHORT/UINT index buffers. Node-graph transforms are not applied, and materials, normals, UVs, and vertex colors are ignored by this minimal reader. Uses the tinygltf header-only library, fetched at configure time. Enable with VDB_TOOL_USE_GLTF.
• Action "-read" now supports ASCII XYZ point files (.xyz), where each line holds the x, y and z coordinates of a single point separated by whitespace. Blank lines and lines beginning with '#' are skipped as comments; any other line that does not parse as three numbers raises an error identifying the offending text. The points are accumulated as a vertex-only primitive (no faces), suitable for conversion to a VDB via -points2ls or -points2vdb. The data can also be streamed from standard input using the name "stdin.xyz".
• Action "-mesh2ls/mesh2sdf" now supports generation of asymmetric narrow bands.
• Improved error reporting when using unsupported actions and options.
• Actions can now have multiple names (or aliases), e.g. "-read", "-import", "-load", "-i".
• Added cutoff option to "-ls2fog" action.
• Improved error reporting in vdb_tool (prints diagnostic error messages).
• Improved error handling in vdb_tool (will recover from non-fatal errors).
• Improved parsing of configuration files (tabs and trailing comments are now ignored).
• Improved unit-tests and Doxygen documentation for all files related to the vdb_tool.
• Improved format and information for the -print action: output is now a column-aligned Unicode table, and a new "level" option controls detail. level=0 is the base table (age, name, type, class, dim, voxels, dx, size), now including the grid class (level set, fog volume, etc.). level=1 adds the active-voxel bounding box in both index space and world space and a per-level tree node-count column labeled "32->16->8" (e.g. "2->10->58" = 2 upper-internal, 10 lower-internal, 58 leaf nodes). level=2 additionally reports the background value, the active value range, and -- for level sets -- the world-space surface area and enclosed volume (via tools::levelSetArea / tools::levelSetVolume).
• Action "-if" now accepts infix and RPN test expressions (via the Calculator), in addition to the existing 0/1/true/false literals and {...} expressions. For example "-if 'n > 0'" works where the variable n is read from the Processor's string memory, so "-for n=0,10 -if 'n > 5' ... -end" branches per iteration.
• Actions "-for", "-each" and "-files" gained a "quiet" option (default false) that suppresses the per-iteration "Processing: ..." line for that loop without silencing the rest of the pipeline (unlike the global -quiet).
• Action "-log" now redirects std::cerr and std::cout in addition to std::clog (so warnings/errors and -print output also land in the log file), uses unit-buffered output so "watch"/"tail -f" see updates in real time, writes a self-describing header (date, version, full command line), and gained "append" (append rather than truncate) and "tee" (also echo to the terminal, default true) options.

Fixes:

• Fixed issues in Geometry::readSTL so it works on all files in Thingi10K.
• Fixed bug when reading ascii ply files and off files with normals.
• Fixed out-of-bounds read in -slice when the grid's active values had been modified (e.g. by -forOnValues) so that sampled values fell outside the precomputed extrema range; the LUT index is now clamped to [0, 255].
• Fixed several typos in the documentation of vdb_tool.
• Fixed vdb_tool::uuid so it generates a 36-character RFC 4122 v4 UUID.
• Fixed corner cases in getFile/getName/getPath/getExt/replacePath/replaceExt by adopting .
• Fixed handling of whitespace in greedy-anonymous action values (e.g. "-calc a = 1+2" or an -if/-switch expression with spaces) so that shell- or config-file-tokenized arguments are space-joined rather than comma-joined, which previously corrupted the expression.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: Idclip / name: Nick Avramoussis (088527a, 0b37233, 11fe358, 1fe6702, 22c7810, 2ecd8f3, 30666fd, 30b537d, 437875f, 475ef36, 4ec2411, 544190c, 58bc64a, 77b2b5d, 7d506bd, ac2e13d, ae01ce7, d071917, d3d7fcf, de2beee, fc93727)
• ✅ login: apradhana / name: Andre Pradhana (6f99494)
• ✅ login: apradhana / name: apradhana (733ca17)
• ✅ login: areidmeyer / name: Andrew (358f6da)
• ✅ login: boberfly / name: Alex Fuller (c9bd4ca)
• ✅ login: danrbailey / name: Dan Bailey (0ec5a31, 29fdff0, 2a40e47, 3fe1482, 563fe83, 5a79aa3, 69b859b, 6a20fea, 73af5d2, 747f970, af6d21e, be4f270, d11b61f, da3cf29, dc0fe76, e287f84, e6543f6, 09e58af, 1043e16, 2163471, 238e586, 29af0b2, 29b5dff, 2bd9070, 2f05ecd, 49727f6, 4f5a902, 5bde6dc, 609a19f, 7806c8e, 7d6832d, 814ce30, 8f30836, 94c9d49, 9787995, d61220e, d69862a, f476145)
• ✅ login: jmlait / name: Jeff Lait (061597b, 1f57a10, 4b6afe9, 6206bd0, 85a5d3b, 8ba6d31, 8d49a50, 9ac662c, c21e8b1, e94f62b)
• ✅ login: jmlait / name: jmlait (6ae3b06, ba6ec74, c313acf)
• ✅ login: kmuseth / name: Ken Museth (7030e77, a0fefc4, 003e18d, 008fcfd, 01b792f, 01c3c6b, 044cc5f, 05ca232, 0651e12, 0690431, 0b5a185, 0d4cf92, 108743b, 11835c0, 11a90ef, 1235134, 12a79ee, 1615df3, 16289e6, 18ea0dc, 1ad76da, 1ea2961, 1f0d204, 22b1266, 2437d35, 2470dce, 2549b79, 25556b7, 270cd67, 2a1adec, 2b38319, 2b8a2d5, 2bb7fcd, 2d0d8d8, 2e09066, 2f01a8a, 2f28ad5, 31188ad, 3446183, 3582f75, 399cba8, 39efaeb, 3e51b42, 3e84d94, 40132c3, 40e5c1a, 42ea51f, 430ede2, 43c571f, 43de2e3, 443221f, 4492fc8, 4558ef3, 457923e, 465c4d2, 470c99c, 470d617, 47ce0c8, 49ca51a, 4b07d97, 4b150d2, 4bde39b, 4bf0b53, 4c9aab6, 505ca1d, 546b86a, 54b1e49, 54e4078, 555c7a2, 5594b15, 568b07c, 59b4503, 5b3312d, 5c26f15, 5c8adc0, 5d1d089, 5d82714, 5eb1964, 5eb31b2, 600fbad, 6188eaa, 62846aa, 6588c44, 6738c4d, 6909648, 6a30a43, 6a7716f, 6c72d4b, 6cbc9c2, 6dba711, 6e3851a, 6ebf82a, 70018ed, 706e2fd, 7070cc1, 7331cbb, 752b0e5, 7558cbe, 7657e2b, 76f071c, 76f924b, 7817e18, 7829dfb, 7848ce8, 78752e1, 798ef00, 7adde81, 7bda52e, 7bda688, 7c64387, 7cf4198, 7e4a4e4, 8269e25, 8573ed6, 87a5890, 88bc745, 89c2612, 8a8be3a, 9007165, 919e559, 91a139f, 931a489, 937514f, 94f6d6e, 95866ef, 968ce71, 9737554, 9a1119b, 9bc561d, 9ce9ce9, 9d86849, 9e46019, 9e72544, 9e7452f, 9f0a4cb, a183a92, a423053, a49e070, a734ef6, a8525b9, a864184, acd3675, ad5ae9f, af49e45, afe9ab7, affe5ee, b1c703c, b3dcbfd, b43b383, b4aed3a, b570fb0, b5ef5d7, b61f588, b740c83, b743393, b8a7017, b8c9a00, b922803, bb25954, bc47e5c, bd9ab12, be5c73c, bfd4414, c22ff41, c388cc9, c3a13f7, c3b9e5a, c4b1fdf, c51f7a6, c5cb54e, c64832b, c7930b0, c7e40ea, c8b526d, ca56b73, cdf3a40, d0078f9, d058703, d1954e8, d3c437c, d563f21, d601927, d8a0d4c, dffaed7, e35666e, e3b3268, e75939d, e80571a, e9e0904, ed67752, ef90fa3, f60a647, f6294c5, f6bb3ab, f6e1318, f7f69d4, fb04324, fe4e9da, 054a43c, 06a38ad, 08ecd14, 09fbad6, 18332dc, 1945a45, 2977ff6, 2e326ec, 32bfd45, 489397f, 49e3749, 5a1d810, 6042fe0, 68baf93, 6e20818, 765400c, 8171958, 821ac83, a275d6a, a43008d, a5f721e, ac952ca, acd69bd, b0d291e, e775b5e, eaf7824)
• ✅ login: rohan-sawhney / name: rohan-sawhney (20e9d75, 43e570e, 70ca7b2, 8416906, fefda65)
• ✅ login: sifakis / name: Eftychios Sifakis (85438f9, 9033991)
• ✅ login: swahtz / name: Jonathan Swartz (0110bf6, 02652dd, 05fe3ab, 0daa226, 272e2d4, 3bf14ef, 476db06, 494a6e5, 72455de, 85aab0d, a036711, d72d0de, de4cecf)

[richhones]

Very high level comment, the design of the kernel and forallvalues (and other iterators) actions looks like it provide a similar interface to what you might do with VDB AX, can you not chain the use of vdb_tool through the vdb_ax cli to provide the same functionality? I can see that some things that are missing in current AX functionality are supported (user-def functions for example) but would love to get efforts towards getting these working in AX instead of making a tool that overlaps in functionality.

[kmuseth]

Very high level comment, the design of the kernel and forallvalues (and other iterators) actions looks like it provide a similar interface to what you might do with VDB AX, can you not chain the use of vdb_tool through the vdb_ax cli to provide the same functionality? I can see that some things that are missing in current AX functionality are supported (user-def functions for example) but would love to get efforts towards getting these working in AX instead of making a tool that overlaps in functionality.

Great observation @richhones, and that has definitely crossed my mind. In fact, that is the goal, but there are two reasons why it shouldn't block this PR.

First, this PR is carefully crafted to introduce a lot of new features without any new mandatory dependencies. As you know, AX requires several new dependencies that would have to be made optional for vdb_tool. Second, while this implementation of stream processing does conceptually overlap with AX, it is a lightweight stand-in for fast prototyping. It has no JIT compilation—so it's presumably much slower—and is by no means a replacement for AX.

So, yes, I completely agree AX should be integrated into vdb_tool moving forward, but I'm not convinced it should block this already massive PR.

However, I'm happy to prioritize AX integration as the next improvement (though the memory management between scoped variables in AX and the vdb_tool will be nontrivial)