Skip to content

Commit 797e2d7

Browse files
Merge pull request #736 from fernandotonon/feat/cli-generate-texture-403
feat(cli): qtmesh material --generate-texture — depth-conditioned mesh texture (#403)
2 parents 655df48 + dc67797 commit 797e2d7

4 files changed

Lines changed: 331 additions & 1 deletion

File tree

CLAUDE.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,8 @@ qtmesh lod model.fbx --auto # auto-generate LODs
6767
qtmesh lod model.fbx --remove -o clean.fbx # strip LODs and save
6868
qtmesh material model.fbx --preset "Metallic-Roughness" -o out.fbx # apply a built-in material preset (writes .material sidecar)
6969
qtmesh material --list-presets # list built-in preset names (incl. PBR templates)
70+
qtmesh material model.fbx --generate-texture "rusty bronze armor" -o out.fbx # AI mesh-aware (depth-conditioned) texture → diffuse (needs SD build + base model; run `uv --unwrap` first if no UVs)
71+
qtmesh material model.fbx --generate-texture "..." --model mybase.safetensors --controlnet-strength 0.8 --width 768 --height 768 -o out.fbx # explicit SD base model + ControlNet strength + size
7072
qtmesh scan ./assets # scan directory for asset issues
7173
qtmesh scan ./assets --profile example-minimal # built-in platform validation preset
7274
qtmesh scan ./assets --list-profiles # list bundled profile ids
@@ -237,7 +239,7 @@ Three singletons manage core state. All run on the main thread. Access via `Clas
237239
- **MeshOptimizerLod** (`src/MeshOptimizerLod.h/cpp`, issue #398): Thin facade over `zeux/meshoptimizer` for LOD generation. Free functions, no singleton. `generateLods(mesh, reductions)` returns one `LodLevel` per requested reduction, each with one `Ogre::IndexData*` per submesh. Uses `meshopt_simplifyWithAttributes` when UV0 is present (preserves UV seams), falls back to `meshopt_simplify` otherwise. Every result is `meshopt_optimizeVertexCache`-reordered (Forsyth) so the LOD is cache-friendly out of the box. Caller takes ownership of the `IndexData*` (commit to `SubMesh::mLodFaceList` or call `destroyLevel`).
238240
- **MeshLodController** (`src/MeshLodController.h/cpp`): Now has `Algorithm` enum (`Ogre` | `Meshopt`) on the C++ overload `generateLods(int, const QVariantList&, Algorithm)`. QML-facing `generateLodsWithAlgo(int, QVariantList, QString)` accepts `"ogre"` / `"meshopt"` for the Inspector backend dropdown. Default is `Ogre` — meshoptimizer's attribute-weighted simplify preserves UV seams + skin weights but in practice produces a softer silhouette than Ogre's stock `MeshLodGenerator` on character meshes, so Ogre stays primary. CLI: `--algo ogre|meshopt` (default `ogre`). MCP `generate_lods` tool: `algo` param (default `ogre`). Sentry breadcrumb category `ai.assist.lod` records the chosen backend when meshopt is used.
239241
- **MeshDecimator** (`src/MeshDecimator.h/cpp`): Same `Algorithm` enum exposed on `decimateEntity(entity, reduction, algo)`. `MeshDecimatorController::applyReductionWithAlgo(double, QString)` is the QML-facing variant the Inspector's Decimate section dropdown calls. CLI `qtmesh decimate ... --algo ogre|meshopt`; MCP `decimate_mesh` `algo` param. Same default and breadcrumb category as the LOD path (`ai.assist.decimate` for meshopt). The post-decimation `promoteFirstLodToBase` also erases the `qtme.faces.<i>` n-gon bindings, otherwise FBXExporter (and EditableMesh) rehydrate the original triangle list off the cached binding and emit the un-decimated mesh.
240-
- **Mesh-aware texture generation** (`src/MeshDepthRenderer.h/cpp` + `MaterialEditorQML::generateMeshTextureFromPrompt`, issue #403): depth-conditioned (ControlNet) texture generation. Renders a grayscale depth map of the selected entity (`MeshDepthRenderer`: offscreen RTT, auto-framed camera from the **-Z front**, flat white-emissive material + scene LINEAR fog for the near=white/far=black gradient — no custom shaders; the grid node + bounding box + other entities are hidden during capture so only the target silhouette is captured), feeds it to sd.cpp's `sd_img_gen_params_t.control_image` with a ControlNet depth model (`control_v11f1p_sd15_depth`, auto-discovered in the sd_models dir; falls back to plain txt2img if absent), and applies the result to the active material's diffuse. **All `#ifdef ENABLE_STABLE_DIFFUSION`-guarded** (flag is OFF by default). Surfaced via the **"Use selected mesh (depth-conditioned)" checkbox** in the Material Editor's existing AI texture-generation panel (`qml/TexturePropertiesPanel.qml`, not a separate dialog), and the MCP `generate_mesh_texture` tool. `SDWorker::generateTextureControlled` carries the control image; `recreateContext` loads `control_net_path` + disables `vae_decode_only` when a ControlNet is set. **Known limitation:** in-app application of the generated diffuse to RTSS-rendered PBR materials whose diffuse TUS is unnamed/numeric is unreliable (the Cook-Torrance SRS only recognizes named `albedo`/`diffuse_map` slots) — as a workaround the Material Editor's Preview section has a **"Save Texture As…"** button (`exportCurrentTexture`) so the user can export the generated image and apply it via other tools. Mapping is planar/view-projection (not a full UV reprojection bake). Sentry breadcrumb category `ai.assist.mesh_texture`.
242+
- **Mesh-aware texture generation** (`src/MeshDepthRenderer.h/cpp` + `MaterialEditorQML::generateMeshTextureFromPrompt`, issue #403): depth-conditioned (ControlNet) texture generation. Renders a grayscale depth map of the selected entity (`MeshDepthRenderer`: offscreen RTT, auto-framed camera from the **-Z front**, flat white-emissive material + scene LINEAR fog for the near=white/far=black gradient — no custom shaders; the grid node + bounding box + other entities are hidden during capture so only the target silhouette is captured), feeds it to sd.cpp's `sd_img_gen_params_t.control_image` with a ControlNet depth model (`control_v11f1p_sd15_depth`, auto-discovered in the sd_models dir; falls back to plain txt2img if absent), and applies the result to the active material's diffuse. **All `#ifdef ENABLE_STABLE_DIFFUSION`-guarded** (flag is OFF by default). Surfaced via the **"Use selected mesh (depth-conditioned)" checkbox** in the Material Editor's existing AI texture-generation panel (`qml/TexturePropertiesPanel.qml`, not a separate dialog), the MCP `generate_mesh_texture` tool, and the **CLI `qtmesh material <file> --generate-texture "<prompt>" [--model <name>] [--controlnet <path>] [--controlnet-strength <0..1>] [--width N] [--height N] [-o out]`** (`CLIPipeline::cmdMaterialGenerateTexture`). The GUI/MCP paths are fire-and-forget (the long-running process applies the result on `SDManager::generationCompleted`); the CLI is a one-shot `_exit()` process so it loads the base model and drives the async `SDManager` worker **synchronously via two local `QEventLoop`s** (one for `modelLoadCompleted`/`modelLoadError`, one for `generationCompleted`/`generationError`/`generationStopped`), then copies the PNG next to the output mesh, binds it as the diffuse TUS on every submesh, and re-exports. `SDWorker::generateTextureControlled` carries the control image; `recreateContext` loads `control_net_path` + disables `vae_decode_only` when a ControlNet is set. **Known limitation:** in-app application of the generated diffuse to RTSS-rendered PBR materials whose diffuse TUS is unnamed/numeric is unreliable (the Cook-Torrance SRS only recognizes named `albedo`/`diffuse_map` slots) — as a workaround the Material Editor's Preview section has a **"Save Texture As…"** button (`exportCurrentTexture`) so the user can export the generated image and apply it via other tools. Mapping is planar/view-projection (not a full UV reprojection bake). Sentry breadcrumb category `ai.assist.mesh_texture`.
241243
- **SkinWeights** (`src/SkinWeights.h/cpp`, issue #402): inverse-distance ("closest-point-on-bone") automatic skin weights. The issue proposed wrapping libigl's bounded biharmonic weights (BBW), but BBW requires tetrahedralization via TetGen — which is **GPL/copyleft**. Adopting it would force the entire binary to GPL and close off Homebrew / Snap / WinGet redistribution under the project's permissive-license stance. This first slice ships a native heuristic with **zero new dependencies**: for each vertex, compute its distance to every bone's segment (line from bone-head to the average of its children, falling back to point distance for leaf bones in the skeleton's bind pose), apply `1/dist^falloff` weighting, keep the top-K bones (default K=4 matches hardware skinning), and normalize. This is the same algorithm Maya / 3dsMax use as their default "smooth bind." Distance cap (`maxInfluenceDistance` × mesh-diagonal) prevents a finger bone from picking up weight on a foot. Optional `skipUnweightedBones` filters Mixamo helper bones. `replaceExisting=false` enables a merge mode for "fill in missing weights" workflows. Surfaced via `qtmesh skin --max-influences N --falloff F -o out`, MCP `compute_skin_weights`, and the **Animation Mode → Mode Tools → "Skinning" section → "Compute Skin Weights…" button** (`qml/SkinWeightsDialog.qml`, driven by `SkinWeightsController` singleton). Lives in Animation Mode (not Edit Mode) because skinning governs how the mesh deforms under animation — a rigging step, not a mesh-topology edit. The button binds to `hasSkinnedSelection` so it disables on static (skeleton-less) meshes. The GUI path runs through `ComputeSkinWeightsCommand` (`src/commands/`) so the auto-skin is **undoable** (Ctrl+Z): the command snapshots every submesh's `VertexBoneAssignmentList` (+ the mesh-level shared list) before the first `redo`, runs `computeAndApply`, and on `undo` restores the snapshot and calls `_compileBoneAssignments` to re-pack the blend buffer. (Unlike the UV-unwrap restore, recompiling is safe here because the vertex buffer object is unchanged — only the blend bytes are rewritten.) Sentry breadcrumb category `ai.assist.skin_weights`. A future slice can plug libigl BBW in behind `-DENABLE_LIBIGL_BBW` for users who accept the GPL implications. Verified on Rumba Dancing.fbx: 69 bones, 5828 verts → 20,129 vertex-bone assignments (avg 3.45 influences/vert), valid glTF round-trip.
242244
- **QuadRetopo** (`src/QuadRetopo.h/cpp`, issue #401): triangle-pairing quad-dominant retopology. The issue proposed wrapping Instant Meshes (Wenzel Jakob), but Instant Meshes ships as a research GUI app with no clean C++ library API and has been dormant since 2016. QuadriFlow (the production-grade alternative used by Blender 3.0+) requires Boost + Eigen + LEMON — heavy deps the project doesn't currently use. This first slice ships a native triangle-pairing backend with **zero new dependencies**: walks every interior edge whose two adjacent faces are triangles and scores the merge by (1) coplanarity (dot product of triangle normals; default `maxAngleDeg=25°`), (2) quad shape (deviation of interior angles from 90°; default `shapeToleranceDeg=65°`), (3) aspect ratio (longest/shortest edge; default `maxAspectRatio=6.0`). Pairs are taken greedily best-first; each triangle claimed at most once. Quads are emitted with opposing-corner winding `(opposing0, sharedA, opposing1, sharedB)`. Output goes through `EditableSubMesh::faces` → `triangulateFaces` (fan retri for GPU) → `writeNgonFacesToMesh` (n-gon binding for exporters / Edit Mode). **No new vertices** are introduced, so UVs and skin weights survive unchanged. Backends are pluggable via the `Algorithm` enum (only `TrianglePair` implemented; future `QuadriFlow` / `InstantMeshes` slot in here). Surfaced via `qtmesh retopo --target-faces N --max-angle DEG -o out`, MCP `retopologize`, and the **Material Mode → Mode Tools → "Quad Retopology…" button** (`qml/QuadRetopoDialog.qml`, driven by `QuadRetopoController` singleton). Sentry breadcrumb category `ai.assist.retopo`. Verified on Rumba Dancing.fbx: 10,220 tris → 6,032 faces (4188 quads + 1844 tris), 82% quad dominance. Hard lower bound on face count is ~50% of input (every triangle paired); strict gates typically land 60-70%.
243245
- **UvUnwrap** (`src/UvUnwrap.h/cpp`, issue #400): xatlas-backed automatic UV unwrap. xatlas is the MIT library Blender and Godot use under the hood — single-translation-unit `xatlas.cpp` vendored via FetchContent and wrapped in an inline `add_library(xatlas STATIC …)` target (no upstream CMake config). Pipeline: extract (positions, indices) per submesh → `xatlas::AddMesh` → `xatlas::Generate` → for each output mesh, rebuild a single-binding VertexData copying every source attribute from `xref` (input vertex id) and overwriting the target UV channel with `xatlas::Vertex::uv / atlas.{width,height}`. Skinned-mesh bone assignments survive the seam splits because we rebuild `SubMesh::BoneAssignmentList` against the new vertex IDs via xref; for shared-vertex meshes the source assignments come from `Mesh::getBoneAssignments()`, not `SubMesh::getBoneAssignments()`. Surfaced via `qtmesh uv --unwrap`/`--info`, MCP `auto_uv_unwrap`, and the **Material Mode → Mode Tools → "Auto UV Unwrap…" button** (`qml/UvUnwrapDialog.qml`, driven by `UvUnwrapController` singleton). Sentry breadcrumb category `ai.assist.uv_unwrap`. The unwrap also erases `qtme.faces.<i>` n-gon bindings (they reference source vertex IDs and become stale). **GUI-safe entry point** (`unwrapEntityToFile`): live skinned meshes cannot survive in-place vertex-data mutation because the active `Ogre::SkeletonInstance` caches the hardware blend buffer and picks up stale state on the first frame after the swap. The GUI path snapshots `vertexData` / `indexData` / `mBoneAssignments` / `blendIndexToBoneIndexMap` for every submesh + the mesh's shared maps, calls `unwrapEntityKeepingOriginals` (which deliberately leaks its own allocations rather than freeing the originals), exports the unwrapped result, then restores the snapshot pointer-for-pointer (deleting only the unwrap's leaked allocations) and pastes the index maps back directly — `_compileBoneAssignments` is NOT called on restore because it would re-pack BLEND_INDICES/WEIGHTS bytes against the live buffer and shatter the on-screen mesh. CLI path uses the destructive `unwrapEntity` since the process exits before rendering.

0 commit comments

Comments
 (0)