Merge pull request #1053 from zeux/idxfilter

Implement index filtering for redundant triangles

Author: zeux

README.md

@@ -42,7 +42,8 @@ When optimizing a mesh, to maximize rendering efficiency you should typically fe
 3. (optional) Overdraw optimization
 4. Vertex fetch optimization
 5. Vertex quantization
-6. (optional) Shadow indexing
+6. Index filtering
+7. (optional) Shadow indexing
 
 ### Indexing
 
@@ -145,6 +146,18 @@ unsigned short pz = meshopt_quantizeHalf(v.z);
 
 Since quantized vertex attributes often need to remain in their compact representations for efficient transfer and storage, they are usually dequantized during vertex processing by configuring the GPU vertex input correctly to expect normalized integers or half precision floats, which often needs no or minimal changes to the shader code. When CPU dequantization is required instead, `meshopt_dequantizeHalf` can be used to convert half precision values back to single precision; for normalized integer formats, the dequantization just requires dividing by 2^N-1 for unorm and 2^(N-1)-1 for snorm variants. For example, manually reversing `meshopt_quantizeUnorm(v, 10)` can be done by dividing by 1023.
 
+### Index filtering
+
+Some meshes may contain triangles that are processed during rendering but do not contribute to the rendered result. If any two vertices of a triangle result in the same position after vertex shader, the triangle is degenerate and will be skipped by the rasterizer. Some triangles may also be duplicates of an earlier triangle with the same post-transform positions and winding, in which case only one of the triangles will be visible depending on depth testing settings (assuming blending is disabled). In either case, such triangles require extra processing and removing them may improve rasterization or ray tracing performance; this library provides an algorithm that removes such triangles from the index buffer:
+
+```c++
+indices.resize(meshopt_filterIndexBuffer(&indices[0], &indices[0], indices.size(), &vertices[0].x, vertices.size(), sizeof(float) * 3, sizeof(Vertex)));
+```
+
+Note that the example above assumes only positions are relevant for transforming the vertices, but for deformable meshes skinning data may need to be added to the vertex portion used as a key; `meshopt_filterIndexBufferMulti` can be useful for these cases if the relevant data is not contiguous.
+
+Filtering after quantization is convenient because quantization may increase the number of redundant triangles if triangles had similar but not identical vertex positions before quantization. However, filtering can be done at any point in the pipeline as soon as the index buffer becomes available; you could also run vertex fetch optimization after filtering, since it will naturally filter out any vertices that may become unused after redundant triangles are eliminated, potentially saving extra memory.
+
 ### Shadow indexing
 
 Many rendering pipelines require meshes to be rendered to depth-only targets, such as shadow maps or during a depth pre-pass, in addition to color/G-buffer targets. While using the same geometry data for both cases is possible, reducing the number of unique vertices for depth-only rendering can be beneficial, especially when the source geometry has many attribute seams due to faceted shading or lightmap texture seams.
@@ -923,6 +936,7 @@ Currently, the following APIs are experimental:
 - `meshopt_computePositionExponent` function
 - `meshopt_opacityMap*` functions (`meshopt_opacityMapMeasure`, `meshopt_opacityMapRasterize`, `meshopt_opacityMapCompact`, `meshopt_opacityMapEntrySize`)
 - `meshopt_generateTangents` function and `meshopt_Tangent*` flags
+- `meshopt_filterIndexBuffer` and `meshopt_filterIndexBufferMulti` functions
 
 ## License
 

demo/main.cpp

@@ -1484,6 +1484,21 @@ void spatialClusterPoints(const Mesh& mesh, size_t cluster_size)
 	    (end - start) * 1000);
 }
 
+void filterTriangles(const Mesh& mesh)
+{
+	double start = timestamp();
+
+	std::vector<unsigned int> newindices(mesh.indices.size());
+	newindices.resize(meshopt_filterIndexBuffer(&newindices[0], &mesh.indices[0], mesh.indices.size(), &mesh.vertices[0], mesh.vertices.size(), sizeof(float) * 3, sizeof(Vertex)));
+
+	double end = timestamp();
+
+	printf("FilterIB : %d triangles -> %d (%.2f%% redundant) in %.2f msec\n",
+	    int(mesh.indices.size() / 3), int(newindices.size() / 3),
+	    double((mesh.indices.size() - newindices.size()) / 3) / double(mesh.indices.size() / 3) * 100.0,
+	    (end - start) * 1000);
+}
+
 void tessellationAdjacency(const Mesh& mesh)
 {
 	double start = timestamp();
@@ -1814,6 +1829,7 @@ void process(const char* path)
 	meshlets(copy, /* scan= */ false, /* uniform= */ true, /* flex= */ false, /* spatial= */ true);
 
 	shadow(copy);
+	filterTriangles(copy);
 	tessellationAdjacency(copy);
 	provoking(copy);
 

demo/tests.cpp

@@ -2555,6 +2555,44 @@ static void simplifyUpdateLocked(unsigned int options)
 	assert(vb[3][3] == 0.2f);
 }
 
+static void filterTriangles()
+{
+	// v0/v3 match fully; v0/v4 match on prefix only
+	const unsigned int vb[] = {
+	    1, 0, 10,
+	    2, 0, 20,
+	    3, 0, 30,
+	    1, 1, 10,
+	    1, 1, 99 //
+	};
+
+	const unsigned int ib[] = {
+	    0, 1, 2, //
+	    3, 1, 2, // dup of (0,1,2)
+	    4, 1, 2, // dup prefix
+	    2, 1, 0, // opposite winding of (0,1,2)
+	    0, 4, 1  // degen prefix
+	};
+
+	// prefix only: vertex key is first 4 bytes
+	unsigned int sib[15];
+	size_t slen = meshopt_filterIndexBuffer(sib, ib, 15, vb, 5, 4, 12);
+
+	unsigned int expecteds[] = {0, 1, 2, 2, 1, 0};
+	assert(slen == sizeof(expecteds) / sizeof(expecteds[0]));
+	assert(memcmp(sib, expecteds, sizeof(expecteds)) == 0);
+
+	// prefix+suffix: vertex key is first 4 and last 4 bytes
+	const meshopt_Stream streams[] = {{vb + 0, 4, 12}, {vb + 2, 4, 12}};
+
+	unsigned int mib[15];
+	size_t mlen = meshopt_filterIndexBufferMulti(mib, ib, 15, 5, streams, 2);
+	unsigned int expectedm[] = {0, 1, 2, 4, 1, 2, 2, 1, 0, 0, 4, 1};
+
+	assert(mlen == sizeof(expectedm) / sizeof(expectedm[0]));
+	assert(memcmp(mib, expectedm, sizeof(expectedm)) == 0);
+}
+
 static void adjacency()
 {
 	// 0 1/4
@@ -3399,6 +3437,7 @@ void runTests()
 	simplifyUpdateLocked(0);
 	simplifyUpdateLocked(meshopt_SimplifySparse);
 
+	filterTriangles();
 	adjacency();
 	tessellation();
 	provoking();

src/indexgenerator.cpp

@@ -127,6 +127,30 @@ struct VertexCustomHasher
 	}
 };
 
+struct TriangleKey
+{
+	unsigned int a, b, c;
+
+	bool operator==(const TriangleKey& other) const
+	{
+		return a == other.a && b == other.b && c == other.c;
+	}
+};
+
+struct TriangleKeyHasher
+{
+	size_t hash(const TriangleKey& k) const
+	{
+		// Optimized Spatial Hashing for Collision Detection of Deformable Objects
+		return (k.a * 73856093) ^ (k.b * 19349663) ^ (k.c * 83492791);
+	}
+
+	bool equal(const TriangleKey& lhs, const TriangleKey& rhs) const
+	{
+		return lhs == rhs;
+	}
+};
+
 struct EdgeHasher
 {
 	const unsigned int* remap;
@@ -304,6 +328,63 @@ static void generateShadowBuffer(unsigned int* destination, const unsigned int*
 	}
 }
 
+static size_t filterIndexBuffer(unsigned int* destination, const unsigned int* indices, size_t index_count, const unsigned int* remap, size_t vertex_count, meshopt_Allocator& allocator)
+{
+	(void)vertex_count;
+
+	TriangleKeyHasher hasher = {};
+	TriangleKey empty = {~0u, ~0u, ~0u};
+
+	size_t face_count = index_count / 3;
+	size_t table_size = hashBuckets(face_count);
+	TriangleKey* table = allocator.allocate<TriangleKey>(table_size);
+	memset(table, -1, table_size * sizeof(TriangleKey));
+
+	size_t write = 0;
+
+	for (size_t i = 0; i < face_count; ++i)
+	{
+		unsigned int a = indices[i * 3 + 0], b = indices[i * 3 + 1], c = indices[i * 3 + 2];
+		assert(a < vertex_count && b < vertex_count && c < vertex_count);
+
+		unsigned int ra = remap[a], rb = remap[b], rc = remap[c];
+
+		// degenerate triangle
+		if (ra == rb || ra == rc || rb == rc)
+			continue;
+
+		// canonicalize triangle wrt winding preserving rotation
+		if (rb < ra && rb < rc)
+		{
+			// abc -> bca
+			unsigned int t = ra;
+			ra = rb, rb = rc, rc = t;
+		}
+		else if (rc < ra && rc < rb)
+		{
+			// abc -> cab
+			unsigned int t = rc;
+			rc = rb, rb = ra, ra = t;
+		}
+
+		TriangleKey key = {ra, rb, rc};
+		TriangleKey* entry = hashLookup(table, table_size, hasher, key, empty);
+
+		// duplicate triangle
+		if (entry->a != ~0u)
+			continue;
+
+		*entry = key;
+
+		destination[write + 0] = a;
+		destination[write + 1] = b;
+		destination[write + 2] = c;
+		write += 3;
+	}
+
+	return write;
+}
+
 } // namespace meshopt
 
 size_t meshopt_generateVertexRemap(unsigned int* destination, const unsigned int* indices, size_t index_count, const void* vertices, size_t vertex_count, size_t vertex_size)
@@ -399,11 +480,49 @@ void meshopt_remapIndexBuffer(unsigned int* destination, const unsigned int* ind
 	}
 }
 
+size_t meshopt_filterIndexBuffer(unsigned int* destination, const unsigned int* indices, size_t index_count, const void* vertices, size_t vertex_count, size_t vertex_size, size_t vertex_stride)
+{
+	using namespace meshopt;
+
+	assert(index_count % 3 == 0);
+	assert(vertex_size > 0 && vertex_size <= 256);
+	assert(vertex_size <= vertex_stride);
+
+	meshopt_Allocator allocator;
+	VertexHasher hasher = {static_cast<const unsigned char*>(vertices), vertex_size, vertex_stride};
+
+	unsigned int* remap = allocator.allocate<unsigned int>(vertex_count);
+	generateVertexRemap(remap, indices, index_count, vertex_count, hasher, allocator);
+
+	return filterIndexBuffer(destination, indices, index_count, remap, vertex_count, allocator);
+}
+
+size_t meshopt_filterIndexBufferMulti(unsigned int* destination, const unsigned int* indices, size_t index_count, size_t vertex_count, const struct meshopt_Stream* streams, size_t stream_count)
+{
+	using namespace meshopt;
+
+	assert(index_count % 3 == 0);
+	assert(stream_count > 0 && stream_count <= 16);
+
+	for (size_t i = 0; i < stream_count; ++i)
+	{
+		assert(streams[i].size > 0 && streams[i].size <= 256);
+		assert(streams[i].size <= streams[i].stride);
+	}
+
+	meshopt_Allocator allocator;
+	VertexStreamHasher hasher = {streams, stream_count};
+
+	unsigned int* remap = allocator.allocate<unsigned int>(vertex_count);
+	generateVertexRemap(remap, indices, index_count, vertex_count, hasher, allocator);
+
+	return filterIndexBuffer(destination, indices, index_count, remap, vertex_count, allocator);
+}
+
 void meshopt_generateShadowIndexBuffer(unsigned int* destination, const unsigned int* indices, size_t index_count, const void* vertices, size_t vertex_count, size_t vertex_size, size_t vertex_stride)
 {
 	using namespace meshopt;
 
-	assert(indices);
 	assert(index_count % 3 == 0);
 	assert(vertex_size > 0 && vertex_size <= 256);
 	assert(vertex_size <= vertex_stride);
@@ -418,7 +537,6 @@ void meshopt_generateShadowIndexBufferMulti(unsigned int* destination, const uns
 {
 	using namespace meshopt;
 
-	assert(indices);
 	assert(index_count % 3 == 0);
 	assert(stream_count > 0 && stream_count <= 16);
 

src/meshoptimizer.h

@@ -103,6 +103,27 @@ MESHOPTIMIZER_API void meshopt_remapVertexBuffer(void* destination, const void*
  */
 MESHOPTIMIZER_API void meshopt_remapIndexBuffer(unsigned int* destination, const unsigned int* indices, size_t index_count, const unsigned int* remap);
 
+/**
+ * Experimental: Filter out redundant triangles from the index buffer and return the number of remaining indices
+ * Triangles are considered redundant if they are degenerate (two vertices have the same vertex key) or duplicate (matching triangle was present earlier).
+ * First vertex_size bytes of every vertex are compared for equality; typically vertex_size should be set to the size of the position attribute.
+ * Note that duplicate triangles with opposite windings are preserved, as they may be needed for double-sided rendering.
+ *
+ * destination must contain enough space for the resulting index buffer (index_count elements)
+ */
+MESHOPTIMIZER_EXPERIMENTAL size_t meshopt_filterIndexBuffer(unsigned int* destination, const unsigned int* indices, size_t index_count, const void* vertices, size_t vertex_count, size_t vertex_size, size_t vertex_stride);
+
+/**
+ * Experimental: Filter out redundant triangles from the index buffer and return the number of remaining indices
+ * Triangles are considered redundant if they are degenerate (two vertices have the same vertex key) or duplicate (matching triangle was present earlier).
+ * All bytes in specified streams are compared for equality; streams should include attributes relevant for position transform (e.g. bone influences).
+ * Note that duplicate triangles with opposite windings are preserved, as they may be needed for double-sided rendering.
+ *
+ * destination must contain enough space for the resulting index buffer (index_count elements)
+ * stream_count must be <= 16
+ */
+MESHOPTIMIZER_EXPERIMENTAL size_t meshopt_filterIndexBufferMulti(unsigned int* destination, const unsigned int* indices, size_t index_count, size_t vertex_count, const struct meshopt_Stream* streams, size_t stream_count);
+
 /**
  * Generate index buffer that can be used for more efficient rendering when only a subset of the vertex attributes is necessary
  * All vertices that are binary equivalent (wrt first vertex_size bytes) map to the first vertex in the original vertex buffer.
@@ -1012,6 +1033,10 @@ inline size_t meshopt_generateVertexRemapCustom(unsigned int* destination, const
 template <typename T>
 inline void meshopt_remapIndexBuffer(T* destination, const T* indices, size_t index_count, const unsigned int* remap);
 template <typename T>
+inline size_t meshopt_filterIndexBuffer(T* destination, const T* indices, size_t index_count, const void* vertices, size_t vertex_count, size_t vertex_size, size_t vertex_stride);
+template <typename T>
+inline size_t meshopt_filterIndexBufferMulti(T* destination, const T* indices, size_t index_count, size_t vertex_count, const struct meshopt_Stream* streams, size_t stream_count);
+template <typename T>
 inline void meshopt_generateShadowIndexBuffer(T* destination, const T* indices, size_t index_count, const void* vertices, size_t vertex_count, size_t vertex_size, size_t vertex_stride);
 template <typename T>
 inline void meshopt_generateShadowIndexBufferMulti(T* destination, const T* indices, size_t index_count, size_t vertex_count, const meshopt_Stream* streams, size_t stream_count);
@@ -1266,6 +1291,24 @@ inline void meshopt_remapIndexBuffer(T* destination, const T* indices, size_t in
 	meshopt_remapIndexBuffer(out.data, indices ? in.data : NULL, index_count, remap);
 }
 
+template <typename T>
+inline size_t meshopt_filterIndexBuffer(T* destination, const T* indices, size_t index_count, const void* vertices, size_t vertex_count, size_t vertex_size, size_t vertex_stride)
+{
+	meshopt_IndexAdapter<T> in(NULL, indices, index_count);
+	meshopt_IndexAdapter<T> out(destination, NULL, index_count);
+
+	return meshopt_filterIndexBuffer(out.data, in.data, index_count, vertices, vertex_count, vertex_size, vertex_stride);
+}
+
+template <typename T>
+inline size_t meshopt_filterIndexBufferMulti(T* destination, const T* indices, size_t index_count, size_t vertex_count, const struct meshopt_Stream* streams, size_t stream_count)
+{
+	meshopt_IndexAdapter<T> in(NULL, indices, index_count);
+	meshopt_IndexAdapter<T> out(destination, NULL, index_count);
+
+	return meshopt_filterIndexBufferMulti(out.data, in.data, index_count, vertex_count, streams, stream_count);
+}
+
 template <typename T>
 inline void meshopt_generateShadowIndexBuffer(T* destination, const T* indices, size_t index_count, const void* vertices, size_t vertex_count, size_t vertex_size, size_t vertex_stride)
 {