Skip to content

Commit 5991f8a

Browse files
JimBobSquarePantsJimBobSquarePants
committed
Update tolerances
1 parent 6e0b36c commit 5991f8a

3 files changed

Lines changed: 412 additions & 9 deletions

File tree

‎src/ImageSharp.Drawing/Processing/Backends/MESH_GRADIENT_TYPES.md‎

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
# Mesh Gradient Type Sketch
2+
3+
This note sketches the small public surface needed to express mesh-style gradients with the existing canvas and backend pipeline.
4+
5+
## Design Decision
6+
7+
Do not add a single `MeshGradientBrush` for the triangulated case.
8+
9+
A triangulated mesh gradient can be expressed as multiple existing fills:
10+
11+
- `Polygon` provides each triangle's geometry.
12+
- `PathGradientBrush` provides each triangle's three point/color pairs.
13+
- `Path.ToLinearGeometry(Vector2 scale)` keeps geometry memoization through the existing `LinearGeometryCache`.
14+
- CPU and WebGPU already render triangle-shaped `PathGradientBrush` fills.
15+
16+
This keeps the feature as a canvas convenience over existing primitives rather than a new backend concept.
17+
18+
## Public Types
19+
20+
```csharp
21+
namespace SixLabors.ImageSharp.Drawing.Processing;
22+
23+
public readonly struct CubicGradientPatch
24+
{
25+
public CubicGradientPatch(
26+
ReadOnlySpan<PointF> cubicControlPoints,
27+
ReadOnlySpan<Color> cornerColors);
28+
29+
public PointF[] CubicControlPoints { get; }
30+
31+
public Color[] CornerColors { get; }
32+
}
33+
34+
public enum MeshPrimitiveMode
35+
{
36+
Triangles,
37+
TriangleStrip
38+
}
39+
40+
public readonly struct MeshVertex
41+
{
42+
public MeshVertex(PointF position, Color color);
43+
44+
public PointF Position { get; }
45+
46+
public Color Color { get; }
47+
}
48+
49+
public sealed class DrawingMesh
50+
{
51+
private readonly MeshVertex[] vertices;
52+
private readonly ushort[] indices;
53+
54+
public DrawingMesh(
55+
ReadOnlySpan<MeshVertex> vertices,
56+
ReadOnlySpan<ushort> indices,
57+
MeshPrimitiveMode primitiveMode);
58+
59+
public ReadOnlySpan<MeshVertex> Vertices => this.vertices;
60+
61+
public ReadOnlySpan<ushort> Indices => this.indices;
62+
63+
public MeshPrimitiveMode PrimitiveMode { get; }
64+
}
65+
```
66+
67+
## Canvas Surface
68+
69+
These should be first-class canvas methods on `DrawingCanvas` and implemented by `DrawingCanvas<TPixel>`.
70+
71+
```csharp
72+
namespace SixLabors.ImageSharp.Drawing.Processing;
73+
74+
public abstract class DrawingCanvas : IDisposable
75+
{
76+
public abstract void FillPatch(
77+
CubicGradientPatch patch,
78+
GraphicsOptions? options = null);
79+
80+
public abstract void DrawMesh(
81+
DrawingMesh mesh,
82+
GraphicsOptions? options = null);
83+
}
84+
```
85+
86+
## Lowering
87+
88+
`DrawMesh(...)` expands the submitted mesh into ordinary triangle fills before the backend sees the scene.
89+
90+
`MeshPrimitiveMode` selects the index traversal:
91+
92+
```csharp
93+
switch (mesh.PrimitiveMode)
94+
{
95+
case MeshPrimitiveMode.Triangles:
96+
this.FillIndexedTriangles(mesh);
97+
break;
98+
case MeshPrimitiveMode.TriangleStrip:
99+
this.FillTriangleStrip(mesh);
100+
break;
101+
}
102+
```
103+
104+
`Triangles` consumes the index buffer in independent groups of three:
105+
106+
```csharp
107+
private void FillIndexedTriangles(DrawingMesh mesh)
108+
{
109+
for (int i = 0; i < mesh.Indices.Length; i += 3)
110+
{
111+
MeshVertex v0 = mesh.Vertices[mesh.Indices[i]];
112+
MeshVertex v1 = mesh.Vertices[mesh.Indices[i + 1]];
113+
MeshVertex v2 = mesh.Vertices[mesh.Indices[i + 2]];
114+
115+
this.FillTriangle(v0, v1, v2);
116+
}
117+
}
118+
```
119+
120+
`TriangleStrip` consumes one new vertex per triangle and flips the first two vertices on alternating triangles so the winding remains consistent:
121+
122+
```csharp
123+
private void FillTriangleStrip(DrawingMesh mesh)
124+
{
125+
for (int i = 0; i <= mesh.Indices.Length - 3; i++)
126+
{
127+
int i0 = mesh.Indices[i];
128+
int i1 = mesh.Indices[i + 1];
129+
int i2 = mesh.Indices[i + 2];
130+
131+
if ((i & 1) != 0)
132+
{
133+
(i0, i1) = (i1, i0);
134+
}
135+
136+
this.FillTriangle(mesh.Vertices[i0], mesh.Vertices[i1], mesh.Vertices[i2]);
137+
}
138+
}
139+
```
140+
141+
Both modes should share one helper:
142+
143+
```csharp
144+
private void FillTriangle(MeshVertex v0, MeshVertex v1, MeshVertex v2)
145+
{
146+
PointF[] points =
147+
[
148+
v0.Position,
149+
v1.Position,
150+
v2.Position
151+
];
152+
153+
Color[] colors =
154+
[
155+
v0.Color,
156+
v1.Color,
157+
v2.Color
158+
];
159+
160+
this.Fill(new PathGradientBrush(points, colors), new Polygon(points));
161+
}
162+
```
163+
164+
`FillPatch(...)` samples the cubic patch into generated mesh vertices, emits two triangles per sampled cell, then lowers those triangles through the same `Polygon` plus `PathGradientBrush` path.
165+
166+
## Backend Impact
167+
168+
No new backend scene command is required for the first implementation. The canvas methods emit normal fill commands, so the existing path remains:
169+
170+
```text
171+
Polygon + PathGradientBrush
172+
CompositionCommand
173+
FlushScene / WebGPUSceneEncoder
174+
DefaultRasterizer / WebGPU fine shader
175+
BrushRenderer / path-gradient WGSL
176+
```
177+
178+
The color interpolation is already implemented by `PathGradientBrushRenderer<TPixel>.FindPointOnTriangle(...)` on CPU and `path_grad_point_on_triangle(...)` in WebGPU.
179+
180+
## Missing From This Design
181+
182+
- General Skia `SkMesh` support with custom vertex and fragment shader programs.
183+
- Arbitrary vertex attributes or varyings beyond position and color.
184+
- Texture coordinates and child shader sampling.
185+
- GPU-resident mesh vertex and index buffers.
186+
- Runtime shader mesh-gradient modes such as inverse-distance or n-linear point-field shading.
187+
- Seam-free rendering guarantees across shared triangle edges.
188+
- A single mesh primitive submitted to the backend as one retained drawable.
189+
- Patch quality controls such as adaptive tessellation or caller-selected subdivision density.
190+
- Non-triangle mesh topologies beyond indexed triangles and triangle strips.

‎src/ImageSharp.Drawing/Processing/Backends/SCENE_REUSE_DESIGN.md‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# Reusable Backend Scene
2+
3+
## Goal
4+
5+
Canvas drawing should have one execution model.
6+
7+
`DrawingCanvas` records an ordered timeline. Disposing the root canvas renders that timeline once. A retained scene is a backend payload created from prepared draw commands; inserting that scene into a later canvas adds it to that later canvas timeline.
8+
9+
There is no separate immediate renderer. The non-retained path creates short-lived backend scenes during disposal, renders them once, and disposes them.
10+
11+
## Core Model
12+
13+
The canvas is the timeline container.
14+
15+
Backend scenes stay light:
16+
17+
- `DefaultDrawingBackendScene` owns CPU retained draw data.
18+
- `WebGPUDrawingBackendScene` owns WebGPU encoded draw data and reusable render arenas.
19+
- `DrawingBackendScene` is the public retained scene base.
20+
21+
Backend scenes do not contain the canvas timeline. They do not store apply barriers. They do not store child scenes.
22+
23+
## Terms
24+
25+
- **Draw command**: an existing canvas operation that becomes backend drawing work, for example fill, stroke, text, image draw, or layer markers.
26+
- **Command range**: a contiguous run of draw commands sealed into the canvas timeline.
27+
- **Backend scene**: a retained payload created by a backend from one prepared `DrawingCommandBatch`.
28+
- **Inserted scene**: an existing `DrawingBackendScene` recorded through `DrawingCanvas.RenderScene(...)`.
29+
- **Apply barrier**: a target-dependent timeline entry that reads the target, runs ImageSharp processors, and writes the processed snapshot back.
30+
- **Flush barrier**: a command-range boundary created by `DrawingCanvas.Flush()`. It renders nothing by itself.
31+
32+
## Canvas Pipeline
33+
34+
`DrawingCanvas` remains the public recording API.
35+
36+
The pipeline is:
37+
38+
1. Draw operations append commands to `DrawingCanvasBatcher<TPixel>`.
39+
2. `Flush()` seals the current commands into a command-range timeline entry.
40+
3. `Apply(...)` seals current commands and appends an apply barrier.
41+
4. `RenderScene(...)` seals current commands and appends an existing retained scene reference.
42+
5. `Dispose()` restores open canvas state, seals and prepares commands, then replays the timeline once.
43+
44+
During disposal replay:
45+
46+
- command-range entries become short-lived backend scenes through `IDrawingBackend.CreateScene(...)`
47+
- apply-barrier entries create a transient write-back command batch and render it through the same backend scene path
48+
- inserted-scene entries call `IDrawingBackend.RenderScene(...)` on the retained scene supplied by the caller
49+
50+
`RenderScene(...)` is not a target write by itself. It records a retained scene reference at the current timeline position. The scene is rendered only when the canvas containing that entry is disposed.
51+
52+
`Flush()` is also not a target write by itself. It only makes the current command range explicit so later timeline entries cannot merge across that boundary.
53+
54+
```csharp
55+
image.Mutate(x => x.Paint(canvas =>
56+
{
57+
// These commands become the first command-range entry.
58+
canvas.Fill(backgroundBrush, backgroundPath);
59+
60+
// Apply seals the first range and records a read/process/write barrier.
61+
canvas.Apply(effectPath, ctx => ctx.GaussianBlur(6F));
62+
63+
// These commands become a later command-range entry.
64+
canvas.Draw(foregroundPen, foregroundPath);
65+
}));
66+
```
67+
68+
The callback above renders once during canvas disposal.
69+
70+
## Backend Contract
71+
72+
Scene creation is untyped and target-frame free. It only needs the active configuration, target bounds, prepared draw commands, and any resources that must remain alive with the scene.
73+
74+
```csharp
75+
public interface IDrawingBackend
76+
{
77+
public DrawingBackendScene CreateScene(
78+
Configuration configuration,
79+
Rectangle targetBounds,
80+
DrawingCommandBatch commandBatch,
81+
IReadOnlyList<IDisposable>? ownedResources = null);
82+
83+
public void RenderScene<TPixel>(
84+
Configuration configuration,
85+
ICanvasFrame<TPixel> target,
86+
DrawingBackendScene scene)
87+
where TPixel : unmanaged, IPixel<TPixel>;
88+
89+
public void ReadRegion<TPixel>(
90+
Configuration configuration,
91+
ICanvasFrame<TPixel> target,
92+
Rectangle sourceRectangle,
93+
Buffer2DRegion<TPixel> destination)
94+
where TPixel : unmanaged, IPixel<TPixel>;
95+
}
96+
```
97+
98+
`CreateScene(...)` does not validate `TPixel`, because there is no `TPixel` at scene creation time.
99+
100+
`RenderScene<TPixel>(...)` is the typed target boundary. Backends validate the retained scene type, target capabilities, target bounds, and pixel format requirements there.
101+
102+
`ReadRegion<TPixel>(...)` is the typed readback boundary. Apply barriers use it to copy target pixels into a temporary `Image<TPixel>`.
103+
104+
## DrawingCommandBatch
105+
106+
`DrawingCommandBatch` is the backend handoff for one contiguous command range.
107+
108+
It contains:
109+
110+
- the prepared `CompositionSceneCommand` list
111+
- the command count
112+
- whether the range contains layer boundary commands
113+
114+
One command batch creates one backend scene.
115+
116+
## Apply Barrier
117+
118+
`ApplyBarrier` is shared canvas timeline data, not CPU or WebGPU scene data.
119+
120+
It stores the recorded processor operation and the geometry/state needed to reproduce the readback region at replay time:
121+
122+
- path
123+
- drawing options
124+
- clip paths
125+
- canvas bounds
126+
- target bounds
127+
- destination offset
128+
- layer state
129+
- processor delegate
130+
131+
When disposal replay reaches an apply barrier:
132+
133+
1. The barrier computes the target-local source rectangle.
134+
2. The backend reads that region into a temporary `Image<TPixel>`.
135+
3. The processor delegate runs against the temporary image.
136+
4. The barrier creates a transient image-brush draw command.
137+
5. The canvas renders that command batch through `CreateScene(...)` and `RenderScene(...)`.
138+
6. The temporary image is disposed.
139+
140+
The processed image is per-render temporary state. A retained backend scene never stores pixels produced by an apply barrier.
141+
142+
## CPU Scene
143+
144+
`DefaultDrawingBackendScene` stores a CPU `FlushScene`.
145+
146+
`DefaultDrawingBackend.CreateScene(...)` lowers a prepared command batch into `FlushScene` using target bounds and the active configuration. It does not need the target frame and does not require a `TPixel`.
147+
148+
`DefaultDrawingBackend.RenderScene<TPixel>(...)` acquires the CPU destination region from the target, validates the retained scene type and bounds, and executes the `FlushScene` into that destination.
149+
150+
## WebGPU Scene
151+
152+
`WebGPUDrawingBackendScene` stores a `WebGPUEncodedScene`.
153+
154+
It also owns:
155+
156+
- one reusable resource arena slot
157+
- one reusable scheduling arena slot
158+
- the retained scratch-size budget discovered across renders
159+
- the backend that should receive cached arenas on disposal
160+
161+
`WebGPUDrawingBackend.CreateScene(...)` only encodes the command batch. It uses target bounds and the allocator, but it does not create a `WebGPUFlushContext`, does not validate `TPixel`, and does not store the target texture format.
162+
163+
`WebGPUDrawingBackend.RenderScene<TPixel>(...)` owns the render-scoped WebGPU work:
164+
165+
1. Resolve `TPixel` to the supported WebGPU texture format and required feature.
166+
2. Validate that the target exposes a native WebGPU surface with matching bounds and device support.
167+
3. Rent the retained scene arenas or backend arenas.
168+
4. Create the staged scene resources for the current render.
169+
5. Dispatch the staged GPU pipeline.
170+
6. Update retained scratch sizes.
171+
7. Return arenas to the scene or backend cache.
172+
173+
The retained WebGPU scene does not store texture format or required feature. Those belong to the typed render target boundary.
174+
175+
## Pixel Format Rule
176+
177+
`WebGPUTextureFormat` is the public closed set of supported WebGPU formats. Internal code does not repeatedly validate it.
178+
179+
`TPixel` is validated only when typed pixels are introduced:
180+
181+
- `RenderScene<TPixel>(...)`, when rendering to an `ICanvasFrame<TPixel>`
182+
- `ReadRegion<TPixel>(...)`, when copying target pixels into CPU memory
183+
- public typed readback helpers, through the backend readback path
184+
185+
## Ownership
186+
187+
Pending image resources move into the backend scene that references them.
188+
189+
Apply snapshots are not retained. They are created and disposed during barrier execution.
190+
191+
The retained scene owns:
192+
193+
- its backend payload
194+
- resources captured by that payload
195+
- reusable backend arenas or scratch state
196+
197+
Disposing the retained scene releases its backend payload and owned resources.
198+
199+
## Behavioral Summary
200+
201+
- `Dispose()` is the canvas operation that writes the recorded timeline to the target.
202+
- `Flush()` seals pending commands into the recorded timeline and renders nothing.
203+
- `Apply(...)` records a replay-time barrier and renders nothing immediately.
204+
- `RenderScene(...)` records an existing retained scene reference and renders nothing immediately.
205+
- `CreateScene(...)` creates a retained backend payload from prepared draw commands and renders nothing.
206+
- `RenderScene<TPixel>(...)` is the typed target write boundary.
207+
- One `DrawingCommandBatch` creates one backend scene.
208+
- The canvas timeline preserves command ranges, apply barriers, and inserted retained scenes in recorded order.

0 commit comments

Comments
 (0)