Skip to content

Commit 887475f

Browse files
mhochstegermhochsteger
committed
Selection documentation
1 parent 83bb782 commit 887475f

1 file changed

Lines changed: 195 additions & 0 deletions

File tree

‎docs/selection.ipynb‎

Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "9af1d62c-cb0d-4d84-8932-840779c888cb",
6+
"metadata": {},
7+
"source": [
8+
"## How to use\n",
9+
"\n",
10+
"To enable selection (run a callback when you click on a rendered object) is enabled by two steps:\n",
11+
"\n",
12+
"1) Run `scene.select(x,y)` when the user clicks on the canvas.\n",
13+
"2) The scene will search the render_object that is drawn at the given pixel and triggers callbacks that are set via `render_object.on_select(callback)`. The callback gets one argument of type `webgpu.renderer.SelectEvent`.\n",
14+
"\n",
15+
"The example below demonstrates this by drawing two types of objects (cones and cylinders) and registers two different callbacks. The selection is done via click with left mouse button (`ev['button'] == 0`). A string with selected type and number is shown on the top left of the canvas using a `Labels` object."
16+
]
17+
},
18+
{
19+
"cell_type": "code",
20+
"execution_count": null,
21+
"id": "a46a44f6-ecba-4f29-98e9-1a4245e2d702",
22+
"metadata": {},
23+
"outputs": [],
24+
"source": [
25+
"import webgpu.jupyter as wj\n",
26+
"import numpy as np\n",
27+
"from webgpu.shapes import ShapeRenderer, generate_cylinder, generate_cone\n",
28+
"from webgpu import Colormap, Labels\n",
29+
"\n",
30+
"rand = np.random.random\n",
31+
"\n",
32+
"thickness = 1e-2 * 10\n",
33+
"length = 0.3\n",
34+
"subdivision = 8\n",
35+
"\n",
36+
"# Render two types of objects: cones and cylinders\n",
37+
"cylinder = generate_cylinder(subdivision, thickness)\n",
38+
"N = 10\n",
39+
"cmap = Colormap(0,1)\n",
40+
"cylinders = ShapeRenderer(cylinder, rand((N,3)), length * rand((N,3)), 0.5*rand(2*N), colormap=cmap)\n",
41+
"cone = generate_cone(subdivision, thickness)\n",
42+
"cones = ShapeRenderer(cone, rand((N,3)), length * rand((N,3)), 0.5+0.5*rand(2*N), colormap=cmap)\n",
43+
"\n",
44+
"label = Labels([\"\"], [[-1, -1]], font_size=16)\n",
45+
"\n",
46+
"scene = wj.Draw([cylinders, cones, label])\n",
47+
"\n",
48+
"def set_label(s):\n",
49+
" label.labels[0] = s\n",
50+
" label.set_needs_update()\n",
51+
" scene.render()\n",
52+
"\n",
53+
"# Register callbacks for selection\n",
54+
"def on_select_cone(ev):\n",
55+
" set_label(f\"Selected cone {ev.uint32[0]} with value {ev.float32[1]}\")\n",
56+
"\n",
57+
"cones.on_select(on_select_cone)\n",
58+
" \n",
59+
"def on_select_cyl(ev):\n",
60+
" set_label(f\"Selected cylinder {ev.uint32[0]} with value {ev.float32[1]}\")\n",
61+
"\n",
62+
"cylinders.on_select(on_select_cyl)\n",
63+
"\n",
64+
"def on_click(ev):\n",
65+
" if ev['button'] == 0: # left mouse button\n",
66+
" # This finds the render object which is at the cursor and calls the 'on_select' callback of the correct render object\n",
67+
" scene.select(ev['canvasX'], ev['canvasY'])\n",
68+
"\n",
69+
"# Call on_click when the user clicks on the canvas\n",
70+
"scene.input_handler.on_click(on_click)\n",
71+
"\n",
72+
" "
73+
]
74+
},
75+
{
76+
"cell_type": "markdown",
77+
"id": "d00b0068-4e83-479d-92d3-0bc93c21f299",
78+
"metadata": {},
79+
"source": [
80+
"## How does it work?\n",
81+
"\n",
82+
"Each render object that supports selection has two rendering pipelines, the \"usual\" one to render the object and a \"selection_pipeline\" to render information about the object encoded in the color of the resulting pixel. The selection pipeline has usually the same properties as the rendering pipeline, but a **different fragment shader**.\n",
83+
"\n",
84+
"The code below shows the input struct and the two different fragment shaders for the ShapeRenderer above. Note that the output of a selection shader must have type `vec4<u32>`, i.e. four unsigned 32 bit integers. To store floats, you can use `bitcast<u32>(float_value)` and interpret the binary buffer accordingly on the python side.\n",
85+
"\n",
86+
"```rust\n",
87+
"\n",
88+
"// Input structure for fragment shaders\n",
89+
"struct ShapeVertexOut {\n",
90+
" @builtin(position) position: vec4f,\n",
91+
" @location(0) p: vec3f,\n",
92+
" @location(1) normal: vec3f,\n",
93+
" @location(2) value: vec4f,\n",
94+
" @location(3) @interpolate(flat) instance: u32,\n",
95+
"};\n",
96+
"\n",
97+
"// Default fragment shader\n",
98+
"@fragment fn shape_fragment_main_value(\n",
99+
" input: ShapeVertexOut,\n",
100+
") -> @location(0) vec4f {\n",
101+
" let color = getColor(input.value.x);\n",
102+
" return lightCalcColor(input.p, input.normal, color);\n",
103+
"}\n",
104+
"\n",
105+
"// Selection fragment shader\n",
106+
"@fragment fn shape_fragment_main_select(\n",
107+
" input: ShapeVertexOut,\n",
108+
") -> @location(0) vec4<u32> {\n",
109+
" // @RENDER_OBJECT_ID@ has to be first here, that's used by the scene to know which render_object was drawn at this pixel\n",
110+
" // Write the instance (i.e. which cylinder is this) and the function value (must be casted to u32, well be casted back to float32 in python)\n",
111+
" return vec4<u32>(@RENDER_OBJECT_ID@, input.instance, bitcast<u32>(input.value.x), 0);\n",
112+
"}\n",
113+
"\n",
114+
"```\n"
115+
]
116+
},
117+
{
118+
"cell_type": "markdown",
119+
"id": "c8115176-5b9c-4fb0-ab7a-b2889b3eb07f",
120+
"metadata": {},
121+
"source": [
122+
"## Change the behavior\n",
123+
"\n",
124+
"The example below creates a custom class `MyShapeRenderer` which is derived from `ShapeRenderer` and changes only the fragment shader code used for selection. Now we are storing the original position at the mouse cursor instead."
125+
]
126+
},
127+
{
128+
"cell_type": "code",
129+
"execution_count": null,
130+
"id": "56674943-895e-46ec-80cd-2c1c2fbf6b79",
131+
"metadata": {},
132+
"outputs": [],
133+
"source": [
134+
"select_shader = \"\"\"\n",
135+
"@fragment fn my_select_shader(\n",
136+
" input: ShapeVertexOut,\n",
137+
") -> @location(0) vec4<u32> {\n",
138+
" // Encode the current position int the last three values as unsigned integers\n",
139+
" return vec4<u32>(@RENDER_OBJECT_ID@, bitcast<vec3<u32>>(input.p));\n",
140+
"}\n",
141+
"\"\"\"\n",
142+
"\n",
143+
"class MyShapeRenderer(ShapeRenderer):\n",
144+
" select_entry_point = \"my_select_shader\"\n",
145+
" \n",
146+
" def get_shader_code(self):\n",
147+
" return super().get_shader_code() + select_shader\n",
148+
"\n",
149+
"\n",
150+
"mycylinders = MyShapeRenderer(cylinder, rand((N,3)), rand((N,3)), rand(2*N))\n",
151+
"mylabel = Labels([\"\"], [[-1, -.8]], font_size=14)\n",
152+
"\n",
153+
"def on_select_mycyl(ev):\n",
154+
" mylabel.labels[0] = f\"Selected mycylinder at point {ev.float32}\"\n",
155+
" mylabel.set_needs_update()\n",
156+
" myscene.render()\n",
157+
"\n",
158+
"mycylinders.on_select(on_select_mycyl)\n",
159+
"\n",
160+
"myscene = wj.Draw([mycylinders, mylabel])\n",
161+
" \n",
162+
"myscene.input_handler.on_click(lambda ev: myscene.select(ev['canvasX'], ev['canvasY']))"
163+
]
164+
},
165+
{
166+
"cell_type": "code",
167+
"execution_count": null,
168+
"id": "aa24b33e-1897-4072-80ff-153f4fc723b4",
169+
"metadata": {},
170+
"outputs": [],
171+
"source": []
172+
}
173+
],
174+
"metadata": {
175+
"kernelspec": {
176+
"display_name": "Python 3 (ipykernel)",
177+
"language": "python",
178+
"name": "python3"
179+
},
180+
"language_info": {
181+
"codemirror_mode": {
182+
"name": "ipython",
183+
"version": 3
184+
},
185+
"file_extension": ".py",
186+
"mimetype": "text/x-python",
187+
"name": "python",
188+
"nbconvert_exporter": "python",
189+
"pygments_lexer": "ipython3",
190+
"version": "3.13.3"
191+
}
192+
},
193+
"nbformat": 4,
194+
"nbformat_minor": 5
195+
}

0 commit comments

Comments
 (0)