Skip to content

Commit 67f3410

Browse files
akoako
authored
Merge pull request #28 from engalar/feat/custom-widget-refactor
feat: pluggable widget engine with data-driven definitions
2 parents f6e5e63 + 664144c commit 67f3410

28 files changed

+10415
-2864
lines changed

‎.claude/skills/mendix/custom-widgets.md‎

Lines changed: 337 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,337 @@
1+
---
2+
name: mendix-custom-widgets
3+
description: Use when writing MDL for GALLERY, COMBOBOX, or third-party pluggable widgets in CREATE PAGE / ALTER PAGE statements. Covers built-in widget syntax, child slots (TEMPLATE/FILTER), adding new custom widgets via .def.json, and engine internals.
4+
---
5+
6+
# Custom & Pluggable Widgets in MDL
7+
8+
## Built-in Pluggable Widgets
9+
10+
### GALLERY
11+
12+
Card-layout list with optional template content and filters.
13+
14+
```sql
15+
GALLERY galleryName (
16+
DataSource: DATABASE FROM Module.Entity SORT BY Name ASC,
17+
Selection: Single | Multiple | None
18+
) {
19+
TEMPLATE template1 {
20+
DYNAMICTEXT title (Content: '{1}', ContentParams: [{1} = Name], RenderMode: H4)
21+
DYNAMICTEXT info (Content: '{1}', ContentParams: [{1} = Email])
22+
}
23+
FILTER filter1 {
24+
TEXTFILTER searchName (Attribute: Name)
25+
NUMBERFILTER searchScore (Attribute: Score)
26+
DROPDOWNFILTER searchStatus (Attribute: Status)
27+
DATEFILTER searchDate (Attribute: CreatedAt)
28+
}
29+
}
30+
```
31+
32+
- `TEMPLATE` block -> mapped to `content` property (child widgets rendered per row)
33+
- `FILTER` block -> mapped to `filtersPlaceholder` property (shown above list)
34+
- `Selection: None` omits the selection property (default if omitted)
35+
- Children written directly under GALLERY (no container) go to the first slot with `mdlContainer: "TEMPLATE"`
36+
37+
### COMBOBOX
38+
39+
Two modes depending on the attribute type:
40+
41+
```sql
42+
-- Enumeration mode (Attribute is an enum)
43+
COMBOBOX cbStatus (Label: 'Status', Attribute: Status)
44+
45+
-- Association mode (Attribute is an association)
46+
COMBOBOX cmbCustomer (
47+
Label: 'Customer',
48+
Attribute: Order_Customer,
49+
DataSource: DATABASE Module.Customer,
50+
CaptionAttribute: Name
51+
)
52+
```
53+
54+
- Engine detects association mode when `DataSource` is present (`hasDataSource` condition)
55+
- `CaptionAttribute` is the display attribute on the **target** entity
56+
- In association mode, mapping order matters: DataSource must resolve before Association (sets entityContext)
57+
58+
## Adding a Third-Party Widget
59+
60+
### Step 1 -- Extract .def.json from .mpk
61+
62+
```bash
63+
mxcli widget extract --mpk widgets/MyWidget.mpk
64+
# Output: .mxcli/widgets/mywidget.def.json
65+
66+
# Override MDL keyword
67+
mxcli widget extract --mpk widgets/MyWidget.mpk --mdl-name MYWIDGET
68+
```
69+
70+
The `extract` command parses the .mpk (ZIP archive containing `package.xml` + widget XML) and auto-infers operations from XML property types:
71+
72+
| XML Type | Operation | MDL Source Key |
73+
|----------|-----------|----------------|
74+
| attribute | attribute | `Attribute` |
75+
| association | association | `Association` |
76+
| datasource | datasource | `DataSource` |
77+
| selection | selection | `Selection` |
78+
| widgets | widgets (child slot) | container name (key uppercased) |
79+
| boolean/string/enumeration/integer/decimal | primitive | hardcoded `Value` from defaultValue |
80+
| action/expression/textTemplate/object/icon/image/file | *skipped* | too complex for auto-mapping |
81+
82+
Skipped types require manual configuration in the .def.json.
83+
84+
### Step 2 -- Extract BSON template from Studio Pro
85+
86+
The .def.json only describes mapping rules. The engine also needs a **template JSON** with the complete Type + Object BSON structure.
87+
88+
```bash
89+
# 1. In Studio Pro: drag the widget onto a test page, save the project
90+
# 2. Extract the widget's BSON:
91+
mxcli bson dump -p App.mpr --type page --object "Module.TestPage" --format json
92+
# 3. Extract the Type and Object fields from the CustomWidget, save as:
93+
```
94+
95+
Place at: `project/.mxcli/widgets/mywidget.json`
96+
97+
Template JSON format:
98+
99+
```json
100+
{
101+
"widgetId": "com.vendor.widget.MyWidget",
102+
"name": "My Widget",
103+
"version": "1.0.0",
104+
"extractedFrom": "TestModule.TestPage",
105+
"type": {
106+
"$ID": "aa000000000000000000000000000001",
107+
"$Type": "CustomWidgets$CustomWidgetType",
108+
"WidgetId": "com.vendor.widget.MyWidget",
109+
"PropertyTypes": [
110+
{
111+
"$ID": "aa000000000000000000000000000010",
112+
"$Type": "CustomWidgets$WidgetPropertyType",
113+
"PropertyKey": "datasource",
114+
"ValueType": { "$ID": "...", "Type": "DataSource" }
115+
}
116+
]
117+
},
118+
"object": {
119+
"$ID": "aa000000000000000000000000000100",
120+
"$Type": "CustomWidgets$WidgetObject",
121+
"TypePointer": "aa000000000000000000000000000001",
122+
"Properties": [
123+
2,
124+
{
125+
"$ID": "...",
126+
"$Type": "CustomWidgets$WidgetProperty",
127+
"TypePointer": "aa000000000000000000000000000010",
128+
"Value": {
129+
"$Type": "CustomWidgets$WidgetValue",
130+
"DataSource": null,
131+
"AttributeRef": null,
132+
"PrimitiveValue": "",
133+
"Widgets": [2],
134+
"Selection": "None"
135+
}
136+
}
137+
]
138+
}
139+
}
140+
```
141+
142+
**CRITICAL**: Template must include both `type` (PropertyTypes schema) and `object` (default WidgetObject with all property values). Extract from a real Studio Pro MPR -- do NOT generate programmatically. Mismatched structure causes CE0463.
143+
144+
### Step 3 -- Place files
145+
146+
```
147+
project/.mxcli/widgets/mywidget.def.json <- project scope (highest priority)
148+
project/.mxcli/widgets/mywidget.json <- template JSON (same directory)
149+
~/.mxcli/widgets/mywidget.def.json <- global scope
150+
```
151+
152+
Set `"templateFile": "mywidget.json"` in the .def.json. Project definitions override global ones; global overrides embedded.
153+
154+
### Step 4 -- Use in MDL
155+
156+
```sql
157+
MYWIDGET myWidget1 (DataSource: DATABASE Module.Entity, Attribute: Name) {
158+
TEMPLATE content1 {
159+
DYNAMICTEXT label1 (Content: '{1}', ContentParams: [{1}=Name])
160+
}
161+
}
162+
```
163+
164+
## .def.json Reference
165+
166+
```json
167+
{
168+
"widgetId": "com.vendor.widget.web.mywidget.MyWidget",
169+
"mdlName": "MYWIDGET",
170+
"templateFile": "mywidget.json",
171+
"defaultEditable": "Always",
172+
"propertyMappings": [
173+
{"propertyKey": "datasource", "source": "DataSource", "operation": "datasource"},
174+
{"propertyKey": "attribute", "source": "Attribute", "operation": "attribute"},
175+
{"propertyKey": "someFlag", "value": "true", "operation": "primitive"}
176+
],
177+
"childSlots": [
178+
{"propertyKey": "content", "mdlContainer": "TEMPLATE", "operation": "widgets"}
179+
],
180+
"modes": [
181+
{
182+
"name": "association",
183+
"condition": "hasDataSource",
184+
"propertyMappings": [
185+
{"propertyKey": "optionsSource", "value": "association", "operation": "primitive"},
186+
{"propertyKey": "assocDS", "source": "DataSource", "operation": "datasource"},
187+
{"propertyKey": "assoc", "source": "Association", "operation": "association"}
188+
]
189+
},
190+
{
191+
"name": "default",
192+
"propertyMappings": [
193+
{"propertyKey": "attr", "source": "Attribute", "operation": "attribute"}
194+
]
195+
}
196+
]
197+
}
198+
```
199+
200+
### Mode Conditions
201+
202+
| Condition | Checks |
203+
|-----------|--------|
204+
| `hasDataSource` | AST widget has a `DataSource` property |
205+
| `hasAttribute` | AST widget has an `Attribute` property |
206+
| `hasProp:XYZ` | AST widget has a property named `XYZ` |
207+
208+
Modes are evaluated in definition order -- first match wins. A mode with no `condition` is the default fallback.
209+
210+
### 6 Built-in Operations
211+
212+
| Operation | What it does | Typical Source |
213+
|-----------|-------------|----------------|
214+
| `attribute` | Sets `Value.AttributeRef` on a WidgetProperty | `Attribute` |
215+
| `association` | Sets `Value.AttributeRef` + `Value.EntityRef` | `Association` |
216+
| `primitive` | Sets `Value.PrimitiveValue` | static `value` or property name |
217+
| `datasource` | Sets `Value.DataSource` (serialized BSON) | `DataSource` |
218+
| `selection` | Sets `Value.Selection` (mode string) | `Selection` |
219+
| `widgets` | Replaces `Value.Widgets` array with child widget BSON | child slot |
220+
221+
### Mapping Order Constraints
222+
223+
- **`Association` source must come AFTER `DataSource` source** in the mappings array. The association operation depends on `entityContext` set by a prior DataSource mapping. The registry validates this at load time.
224+
- **`value` takes priority over `source`**: if both are set, the static `value` is used.
225+
226+
### Source Resolution
227+
228+
| Source | Resolution logic |
229+
|--------|-----------------|
230+
| `Attribute` | `w.GetAttribute()` -> `pageBuilder.resolveAttributePath()` |
231+
| `DataSource` | `w.GetDataSource()` -> `pageBuilder.buildDataSourceV3()` -> also updates `entityContext` |
232+
| `Association` | `w.GetAttribute()` -> `pageBuilder.resolveAssociationPath()` + uses current `entityContext` |
233+
| `Selection` | `w.GetSelection()` or `mapping.Default` fallback |
234+
| `CaptionAttribute` | `w.GetStringProp("CaptionAttribute")` -> auto-prefixed with `entityContext` if relative |
235+
| *(other)* | Treated as generic property name: `w.GetStringProp(source)` |
236+
237+
## Engine Internals
238+
239+
### Build Pipeline
240+
241+
When `buildWidgetV3()` encounters an unrecognized widget type:
242+
243+
```
244+
1. Registry lookup: widgetRegistry.Get("MYWIDGET") -> WidgetDefinition
245+
2. Template loading: GetTemplateFullBSON(widgetID, idGenerator, projectPath)
246+
a. Load JSON from embed.FS (or .mxcli/widgets/)
247+
b. Augment from project's .mpk (if newer version available)
248+
c. Phase 1: Collect all $ID values -> generate new UUID mapping
249+
d. Phase 2: Convert Type JSON -> BSON, extract PropertyTypeIDMap
250+
e. Phase 3: Convert Object JSON -> BSON (TypePointer remapped via same mapping)
251+
f. Placeholder leak check (aa000000-prefix IDs must all be remapped)
252+
3. Mode selection: evaluateCondition() on each mode in order -> first match wins
253+
4. Property mappings: for each mapping, resolveMapping() -> OperationFunc()
254+
Each operation locates the WidgetProperty by matching TypePointer against PropertyTypeIDMap
255+
5. Child slots: group AST children by container name, build to BSON, embed via opWidgets
256+
6. Assemble CustomWidget{RawType, RawObject, PropertyTypeIDMap, ObjectTypeID}
257+
```
258+
259+
### PropertyTypeIDMap
260+
261+
The map links PropertyKey names (from .def.json) to their BSON IDs:
262+
263+
```
264+
PropertyTypeIDMap["datasource"] = {
265+
PropertyTypeID: "a1b2c3d4...", // $ID of WidgetPropertyType in Type
266+
ValueTypeID: "e5f6a7b8...", // $ID of ValueType within PropertyType
267+
DefaultValue: "",
268+
ValueType: "DataSource", // Type string
269+
ObjectTypeID: "...", // For nested object list properties
270+
}
271+
```
272+
273+
Operations use this map to locate the correct WidgetProperty in the Object's Properties array by comparing `TypePointer` (binary GUID) against `PropertyTypeID`.
274+
275+
### MPK Augmentation
276+
277+
At template load time, `augmentFromMPK()` checks if the project has a newer `.mpk` for the widget:
278+
279+
```
280+
project/widgets/*.mpk -> FindMPK(projectDir, widgetID) -> ParseMPK()
281+
-> AugmentTemplate(clone, mpkDef)
282+
-> Add missing properties from newer .mpk version
283+
-> Remove stale properties no longer in .mpk
284+
```
285+
286+
This reduces CE0463 errors from widget version drift without requiring manual template re-extraction.
287+
288+
### 3-Tier Registry
289+
290+
| Priority | Location | Scope |
291+
|----------|----------|-------|
292+
| 1 (highest) | `<project>/.mxcli/widgets/*.def.json` | Project |
293+
| 2 | `~/.mxcli/widgets/*.def.json` | Global (user) |
294+
| 3 (lowest) | `sdk/widgets/definitions/*.def.json` (embedded) | Built-in |
295+
296+
Higher priority definitions override lower ones with the same MDL name (case-insensitive).
297+
298+
## Verify & Debug
299+
300+
```bash
301+
# List registered widgets
302+
mxcli widget list -p App.mpr
303+
304+
# Check after creating a page
305+
mxcli check script.mdl -p App.mpr --references
306+
307+
# Full mx check (catches CE0463)
308+
~/.mxcli/mxbuild/*/modeler/mx check App.mpr
309+
310+
# Debug CE0463 -- compare NDSL dumps
311+
mxcli bson dump -p App.mpr --type page --object "Module.PageName" --format ndsl
312+
```
313+
314+
## Common Mistakes
315+
316+
| Mistake | Fix |
317+
|---------|-----|
318+
| CE0463 after page creation | Template version mismatch -- extract fresh template from Studio Pro MPR, or ensure .mpk augmentation picks up new properties |
319+
| Widget not recognized | Check `mxcli widget list`; .def.json must be in `.mxcli/widgets/` with `.def.json` extension |
320+
| TEMPLATE content missing | Widget needs `childSlots` entry with `"mdlContainer": "TEMPLATE"` |
321+
| Association COMBOBOX shows enum behavior | Add `DataSource` to trigger association mode (`hasDataSource` condition) |
322+
| Association mapping fails | Ensure DataSource mapping appears **before** Association mapping in the array |
323+
| Custom widget not found | Place .def.json in `.mxcli/widgets/` inside the project directory |
324+
| Placeholder ID leak error | Template JSON has unreferenced `$ID` values starting with `aa000000` -- ensure all IDs are in the `collectIDs` traversal path |
325+
326+
## Key Source Files
327+
328+
| File | Purpose |
329+
|------|---------|
330+
| `mdl/executor/widget_engine.go` | PluggableWidgetEngine, 6 operations, Build() pipeline |
331+
| `mdl/executor/widget_registry.go` | 3-tier WidgetRegistry, definition validation |
332+
| `sdk/widgets/loader.go` | Template loading, ID remapping, MPK augmentation |
333+
| `sdk/widgets/mpk/mpk.go` | .mpk ZIP parsing, XML property extraction |
334+
| `cmd/mxcli/cmd_widget.go` | `mxcli widget extract/list` CLI commands |
335+
| `sdk/widgets/definitions/*.def.json` | Built-in widget definitions (ComboBox, Gallery) |
336+
| `sdk/widgets/templates/mendix-11.6/*.json` | Embedded BSON templates |
337+
| `mdl/executor/cmd_pages_builder_input.go` | `updateWidgetPropertyValue()` -- TypePointer matching |

0 commit comments

Comments
 (0)