|
| 1 | +# Pluggable Widget Engine |
| 2 | + |
| 3 | +The Pluggable Widget Engine replaces hardcoded Go builder functions with a data-driven system. Widget behavior is described in declarative `.def.json` files, and a generic engine applies them against BSON templates at build time. |
| 4 | + |
| 5 | +## Architecture Overview |
| 6 | + |
| 7 | +``` |
| 8 | +MDL Script: COMBOBOX cmbStatus (Label: 'Status', Attribute: Priority) |
| 9 | + │ |
| 10 | + ▼ |
| 11 | +┌─────────────────────────────────────────────────────────────┐ |
| 12 | +│ WidgetRegistry │ |
| 13 | +│ 3-tier lookup: embedded → ~/.mxcli/widgets/ → .mxcli/ │ |
| 14 | +│ │ |
| 15 | +│ "COMBOBOX" → combobox.def.json │ |
| 16 | +│ "GALLERY" → gallery.def.json │ |
| 17 | +│ "MYWIDGET" → mywidget.def.json (user-defined) │ |
| 18 | +└─────────────────────┬───────────────────────────────────────┘ |
| 19 | + │ |
| 20 | + ▼ |
| 21 | +┌─────────────────────────────────────────────────────────────┐ |
| 22 | +│ PluggableWidgetEngine.Build() │ |
| 23 | +│ │ |
| 24 | +│ 1. Load template (combobox.json → augment from .mpk) │ |
| 25 | +│ 2. Select mode (enum mode vs association mode) │ |
| 26 | +│ 3. Apply mappings (set AttributeRef, PrimitiveValue, etc.) │ |
| 27 | +│ 4. Apply child slots (embed child widgets into BSON) │ |
| 28 | +│ 5. Assemble widget (CustomWidgets$CustomWidget) │ |
| 29 | +│ │ |
| 30 | +│ OperationRegistry │ |
| 31 | +│ ┌──────────┬───────────┬───────────┬──────────┬─────────┐ │ |
| 32 | +│ │attribute │association│ primitive │datasource│ widgets │ │ |
| 33 | +│ │ │ │ │selection │ │ │ |
| 34 | +│ └──────────┴───────────┴───────────┴──────────┴─────────┘ │ |
| 35 | +└─────────────────────────────────────────────────────────────┘ |
| 36 | + │ |
| 37 | + ▼ |
| 38 | + CustomWidget BSON |
| 39 | + (written to .mpr / .mxunit) |
| 40 | +``` |
| 41 | + |
| 42 | +## Widget Definitions (`.def.json`) |
| 43 | + |
| 44 | +Each pluggable widget has a definition file that maps MDL syntax to template properties. |
| 45 | + |
| 46 | +### ComboBox Example (two modes) |
| 47 | + |
| 48 | +```json |
| 49 | +{ |
| 50 | + "widgetId": "com.mendix.widget.web.combobox.Combobox", |
| 51 | + "mdlName": "COMBOBOX", |
| 52 | + "templateFile": "combobox.json", |
| 53 | + "defaultEditable": "Always", |
| 54 | + "modes": [ |
| 55 | + { |
| 56 | + "name": "association", |
| 57 | + "condition": "hasDataSource", |
| 58 | + "propertyMappings": [ |
| 59 | + {"propertyKey": "optionsSourceType", "value": "association", "operation": "primitive"}, |
| 60 | + {"propertyKey": "optionsSourceAssociationDataSource", "source": "DataSource", "operation": "datasource"}, |
| 61 | + {"propertyKey": "attributeAssociation", "source": "Association", "operation": "association"}, |
| 62 | + {"propertyKey": "optionsSourceAssociationCaptionAttribute", "source": "CaptionAttribute", "operation": "attribute"} |
| 63 | + ] |
| 64 | + }, |
| 65 | + { |
| 66 | + "name": "default", |
| 67 | + "propertyMappings": [ |
| 68 | + {"propertyKey": "attributeEnumeration", "source": "Attribute", "operation": "attribute"} |
| 69 | + ] |
| 70 | + } |
| 71 | + ] |
| 72 | +} |
| 73 | +``` |
| 74 | + |
| 75 | +### Gallery Example (child slots) |
| 76 | + |
| 77 | +```json |
| 78 | +{ |
| 79 | + "widgetId": "com.mendix.widget.web.gallery.Gallery", |
| 80 | + "mdlName": "GALLERY", |
| 81 | + "templateFile": "gallery.json", |
| 82 | + "propertyMappings": [ |
| 83 | + {"propertyKey": "datasource", "source": "DataSource", "operation": "datasource"}, |
| 84 | + {"propertyKey": "itemSelection", "source": "Selection", "operation": "selection", "default": "Single"}, |
| 85 | + {"propertyKey": "pageSize", "value": "20", "operation": "primitive"} |
| 86 | + ], |
| 87 | + "childSlots": [ |
| 88 | + {"propertyKey": "content", "mdlContainer": "TEMPLATE", "operation": "widgets"}, |
| 89 | + {"propertyKey": "filtersPlaceholder", "mdlContainer": "FILTER", "operation": "widgets"} |
| 90 | + ] |
| 91 | +} |
| 92 | +``` |
| 93 | + |
| 94 | +## Mode Selection |
| 95 | + |
| 96 | +Modes are evaluated in definition order. The first mode whose condition matches the MDL AST is selected. A mode without a condition is the fallback (default). |
| 97 | + |
| 98 | +| Condition | Checks | |
| 99 | +|-----------|--------| |
| 100 | +| `hasDataSource` | AST widget has a `DataSource` property | |
| 101 | +| `hasAttribute` | AST widget has an `Attribute` property | |
| 102 | +| `hasProp:XYZ` | AST widget has a property named `XYZ` | |
| 103 | +| *(none)* | Fallback -- selected if no other mode matches | |
| 104 | + |
| 105 | +## Six Built-in Operations |
| 106 | + |
| 107 | +| Operation | What It Sets | Input | |
| 108 | +|-----------|-------------|-------| |
| 109 | +| `attribute` | `Value.AttributeRef` | Qualified path (`Module.Entity.Attr`) | |
| 110 | +| `association` | `Value.EntityRef` (IndirectEntityRef + EntityRefStep) | Association path + target entity | |
| 111 | +| `primitive` | `Value.PrimitiveValue` | Static string value | |
| 112 | +| `datasource` | `Value.DataSource` | BSON data source object | |
| 113 | +| `selection` | `Value.Selection` | `"Single"`, `"Multi"`, or `"None"` | |
| 114 | +| `widgets` | `Value.Widgets` array | Serialized child widget BSON | |
| 115 | + |
| 116 | +All operations are registered in an `OperationRegistry`. Custom operations can be added without modifying the engine. |
| 117 | + |
| 118 | +## Mapping Order Dependency |
| 119 | + |
| 120 | +**The engine processes property mappings in array order.** Some operations depend on side effects: |
| 121 | + |
| 122 | +- `datasource` sets `pageBuilder.entityContext` as a side effect |
| 123 | +- `association` reads `pageBuilder.entityContext` to resolve the target entity |
| 124 | + |
| 125 | +Therefore, in any mode using both, **datasource must come before association** in the mappings array. This is enforced at definition load time -- a validation error is raised if an `association` mapping appears before any `datasource` mapping. |
| 126 | + |
| 127 | +## Source/Operation Compatibility |
| 128 | + |
| 129 | +Not all source/operation combinations are valid. These are rejected at load time: |
| 130 | + |
| 131 | +| Source | Incompatible Operations | |
| 132 | +|--------|------------------------| |
| 133 | +| `Attribute` | `association`, `datasource` | |
| 134 | +| `Association` | `attribute`, `datasource` | |
| 135 | +| `DataSource` | `attribute`, `association` | |
| 136 | + |
| 137 | +## Version Handling |
| 138 | + |
| 139 | +### The Problem |
| 140 | + |
| 141 | +Widget property schemas change between Mendix versions. The Gallery widget has 23 properties in Mendix 10.24 (widget v3.0.1) but 33 in 11.6.3. Writing BSON with wrong properties causes CE0463 ("widget definition changed"). |
| 142 | + |
| 143 | +### The Solution: Baseline + Augmentation |
| 144 | + |
| 145 | +``` |
| 146 | +Embedded template (11.6.0 baseline) |
| 147 | + │ |
| 148 | + ▼ |
| 149 | + Deep clone (never mutate cache) |
| 150 | + │ |
| 151 | + ▼ |
| 152 | + augmentFromMPK() |
| 153 | + ├── FindMPK(projectDir, widgetID) |
| 154 | + │ Scan project/widgets/*.mpk |
| 155 | + │ Match by widget ID from package.xml |
| 156 | + │ |
| 157 | + ├── ParseMPK(mpkPath) |
| 158 | + │ Extract property definitions from widget XML |
| 159 | + │ Extract widget version from package.xml |
| 160 | + │ |
| 161 | + └── AugmentTemplate(clone, mpkDef) |
| 162 | + Compare template keys vs .mpk keys: |
| 163 | + ├── Missing (in .mpk, not in template) → clone from exemplar or create |
| 164 | + └── Stale (in template, not in .mpk) → remove |
| 165 | + │ |
| 166 | + ▼ |
| 167 | + Augmented template (matches project's widget version) |
| 168 | + │ |
| 169 | + ▼ |
| 170 | + 3-phase BSON conversion (ID remap → Type → Object) |
| 171 | +``` |
| 172 | + |
| 173 | +### Why the Baseline Must Be Conservative |
| 174 | + |
| 175 | +The embedded template should be from the **oldest supported Mendix version**, not the newest: |
| 176 | + |
| 177 | +- **Augmentation adds properties:** New properties in a newer `.mpk` are cloned from type-matching exemplars or created with sensible defaults. This is reliable. |
| 178 | +- **Augmentation removes properties:** Extra properties from the baseline that don't exist in an older `.mpk` are stripped cleanly. |
| 179 | +- **Augmentation cannot restructure:** If a newer template has fundamentally different property types, nesting, or internal structure, augmentation fails silently. Starting from an older baseline avoids this. |
| 180 | + |
| 181 | +### Example: Gallery Across Versions |
| 182 | + |
| 183 | +| Version | Properties | Augmentation Action | |
| 184 | +|---------|-----------|-------------------| |
| 185 | +| 10.24 (v3.0.1) | 23 | Remove 10 stale properties from 11.6 baseline | |
| 186 | +| 11.6.0 | 23 | No augmentation needed (matches baseline) | |
| 187 | +| 11.6.3 | 33 | Add 10 missing properties from .mpk | |
| 188 | +| 11.8.0 | 33+ | Add any new properties from .mpk | |
| 189 | + |
| 190 | +## Widget Registry (3-Tier) |
| 191 | + |
| 192 | +Definitions are loaded in priority order. Higher-priority definitions override lower ones: |
| 193 | + |
| 194 | +| Priority | Location | Use Case | |
| 195 | +|----------|----------|----------| |
| 196 | +| 1 (highest) | `<project>/.mxcli/widgets/*.def.json` | Project-specific widget overrides | |
| 197 | +| 2 | `~/.mxcli/widgets/*.def.json` | Global user-defined widgets | |
| 198 | +| 3 (lowest) | `sdk/widgets/definitions/` (embedded) | Built-in ComboBox, Gallery | |
| 199 | + |
| 200 | +Definitions are looked up by MDL name (case-insensitive): `COMBOBOX`, `GALLERY`, or any custom name. |
| 201 | + |
| 202 | +### Validation at Load Time |
| 203 | + |
| 204 | +When a definition is loaded, it is validated: |
| 205 | + |
| 206 | +- All `operation` fields must reference a registered operation |
| 207 | +- `Source` and `operation` must be compatible (see table above) |
| 208 | +- `association` mappings must appear after `datasource` mappings |
| 209 | +- `widgetId` and `mdlName` must be non-empty |
| 210 | + |
| 211 | +Invalid definitions produce an immediate error rather than failing silently at build time. |
| 212 | + |
| 213 | +## Adding a Custom Widget |
| 214 | + |
| 215 | +### Step 1: Extract from `.mpk` |
| 216 | + |
| 217 | +```bash |
| 218 | +mxcli widget extract --mpk widgets/MyRatingWidget.mpk |
| 219 | +# Creates: .mxcli/widgets/myratingwidget.def.json (skeleton) |
| 220 | +``` |
| 221 | + |
| 222 | +The `extract` command parses the `.mpk` ZIP, reads the widget XML, and auto-infers operations from property types: |
| 223 | + |
| 224 | +| XML Type | Inferred Operation | |
| 225 | +|----------|-------------------| |
| 226 | +| `attribute` | `attribute` | |
| 227 | +| `association` | `association` | |
| 228 | +| `datasource` | `datasource` | |
| 229 | +| `expression` | *(skipped -- manual)* | |
| 230 | +| `widgets` | `widgets` (child slot) | |
| 231 | +| `enumeration` | `primitive` | |
| 232 | +| `boolean` | `primitive` | |
| 233 | +| `integer` | `primitive` | |
| 234 | + |
| 235 | +### Step 2: Extract Template |
| 236 | + |
| 237 | +Create the widget in Studio Pro, then extract its BSON: |
| 238 | + |
| 239 | +```bash |
| 240 | +mxcli bson dump -p App.mpr --type page --object "Module.TestPage" --format json |
| 241 | +``` |
| 242 | + |
| 243 | +Save the `type` and `object` sections as `.mxcli/widgets/myratingwidget.json`. |
| 244 | + |
| 245 | +### Step 3: Edit the Definition |
| 246 | + |
| 247 | +Adjust the generated `.def.json` to map MDL properties correctly. Set the `mdlName` to your desired keyword: |
| 248 | + |
| 249 | +```json |
| 250 | +{ |
| 251 | + "widgetId": "com.example.widget.RatingWidget", |
| 252 | + "mdlName": "RATING", |
| 253 | + "templateFile": "myratingwidget.json", |
| 254 | + "propertyMappings": [ |
| 255 | + {"propertyKey": "ratingAttribute", "source": "Attribute", "operation": "attribute"}, |
| 256 | + {"propertyKey": "maxRating", "value": "5", "operation": "primitive"} |
| 257 | + ] |
| 258 | +} |
| 259 | +``` |
| 260 | + |
| 261 | +### Step 4: Use in MDL |
| 262 | + |
| 263 | +```sql |
| 264 | +CREATE PAGE MyModule.ReviewPage (...) { |
| 265 | + CONTAINER main () { |
| 266 | + RATING starRating (Label: 'Rating', Attribute: Review_Rating) |
| 267 | + } |
| 268 | +} |
| 269 | +``` |
| 270 | + |
| 271 | +## What Stays Hardcoded |
| 272 | + |
| 273 | +**Native Mendix widgets** (TextBox, DataView, ListView, LayoutGrid, Container, etc.) use `Forms$TextBox`, `Forms$DataView`, etc. -- NOT `CustomWidgets$CustomWidget`. These stay as hardcoded builders because they have fundamentally different BSON structures and don't use the template system. |
| 274 | + |
| 275 | +## Key Source Files |
| 276 | + |
| 277 | +| File | Purpose | |
| 278 | +|------|---------| |
| 279 | +| `mdl/executor/widget_engine.go` | PluggableWidgetEngine, OperationRegistry, Build() pipeline | |
| 280 | +| `mdl/executor/widget_registry.go` | 3-tier definition loading, validation | |
| 281 | +| `sdk/widgets/definitions/*.def.json` | Built-in widget definitions | |
| 282 | +| `sdk/widgets/loader.go` | Template loading, 3-phase ID remapping | |
| 283 | +| `sdk/widgets/augment.go` | MPK augmentation (add/remove properties) | |
| 284 | +| `sdk/widgets/mpk/mpk.go` | .mpk ZIP parsing, property extraction | |
| 285 | +| `sdk/widgets/templates/mendix-11.6/*.json` | Embedded baseline templates | |
| 286 | +| `cmd/mxcli/cmd_widget.go` | `mxcli widget extract/list` CLI | |
0 commit comments