Merge branch 'main' into feature/213-catalog-search

Author: ako

.claude/commands/mxcli-dev/proposal.md

@@ -0,0 +1,159 @@
+# /mxcli-dev:proposal — Create Feature Proposal
+
+Guide the contributor through creating a well-structured feature proposal for
+mxcli. The proposal goes in `docs/11-proposals/PROPOSAL_<name>.md`.
+
+## Process
+
+Work through these phases **interactively** — ask the user questions, don't
+guess. Each phase produces concrete output.
+
+### Phase 1: Understand the Feature
+
+Ask the user:
+
+1. **What** do you want to add? (one sentence)
+2. **Why** — what problem does this solve, or what user story does it enable?
+3. **Which Mendix version** introduced this capability? (affects version-gating)
+4. **Does this add MDL syntax?** (CREATE/ALTER/DROP/SHOW/DESCRIBE statements)
+5. **Does this touch BSON serialization?** (reading or writing Mendix documents)
+
+If the user isn't sure about version or BSON, help them find out:
+- Version: check `reference/mendixmodellib/reflection-data/` or Mendix release notes
+- BSON: check if similar features exist in `sdk/mpr/parser*.go` or `sdk/mpr/writer*.go`
+
+### Phase 2: BSON Investigation (if applicable)
+
+**CRITICAL**: If the feature reads or writes Mendix documents, investigate the
+BSON structure BEFORE designing syntax. Wrong assumptions here cause CE errors
+in Studio Pro that are painful to debug.
+
+1. **Find a working example** — ask the user:
+   - "Do you have a test `.mpr` project with this feature already configured in Studio Pro?"
+   - If yes: use `mxcli bson dump` or `mxcli bson discover` to extract the BSON structure
+   - If no: ask them to create a minimal example in Studio Pro first
+
+2. **Extract the BSON structure**:
+   ```bash
+   # Find the document type
+   ./bin/mxcli -p project.mpr -c "SHOW STRUCTURE ALL" | grep -i <feature-name>
+
+   # Dump the raw BSON
+   ./bin/mxcli bson dump -p project.mpr --type "<BSON $Type>"
+
+   # Or discover all instances of a type
+   ./bin/mxcli bson discover -p project.mpr --pattern "<pattern>"
+   ```
+
+3. **Document the BSON structure** in the proposal:
+   - Storage name (`$Type` field) — verify against `reference/mendixmodellib/reflection-data/`
+   - All fields with types and observed values
+   - Any fields that use Mendix-internal IDs (pointers to other documents)
+   - Note any counter-intuitive naming (like Parent/Child pointer inversion)
+
+4. **Check for storage-name vs qualified-name mismatches** — consult the table
+   in CLAUDE.md under "BSON Storage Names vs Qualified Names". If the feature
+   uses a type that has a known mismatch, document it prominently.
+
+### Phase 3: Check for Overlap
+
+Before writing the proposal, search for existing work:
+
+```bash
+# Existing proposals
+ls docs/11-proposals/ | grep -i <feature>
+
+# Existing implementations
+grep -r "<feature>" mdl/executor/ sdk/mpr/ --include="*.go" -l
+
+# Existing test coverage
+ls mdl-examples/doctype-tests/ | grep -i <feature>
+```
+
+If there's overlap, discuss with the user whether to extend the existing work
+or start fresh.
+
+### Phase 4: Design MDL Syntax (if applicable)
+
+If the feature adds MDL statements, read `.claude/skills/design-mdl-syntax.md`
+first, then design syntax that follows these principles:
+
+- Uses standard verbs: `CREATE`, `ALTER`, `DROP`, `SHOW`, `DESCRIBE`
+- Reads as English — a business analyst understands it
+- Uses `Module.Element` qualified names everywhere
+- Property format: `( Key: value, ... )` with colon separators
+- One example is enough for an LLM to generate correct variants
+- Adding one property is a one-line diff
+
+Present the proposed syntax to the user and ask for feedback before proceeding.
+
+### Phase 5: Write the Proposal
+
+Create `docs/11-proposals/PROPOSAL_<snake_case_name>.md` with this structure:
+
+```markdown
+# Proposal: <Title>
+
+**Status:** Draft
+**Date:** <today's date>
+
+## Problem Statement
+
+<What problem does this solve? Who benefits?>
+
+## BSON Structure
+
+<Document the storage format. Include the $Type, all fields, pointer
+relationships, and any gotchas. If not applicable, explain why.>
+
+## Proposed MDL Syntax
+
+<Show CREATE/ALTER/DROP/SHOW/DESCRIBE examples. If not MDL syntax,
+describe the CLI commands or API changes.>
+
+## Implementation Plan
+
+<Which files need to change? What's the order of operations?>
+
+### Files to modify/create
+
+| File | Change |
+|------|--------|
+| `mdl/grammar/MDLParser.g4` | Add rule for ... |
+| `mdl/ast/...` | New AST node |
+| ... | ... |
+
+## Version Compatibility
+
+<Which Mendix version introduced this? Does it need version-gating?>
+
+## Test Plan
+
+<What test scripts go in mdl-examples/doctype-tests/? What roundtrip
+tests are needed?>
+
+## Open Questions
+
+<What's unresolved? What needs user input or further investigation?>
+```
+
+### Phase 6: Confirm with User
+
+Show the user the proposal and ask:
+- "Does this look right?"
+- "Anything I should add or change?"
+- "Ready to commit?"
+
+If confirmed, commit the proposal file.
+
+---
+
+## Important Reminders
+
+- **Never guess BSON field names.** Always verify against a real `.mpr` file or
+  the reflection data. Wrong field names cause silent data corruption.
+- **Don't skip the BSON investigation** for features that touch Mendix documents.
+  The proposal will be wrong without it, and the implementation will produce
+  CE errors in Studio Pro.
+- **Check existing proposals first.** The `docs/11-proposals/` directory has 30+
+  proposals — some may cover the same ground or provide useful reference.

.claude/commands/mxcli-dev/review.md

@@ -0,0 +1,43 @@
+# /mxcli-dev:review — PR Review
+
+Run a structured review of the current branch's changes against the CLAUDE.md
+checklist, then check the recurring findings table below for patterns that have
+burned us before.
+
+## Steps
+
+1. Run `gh pr view` and `gh pr diff` (or `git diff main...HEAD`) to read the change.
+2. Work through the CLAUDE.md "PR / Commit Review Checklist" in full.
+3. Then check every row in the Recurring Findings table below — flag any match.
+4. Report: blockers first, then moderate issues, then minor. Include a concrete fix
+   option for every blocker (not just "this is wrong").
+5. After the review: **add a row** to the Recurring Findings table for any new
+   pattern not already covered.
+
+---
+
+## Recurring Findings
+
+Patterns caught in real reviews. Each row is a class of mistake worth checking
+proactively. Add a row after every review that surfaces something new.
+
+| # | Finding | Category | Canonical fix |
+|---|---------|----------|---------------|
+| 1 | Formatter emits a keyword not present in `MDLParser.g4` → DESCRIBE output won't re-parse (e.g. `RANGE(...)`) | DESCRIBE roundtrip | Grep grammar before assuming a keyword is valid; if construct can't be expressed yet, emit `-- TypeName(field=value) — not yet expressible in MDL` |
+| 2 | Output uses `$currentObject/Attr` prefix — non-idiomatic; Studio Pro uses bare attribute names | Idiomatic output | Verify against a real Studio Pro BSON sample before choosing a prefix convention |
+| 3 | Malformed BSON field (missing key, wrong type) produces silent garbage output (e.g. `RANGE($x, , )`) | Error handling | Default missing numeric fields to `"0"`; or emit `-- malformed <TypeName>` rather than broken MDL |
+| 4 | No DESCRIBE roundtrip test — grammar gap went undetected until human review | Test coverage | Add roundtrip test: format struct → MDL string → parse → confirm no error |
+| 5 | Hardcoded personal path in committed file (e.g. `/c/Users/Ylber.Sadiku/...`) | Docs quality | Use bare commands (`go test ./...`) without absolute paths in any committed doc or skill |
+| 6 | Docs-only PR cites an unmerged PR as a "model example" — cited PR had blockers | Docs quality | Only cite merged, verified PRs; or annotate with known gaps if citing in-flight work |
+| 7 | Skill/doc table references a function that doesn't exist (e.g. `formatActionStatement()` vs `formatAction()`) | Docs quality | Grep function names before writing: `grep -r "func formatA" mdl/executor/` |
+| 8 | "Always X" rule is too absolute for trivial edge cases (e.g. "always write failing test first" for one-char typos) | Docs quality | Soften to "prefer X" or add an exception clause; include the reasoning so readers can judge edge cases |
+| 9 | Doc comment promises a fallback/feature that doesn't exist in the code (e.g., "raw-map fallback in the client" when no such fallback was implemented) | Docs quality | Grep for function/type names referenced in doc comments to confirm they exist before committing |
+
+---
+
+## After Every Review
+
+- [ ] All blockers have a concrete fix option stated.
+- [ ] Recurring Findings table updated with any new pattern.
+- [ ] If docs-only PR: every function name, path, and PR reference verified against
+      live code before approving.

.claude/skills/debug-bson.md

@@ -25,10 +25,10 @@ Use when encountering:
 ### Step 1: Reproduce the Error
 
 ```bash
-# Create a page via MDL
+# create a page via MDL
 ./bin/mxcli exec script.mdl -p /path/to/app.mpr
 
-# Run mx check to get the error
+# run mx check to get the error
 reference/mxbuild/modeler/mx check /path/to/app.mpr
 ```
 
@@ -58,8 +58,8 @@ import json
 conn = sqlite3.connect('/path/to/app.mpr')
 cursor = conn.cursor()
 
-# Find the document containing the widget
-cursor.execute("SELECT UnitData FROM Unit$ WHERE ContainmentName = 'Document' AND Name = ?", ('PageName',))
+# find the document containing the widget
+cursor.execute("select UnitData from Unit$ where ContainmentName = 'Document' and Name = ?", ('PageName',))
 row = cursor.fetchone()
 doc = bson.decode(row[0])
 
@@ -72,7 +72,7 @@ print(json.dumps(doc, indent=2, default=str))
 Extract the widget's mpk to understand its schema and mode-dependent rules:
 
 ```bash
-# Find the mpk in the project's widgets folder
+# find the mpk in the project's widgets folder
 ls /path/to/project/widgets/*.mpk
 
 # Extract (mpk is a ZIP archive)
@@ -81,8 +81,8 @@ cd /tmp/mpk-widget && unzip /path/to/project/widgets/com.mendix.widget.web.Datag
 ```
 
 Key files inside the mpk:
-- **`{Widget}.xml`** — Property schema: types, defaults, enumerations, nested objects
-- **`{Widget}.editorConfig.js`** — Mode-dependent visibility rules (which properties hide/show based on other values)
+- **`{widget}.xml`** — Property schema: types, defaults, enumerations, nested objects
+- **`{widget}.editorConfig.js`** — Mode-dependent visibility rules (which properties hide/show based on other values)
 - **`package.xml`** — Package version metadata
 
 ### Step 5: Read editorConfig.js for Mode Rules
@@ -108,18 +108,18 @@ Use binary search to find the exact property causing the error:
 # Mutation test: change a single property on a known-good widget
 import bson
 
-# Read the working widget BSON
+# read the working widget BSON
 with open('working-widget.bson', 'rb') as f:
     doc = bson.decode(f.read())
 
-# Change only one property value
+# change only one property value
 # ... modify the specific property ...
 
 # Re-encode and write back
 with open('test-widget.bson', 'wb') as f:
     f.write(bson.encode(doc))
 
-# Then insert back into the MPR and run mx check
+# then insert back into the MPR and run mx check
 ```
 
 ### Step 7: Extract Fresh Templates
@@ -131,7 +131,7 @@ If the widget template is outdated, extract a fresh one:
 reference/mxbuild/modeler/mx convert -p /path/to/app.mpr
 reference/mxbuild/modeler/mx update-widgets /path/to/app.mpr
 
-# Then extract using mxcli
+# then extract using mxcli
 ./bin/mxcli extract-templates -p /path/to/app.mpr -widget "com.mendix.widget.web.datagrid.DataGrid2" -o /tmp/template.json
 ```
 
@@ -159,20 +159,20 @@ Three Mendix tools parse the same BSON but with **different strictness levels**:
 ```bash
 # Self-diff: is the project's BSON clean?
 mx diff project.mpr project.mpr output.mpr
-# Success = all BSON matches schema. Crash = some mxunit has bad properties.
+# success = all BSON matches schema. Crash = some mxunit has bad properties.
 
-# Cross-diff: compare baseline vs modified
+# cross-diff: compare baseline vs modified
 # 1. Extract baseline from git
-mkdir /tmp/baseline && cd project-dir && git archive HEAD -- *.mpr mprcontents/ | tar -x -C /tmp/baseline/
-# 2. Run diff (file names must match!)
+mkdir /tmp/baseline && cd project-dir && git archive head -- *.mpr mprcontents/ | tar -x -C /tmp/baseline/
+# 2. run diff (file names must match!)
 mx diff /tmp/baseline/App.mpr ./App.mpr /tmp/diff-output.mpr
 ```
 
 ### Interpreting mx diff Output
 
 **Detailed error** (when both sides have same-ID objects):
 ```
-Objects with ID b6fc893f-... of type Settings$ServerConfiguration do not have the same properties.
+objects with ID b6fc893f-... of type settings$ServerConfiguration do not have the same properties.
 baseNames = ApplicationRootUrl, ConstantValues, CustomSettings, ..., Tracing;
 newNames = ApplicationRootUrl, ConstantValues, ...
 ```
@@ -186,15 +186,15 @@ Sequence contains no matching element
 
 ### Finding the Offending mxunit File
 
-Write a Go tool (or use the pattern below) to compare property keys of mxcli-written files against Studio Pro-native files of the same `$Type`:
+Write a Go tool (or use the pattern below) to compare property keys of mxcli-written files against Studio Pro-native files of the same `$type`:
 
 ```go
-// Walk mprcontents/, group files by $Type, compare key sets
+// Walk mprcontents/, group files by $type, compare key sets
 // Files with EXTRA keys (vs native files of same type) = the crash cause
 // Files with MISSING keys = also crash cause for mx diff
 ```
 
-The principle: for each `$Type`, ALL instances must have the **exact same set of non-$ property names**. Any deviation → crash.
+The principle: for each `$type`, ALL instances must have the **exact same set of non-$ property names**. Any deviation → crash.
 
 ### Version-Specific Properties
 
@@ -229,7 +229,7 @@ Some properties only exist in certain Mendix versions. Before adding a property
 
 **Symptom**: Studio Pro crashes when opening a project with `System.InvalidOperationException: Sequence contains no matching element` at `Mendix.Modeler.Storage.Mpr.MprProperty..ctor`.
 
-**Root cause**: A BSON document contains a property (field name) that does not exist in the Mendix type definition for its `$Type`. Studio Pro's `MprProperty` constructor uses `First()` to look up each BSON field in the type cache, and crashes on unrecognized fields.
+**Root cause**: A BSON document contains a property (field name) that does not exist in the Mendix type definition for its `$type`. Studio Pro's `MprProperty` constructor uses `First()` to look up each BSON field in the type cache, and crashes on unrecognized fields.
 
 **Diagnosis workflow**:
 
@@ -242,10 +242,10 @@ type_props = defaultdict(set)
 
 def walk_bson(obj, tp):
     if isinstance(obj, dict):
-        t = obj.get("$Type", "")
+        t = obj.get("$type", "")
         if t:
             for k in obj.keys():
-                if k not in ("$Type", "$ID"):
+                if k not in ("$type", "$ID"):
                     tp[t].add(k)
         for v in obj.values():
             walk_bson(v, tp)
@@ -272,7 +272,7 @@ for t, props in crash_props.items():
 
 3. **Extra properties = the crash cause**. The fix is to remove those fields from the writer function.
 
-**Example**: `DomainModels$CrossAssociation` had `ParentConnection` and `ChildConnection` copied from `DomainModels$Association`, but these fields don't exist on `CrossAssociation`. Removing them fixed the crash.
+**Example**: `DomainModels$CrossAssociation` had `ParentConnection` and `ChildConnection` copied from `DomainModels$association`, but these fields don't exist on `CrossAssociation`. Removing them fixed the crash.
 
 **Key principle**: When copying serialization code between similar types (e.g., Association → CrossAssociation), always verify which fields belong to each type by checking a baseline project's BSON.
 
@@ -282,7 +282,7 @@ for t, props in crash_props.items():
 
 **Root cause**: Two variants:
 
-1. **Association stored as Attribute**: In `ChangeActionItem` BSON, an association name was written to the `Attribute` field instead of the `Association` field. Check the executor code that builds `MemberChange` — it must query the domain model to distinguish associations from attributes.
+1. **Association stored as Attribute**: In `ChangeActionItem` BSON, an association name was written to the `attribute` field instead of the `association` field. Check the executor code that builds `MemberChange` — it must query the domain model to distinguish associations from attributes.
 
 2. **Entity treated as Enumeration**: In `CreateVariableAction` BSON, an entity qualified name was used as `DataTypes$EnumerationType` instead of `DataTypes$ObjectType`. Check `buildDataType()` in the visitor — bare qualified names default to `TypeEnumeration` and need catalog-based disambiguation.