refactor: decouple executor from storage layer#238
Conversation
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Refactors the MDL executor to stop manipulating sdk/mpr/BSON directly and instead operate through backend abstractions (including new mutation interfaces), while migrating various shared value types into mdl/types.
Changes:
- Replaced many
sdk/mprdependencies withmdl/typesand backend interface calls across executor and catalog. - Introduced backend mutation interfaces (
PageMutator,WorkflowMutator, widget builder/serialization backends) and updated widget update flow to use them. - Added extensive MockBackend-based executor tests and adjusted CLI/example wiring to set a backend factory.
Reviewed changes
Copilot reviewed 109 out of 148 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| mdl/executor/cmd_workflows_write.go | Switch workflow UUID/ID generation to mdl/types. |
| mdl/executor/cmd_workflows_mock_test.go | Add mock-backend tests for workflow show/describe. |
| mdl/executor/cmd_widgets.go | Replace raw BSON widget updates with PageMutator. |
| mdl/executor/cmd_settings_mock_test.go | Add mock-backend tests for settings show/describe. |
| mdl/executor/cmd_security_write.go | Migrate security write types from mpr to types. |
| mdl/executor/cmd_security_mock_test.go | Add mock-backend tests for security show/describe paths. |
| mdl/executor/cmd_rest_clients_mock_test.go | Add mock-backend tests for REST clients show/describe. |
| mdl/executor/cmd_rename.go | Convert rename hit types from mpr to types. |
| mdl/executor/cmd_published_rest_mock_test.go | Add mock-backend tests for published REST services. |
| mdl/executor/cmd_pages_mock_test.go | Add mock-backend tests for pages/snippets/layouts list. |
| mdl/executor/cmd_pages_create_v3.go | Build pages/snippets via backend-backed pageBuilder. |
| mdl/executor/cmd_pages_builder_v3_pluggable.go | Remove BSON-based pluggable widget builder helpers. |
| mdl/executor/cmd_pages_builder_v3_layout.go | Swap mpr.GenerateID usages to types.GenerateID. |
| mdl/executor/cmd_pages_builder_input_filters.go | Remove BSON filter widget construction helpers. |
| mdl/executor/cmd_pages_builder_input_cloning.go | Remove BSON deep-clone helpers from executor. |
| mdl/executor/cmd_pages_builder_input.go | Move snippet resolution to backend list APIs. |
| mdl/executor/cmd_pages_builder.go | Refactor pageBuilder to use backend.FullBackend + mdl/types. |
| mdl/executor/cmd_odata_mock_test.go | Add mock-backend tests for OData show/describe. |
| mdl/executor/cmd_odata.go | Replace mpr helpers with types (IDs, EDMX parsing). |
| mdl/executor/cmd_notconnected_mock_test.go | Add tests for “not connected” behavior across commands. |
| mdl/executor/cmd_navigation_mock_test.go | Add mock-backend tests for navigation show/describe. |
| mdl/executor/cmd_navigation.go | Switch navigation document/spec types to mdl/types. |
| mdl/executor/cmd_modules_mock_test.go | Add mock-backend tests for module listing aggregation. |
| mdl/executor/cmd_misc_mock_test.go | Add mock-backend test for version output. |
| mdl/executor/cmd_microflows_mock_test.go | Add mock-backend tests for microflow show/describe. |
| mdl/executor/cmd_microflows_create.go | Switch IDs to types.GenerateID; use backend for lookups. |
| mdl/executor/cmd_microflows_builder_workflow.go | Switch microflow IDs to types.GenerateID. |
| mdl/executor/cmd_microflows_builder_graph.go | Switch graph element IDs to types.GenerateID. |
| mdl/executor/cmd_microflows_builder_flows.go | Switch sequence flow IDs to types.GenerateID. |
| mdl/executor/cmd_microflows_builder_control.go | Switch control structure IDs to types.GenerateID. |
| mdl/executor/cmd_microflows_builder_annotations.go | Switch annotation IDs to types.GenerateID. |
| mdl/executor/cmd_microflows_builder.go | Generalize microflow builder “reader” to backend interface. |
| mdl/executor/cmd_mermaid_mock_test.go | Add mock-backend test for mermaid domain model describe. |
| mdl/executor/cmd_lint.go | Route lint reader via executor accessor. |
| mdl/executor/cmd_jsonstructures_mock_test.go | Add mock-backend tests for JSON structures show. |
| mdl/executor/cmd_jsonstructures.go | Migrate JSON structure helpers/types to mdl/types. |
| mdl/executor/cmd_javascript_actions_mock_test.go | Add mock-backend tests for JavaScript actions. |
| mdl/executor/cmd_javaactions_mock_test.go | Add mock-backend tests for Java actions. |
| mdl/executor/cmd_javaactions.go | Migrate Java action creation IDs to types.GenerateID. |
| mdl/executor/cmd_import_mappings_mock_test.go | Add mock-backend tests for import mappings show. |
| mdl/executor/cmd_import_mappings.go | Switch JSON element typing to types and use backend for DM lookup. |
| mdl/executor/cmd_imagecollections_mock_test.go | Add mock-backend tests for image collections. |
| mdl/executor/cmd_imagecollections.go | Switch image collection typings to mdl/types. |
| mdl/executor/cmd_fragments_mock_test.go | Add tests for fragment listing behavior. |
| mdl/executor/cmd_folders.go | Switch folder info typing to mdl/types. |
| mdl/executor/cmd_features.go | Update comment to reflect “not connected” behavior. |
| mdl/executor/cmd_export_mappings_mock_test.go | Add mock-backend tests for export mappings show. |
| mdl/executor/cmd_export_mappings.go | Switch JSON element typing to types and use backend for DM lookup. |
| mdl/executor/cmd_enumerations_mock_test.go | Refactor + expand mock tests for enumerations show/describe. |
| mdl/executor/cmd_entities_mock_test.go | Add mock-backend tests for entity listing. |
| mdl/executor/cmd_entities.go | Switch entity creation IDs to types.GenerateID. |
| mdl/executor/cmd_diff_local.go | Delegate microflow parsing from raw BSON map to backend. |
| mdl/executor/cmd_dbconnection_mock_test.go | Add mock-backend tests for database connections show/describe. |
| mdl/executor/cmd_datatransformer_mock_test.go | Add mock-backend tests for data transformers show/describe. |
| mdl/executor/cmd_contract.go | Switch EDMX/AsyncAPI parsing and types to mdl/types. |
| mdl/executor/cmd_constants_mock_test.go | Add mock-backend tests for constants show/describe. |
| mdl/executor/cmd_catalog.go | Update local executor creation to use backend field. |
| mdl/executor/cmd_businessevents_mock_test.go | Add mock-backend tests for business event services. |
| mdl/executor/cmd_businessevents.go | Switch channel ID generation to types.GenerateID. |
| mdl/executor/cmd_associations_mock_test.go | Add mock-backend tests for associations show. |
| mdl/executor/cmd_agenteditor_mock_test.go | Add mock-backend tests for agent editor documents. |
| mdl/catalog/builder_references.go | Use types.NavMenuItem in reference extraction. |
| mdl/catalog/builder_navigation.go | Use types.NavMenuItem in navigation indexing. |
| mdl/catalog/builder_contract.go | Switch contract parsing to types.ParseEdmx / types.ParseAsyncAPI. |
| mdl/catalog/builder.go | Update CatalogReader interface to use mdl/types shared structs. |
| mdl/bsonutil/bsonutil.go | Add BSON ID conversion utilities without sdk/mpr dependency. |
| mdl/backend/workflow.go | Replace image backend signatures to use mdl/types. |
| mdl/backend/security.go | Replace security member access/revocation types to mdl/types. |
| mdl/backend/navigation.go | Replace navigation backend types/specs to mdl/types. |
| mdl/backend/mutation.go | Introduce mutation + widget builder/serialization backend interfaces. |
| mdl/backend/mpr/datagrid_builder_test.go | Move deep-clone BSON tests into mpr backend package, using bsonutil. |
| mdl/backend/mock/mock_workflow.go | Update mock image backend signatures to mdl/types. |
| mdl/backend/mock/mock_security.go | Update mock signatures to use mdl/types. |
| mdl/backend/mock/mock_navigation.go | Update mock signatures to use mdl/types. |
| mdl/backend/mock/mock_mutation.go | Add mock implementations for new mutation/builder interfaces. |
| mdl/backend/mock/mock_module.go | Update mock folder list signatures to mdl/types. |
| mdl/backend/mock/mock_microflow.go | Add mock hook for ParseMicroflowFromRaw. |
| mdl/backend/mock/mock_mapping.go | Update JSON structure signatures to mdl/types. |
| mdl/backend/mock/mock_java.go | Update Java/JS action signatures to mdl/types. |
| mdl/backend/mock/mock_infrastructure.go | Port raw unit/rename/widget type signatures to mdl/types; expand agent editor mocks. |
| mdl/backend/mock/mock_connection.go | Switch version/project version types to mdl/types. |
| mdl/backend/mock/backend.go | Extend MockBackend struct for new types + mutation/builder hooks. |
| mdl/backend/microflow.go | Add backend-level ParseMicroflowFromRaw API. |
| mdl/backend/mapping.go | Port mapping backend JSON structures API to mdl/types. |
| mdl/backend/java.go | Port Java backend typed list/read APIs to mdl/types. |
| mdl/backend/infrastructure.go | Port rename/raw unit/widget type APIs to mdl/types. |
| mdl/backend/doc.go | Update package docs to reflect mdl/types shared types. |
| mdl/backend/connection.go | Port connection + folder backend types to mdl/types. |
| mdl/backend/backend.go | Extend FullBackend with mutation/builder/serialization interfaces. |
| examples/create_datagrid2_page/main.go | Wire executor to mpr backend via backend factory. |
| cmd/mxcli/project_tree.go | Switch menu tree node input type to types.NavMenuItem. |
| cmd/mxcli/main.go | Wire CLI executor to mpr backend via backend factory. |
Comments suppressed due to low confidence (1)
mdl/executor/cmd_diff_local.go:503
microflowBsonToMDLclaims it “falls back to a header-only stub if parsing fails”, but it does not handleParseMicroflowFromRawreturningnil. If the backend cannot parse the raw map (or a mock returns nil),renderMicroflowMDLis likely to panic. Add an explicitnilcheck and return the intended stub output (or an empty/diagnostic MDL representation) when parsing returns nil.
func microflowBsonToMDL(ctx *ExecContext, raw map[string]any, qualifiedName string) string {
qn := splitQualifiedName(qualifiedName)
mf := ctx.Backend.ParseMicroflowFromRaw(raw, model.ID(qn.Name), "")
entityNames, microflowNames := buildNameLookups(ctx)
return renderMicroflowMDL(ctx, mf, qn, entityNames, microflowNames, nil)
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
07c9ad9 to
e275c0f
Compare
e275c0f to
66f637a
Compare
AI Code ReviewCritical IssuesNone found. Moderate IssuesNone found. Minor Issues
What Looks Good
RecommendationApprove the PR. This refactor successfully completes the executor decoupling from the storage layer as intended. All changes are focused, consistent with the architectural direction, maintain test coverage, and improve the codebase without introducing functional changes. The PR addresses technical debt while preserving existing functionality through proper abstraction boundaries. Automated review via OpenRouter (Nemotron Super 120B) — workflow source |
66f637a to
62069b1
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 37 out of 37 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Thanks for the review.
|
AI Code ReviewWhat Looks GoodThis PR successfully accomplishes its goal of decoupling the executor from the storage layer through dependency inversion. Key strengths:
The PR represents a significant improvement in separation of concerns, making the executor storage-agnostic and easier to test/maintain. All changes support the single stated goal of architectural decoupling. Automated review via OpenRouter (Nemotron Super 120B) — workflow source |
3d79a42 to
bc18c2b
Compare
AI Code ReviewCritical IssuesNone found. Moderate IssuesNone found. Minor Issues
What Looks Good
RecommendationApprove this PR. The refactor successfully accomplishes its goal of decoupling the executor from the storage layer through the BackendFactory pattern, moves BSON construction responsibilities to the appropriate backend layer, maintains test coverage, and follows the project's architectural direction. The changes are focused, consistent, and improve the codebase's separation of concerns. Automated review via OpenRouter (Nemotron Super 120B) — workflow source |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 37 out of 37 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (2)
mdl/executor/hierarchy.go:68
newContainerHierarchyImplignores errors fromListUnitsandListFolders. If either call fails (e.g., transient backend error), the hierarchy cache is silently incomplete, which can lead to incorrect module/folder resolution later. Consider returning the error (or at least logging) so callers can fail fast instead of building a partial hierarchy.
mdl/executor/executor.go:334Executor.Close()always returns nil even ifbackend.Disconnect()fails. SinceDisconnect()can return an error, propagate it (or wrap it) so callers can detect close/flush failures.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
AI Code ReviewCritical Issues Moderate Issues Minor Issues What Looks Good
Recommendation Automated review via OpenRouter (Nemotron Super 120B) — workflow source |
de25c84 to
f0cf570
Compare
AI Code ReviewWhat Looks GoodThis PR successfully executes the architectural pivot to decouple the executor from the storage layer via dependency inversion. The changes are consistent, focused, and maintain backward compatibility where needed:
RecommendationApprove. This PR is a well-executed, focused refactor that achieves its stated goal of decoupling the executor from the storage layer. The changes are consistent with the project's architectural direction, maintain test coverage, improve error handling, and provide a clean migration path. No critical issues were identified in the reviewed diff. Automated review via OpenRouter (Nemotron Super 120B) — workflow source |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
AI Code ReviewCritical IssuesNone found. Moderate IssuesNone found. Minor Issues
What Looks Good
RecommendationApprove. This PR successfully completes the executor decoupling refactor as described in the architectural plan. It moves storage-layer-specific BSON construction to the appropriate backend layer while maintaining all functionality through the Automated review via OpenRouter (Nemotron Super 120B) — workflow source |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Replace *mpr.Reader/*mpr.Writer with backend.FullBackend throughout executor. Inject BackendFactory to remove mprbackend import from executor_connect.go. Move all remaining write-path BSON construction (DataGrid2, filters, cloning, widget property updates) behind backend interface. - Remove writer/reader fields from Executor struct - Add BackendFactory injection pattern for connect/disconnect - Create mdl/backend/mpr/datagrid_builder.go (1260 lines) - Add BuildDataGrid2Widget, BuildFilterWidget to WidgetBuilderBackend - Delete bson_helpers.go, cmd_pages_builder_input_cloning.go, cmd_pages_builder_input_datagrid.go, cmd_pages_builder_v3_pluggable.go - Remaining BSON: 3 read-only files (describe, diff) — WASM-safe
4d2582a to
18ff068
Compare
AI Code ReviewWhat Looks GoodThis PR successfully accomplishes its goal of decoupling the executor from the storage layer through dependency inversion. Key strengths:
RecommendationApprove - This PR is a well-executed architectural refactor that successfully decouples the executor from storage layer dependencies. It follows the dependency inversion principle, maintains test coverage, updates all consumers consistently, and prepares the codebase for future shared-types work as described in the PR stack. No blocking issues identified. Automated review via OpenRouter (Nemotron Super 120B) — workflow source |
Code ReviewOverviewThis PR completes the executor decoupling that PRs #235–#237 began. Net result: the executor holds no Positive Aspects
Issues & Concerns1. The test file covers only the deep-clone helper (three tests: ID regeneration, arrays, nil preservation). There are no tests for:
These are the highest-value paths in the new file and the most likely to have edge cases. The PR moves them from the executor (where they were also untested) to the backend — a good structural move — but doesn't add new coverage in the process. 2. If a field is renamed (e.g., 3. New
Checklist Against Project Standards
VerdictApprove. The BackendFactory pattern is well-implemented, the spec aliasing fix directly addresses prior review feedback, and the field-drift test is a practical addition. The main gap is that |
Why
After PRs #235–#237, the executor still holds direct references to
*mpr.Readerand*mpr.Writerin its struct, and several command files still construct BSON for datagrids, filters, and widget cloning. This PR completes the decoupling: the executor works primarily through thebackend.FullBackendabstraction.This is the architectural pivot — dependency inversion via a
BackendFactorypattern replaces the direct MPR reader/writer construction.What changed (incremental from PR #237)
32 files changed, +1,523/-3,121 (net -1,598 lines)
Executor struct decoupled:
writerandreaderfields from theExecutorstructBackendFactoryinjection —executor_connect.gorewritten to create backends through the factory instead of constructingmpr.Reader/mpr.Writerdirectlycmd/mxcli/main.goupdated to inject the factoryRemaining BSON extracted:
mdl/backend/mpr/datagrid_builder.go(1,260 lines) — DataGrid2 construction, filter widgets, column building moved from executor to backendmdl/backend/mpr/datagrid_builder_test.go— test coverage for datagrid buildingDeleted from executor (5 files, ~1,800 lines):
bson_helpers.go— BSON utility functionscmd_pages_builder_input_cloning.go— widget cloning logiccmd_pages_builder_input_datagrid.go— datagrid constructioncmd_pages_builder_input_filters.go— filter widget constructioncmd_pages_builder_v3_pluggable.go— pluggable widget BSONResult: The only BSON remaining in the executor is in 3 read-only files (describe, diff) that deserialize for display — no write-path BSON.
Reader()remains as an escape hatch for these read-only paths.Review-driven improvements
OpenPageForMutation,OpenWorkflowForMutation,BuildDataGrid2Widget,BuildFilterWidgetnow return descriptive errors instead of(nil, nil)updateWidgetsInContainerchecks for nil mutator;ParseMicroflowFromRawreturns stub MDL on nilNavigationProfileSpec,NavHomePageSpec,NavMenuItemSpecinsdk/mprare now type aliases ofmdl/types— eliminates silent divergence riskTestFieldCountDriftwith 30 assertions (15 struct pairs) catches field additions/removals in manually-copied typesexecConnect,reconnect,execDisconnectlog disconnect errors;Executor.Close()returns disconnect errorreconnectchecks for nil factory before attempting reconnectioncmd_pages_builder.gobuildDataGrid2Propertyreceiver renamed to_Stack
PR 4/5 in the shared-types refactoring stack.
Merge order: #232 → #235 → #236 → #237 → this → #239