Skip to content

Serialization Optimizations: Vector Conversion, Interpolation, and Deepcopy Review#264

Merged
ocots merged 6 commits intodevelopfrom
refactor/serialization-optimizations
Jan 30, 2026
Merged

Serialization Optimizations: Vector Conversion, Interpolation, and Deepcopy Review#264
ocots merged 6 commits intodevelopfrom
refactor/serialization-optimizations

Conversation

@ocots
Copy link
Copy Markdown
Member

@ocots ocots commented Jan 29, 2026

This PR addresses the future work items identified in the idempotence analysis (#217).

Objectives

  1. Refactor Vector Conversion Loop: Simplify and unify the logic for Matrix/Vector conversion in import_ocp_solution by introducing a helper function, removing 8 duplicated code blocks.
  2. Optimize build_solution: Investigate the necessity of deepcopy calls on functions and remove them if safe, to improve performance and reduce memory usage.
  3. Lossless Function Serialization: Prototype ctdeinterpolate (inverse of ctinterpolate) to enable perfect round-tripping of interpolated functions without information loss during discretization.

Changes

  • Added _json_array_to_matrix helper in ext/CTModelsJSON.jl
  • Refactored import_ocp_solution to use helper
  • Updated src/OCP/Building/solution.jl (deepcopy usage)
  • Prototype ctdeinterpolate implementation

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jan 29, 2026

Breakage test results
Date: 2026-01-29 22:41:20

Name Latest Stable
CTDirect compat: v0.6.10-beta compat: v0.7.1-beta.1
CTFlows compat: v0.6.10-beta compat: v0.7.1-beta.1
OptimalControl compat: v0.6.10-beta compat: v0.6.10-beta

ocots added 5 commits January 29, 2026 18:37
- Replace 8 duplicated stack() conversion blocks with _json_array_to_matrix helper
- Add empirical test proving Vector vs Matrix behavior of stack()
- Document _json_array_to_matrix with real-world examples and context
- Remove broken 'Flat Vector case' test, replace with proper investigation test
- Validate conditional necessity: 1D trajectories → Vector, multi-D → Matrix
- All 1726 tests passing, no regressions

Fixes serialization optimization phase 2 with evidence-based approach.
- Add comprehensive closure independence tests (69/69 tests pass)
- Validate deepcopy necessity for function inputs with parameter modification
- Document closure capture behavior: references vs values
- Add empirical test for deepcopy necessity investigation
- Update progress and walkthrough documentation
- Confirm deepcopy is required for all 7 occurrences due to closure semantics

Key findings:
- Julia closures capture variable references, not values
- deepcopy essential for isolation against external variable modifications
- Test suite now robustly validates closure independence
- Create discretization_utils.jl with generic _discretize_function
- Add _discretize_dual helper for optional dual functions
- Support TimeGridModel and auto-dimension detection
- Create _serialize_solution for complete solution serialization
- Add comprehensive tests (21/21 pass)
- Use public getters for solution access (best practice)
- Prepare for JLD2 improvement and build_solution refactoring

Key features:
- Single generic function instead of 3 specialized ones
- Proper encapsulation using getters
- All dual variables handled (path, state, control, boundary, variable)
- Compatible with build_solution for reconstruction
- Refactor export_ocp_solution to use _serialize_solution
- Refactor import_ocp_solution to reconstruct via build_solution
- Store discrete data instead of function objects
- Eliminates most JLD2 warnings (only OCP model functions remain)
- Create test/extras/Project.toml for testing scripts
- Add test_jld2_roundtrip.jl script for validation
- Round-trip test successful: all values match perfectly

Benefits:
- Same discretization logic as JSON (code reuse)
- No function serialization warnings for solution data
- Perfect reconstruction via build_solution
- Consistent with JSON export strategy
✅ FINAL VALIDATION COMPLETE
- 3424/3424 tests pass (full suite)
- 1735/1735 serialization tests pass
- 21/21 discretization utils tests pass

📋 PHASE 4 SUMMARY:
1. Extract discretization utilities (discretization_utils.jl)
2. Create _serialize_solution using public getters
3. Refactor JLD2 to use discretization (eliminate warnings)
4. Add comprehensive test infrastructure
5. Validate round-trip JLD2 serialization
6. Confirm no regressions in full test suite

🎯 BENEFITS:
- Single generic discretization function (not 3)
- Code reuse between JSON and JLD2
- No function serialization warnings
- Perfect reconstruction via build_solution
- Improved maintainability and documentation

📊 TEST RESULTS:
- Phase 2: 1726/1726 ✓
- Phase 3: 69/69 ✓
- Phase 4: 21/21 ✓
- Full suite: 3424/3424 ✓

Ready for merge into develop branch.
@ocots ocots merged commit 2330ca4 into develop Jan 30, 2026
19 of 20 checks passed
@ocots ocots deleted the refactor/serialization-optimizations branch January 30, 2026 12:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant