Skip to content

Module compartmentalization#1097

Closed
ConnectedSystems wants to merge 10 commits into
compartmentalized-adriafrom
module-compartmentalization
Closed

Module compartmentalization#1097
ConnectedSystems wants to merge 10 commits into
compartmentalized-adriafrom
module-compartmentalization

Conversation

@ConnectedSystems
Copy link
Copy Markdown
Collaborator

@Zapiano the target branch here compartmentalized-adria is a branch off the iv-study-changes branch.

This is a larger set of changes and I don't want to interfere with the study codebase as it should be fairly static from now.

Note we also need to merge in changes on main (including the documentation updates).


Summary

Moves optional, heavyweight analysis tools (MLJ, SIRUS, Bootstrap, HypothesisTests, DataEnvelopmentAnalysis) out of the ADRIA core package and into a new companion package ADRIAAnalysis.jl, structured as a Julia 1.11 workspace member.

This reduces ADRIA's mandatory dependency footprint — users who only run simulations no longer pay the load-time and precompilation cost of ML and statistical inference libraries.


Changes

New: packages/ADRIAAnalysis/

A new Julia package declared as a workspace member in the root Project.toml:

packages/ADRIAAnalysis/
├── Project.toml          # MLJ, SIRUS, Bootstrap, HypothesisTests, DEA, ADRIA as deps
├── Manifest.toml
├── src/
│   ├── ADRIAAnalysis.jl  # package entry point
│   ├── analysis/
│   │   ├── clustering.jl
│   │   ├── data_envelopment.jl
│   │   ├── feature_set.jl
│   │   ├── intervention.jl
│   │   ├── rule_extraction.jl
│   │   └── scenario.jl
│   └── sensitivity/
│       └── sensitivity.jl
└── test/
    └── runtests.jl       # 48 tests covering all moved modules

ADRIA core trimmed

  • Removed exports and includes for all moved analysis functions
  • Removed MLJ, SIRUS, Bootstrap, HypothesisTests, DataEnvelopmentAnalysis from Project.toml dependencies
  • Deleted source files relocated to ADRIAAnalysis:
    • src/analysis/clustering.jl
    • src/analysis/data_envelopment.jl
    • src/analysis/feature_set.jl
    • src/analysis/intervention.jl
    • src/analysis/rule_extraction.jl
    • src/analysis/scenario.jl
    • src/analysis/sensitivity.jl
  • Removed Base.precompile block for moved symbols

AvizExt compatibility fixes

AvizExt previously called ADRIA.analysis.* functions that are now in ADRIAAnalysis. Since AvizExt cannot depend on ADRIAAnalysis (circular), the scenario grouping helpers are duplicated as private functions in ext/AvizExt/_scenario_helpers.jl:

  • _scenario_clusters, _scenario_rcps, _scenario_types — private copies of the moved grouping logic
  • Type signatures in viz/data_envelopment.jl and viz/rule_extraction.jl relaxed from concrete DEAResult/Rule to abstract/untyped to avoid hard imports

Build, Docker, docs

  • Dockerfile: Julia 1.10.4 → 1.11.5
  • build/make.jl: platform-aware sysimage extension (.dll / .dylib / .so)
  • build/precompile_script.jl: restored with real execution trace
  • docs/src/development/docker.md: updated for Julia 1.11 and workspace layout
  • docs/src/usage/analysis.md: added using ADRIAAnalysis import note
  • docs/src/usage/getting_started.md: added ADRIAAnalysis install instructions

Workspace declaration

Root Project.toml gains:

[workspace]
packages = ["packages/ADRIAAnalysis"]

Breaking changes

Old (ADRIA ≤ previous) New (ADRIAAnalysis)
ADRIA.analysis.sensitivity.pawn ADRIAAnalysis.sensitivity.pawn
ADRIA.analysis.cluster_series_by_outcome ADRIAAnalysis.cluster_series_by_outcome
ADRIA.analysis.rules ADRIAAnalysis.rules
ADRIA.analysis.data_envelopment_analysis ADRIAAnalysis.data_envelopment_analysis
ADRIA.analysis.scenario_types ADRIAAnalysis.scenario_types

Users need to add ADRIAAnalysis to their project and replace the ADRIA.analysis.* prefixes.

Installing ADRIAAnalysis (unregistered): Until the package is registered in the General registry, users install it via the subdir Pkg feature (Julia 1.11+):

pkg> add https://github.com/open-AIMS/ADRIA.jl:packages/ADRIAAnalysis

Or explicitly in their Project.toml:

[sources]
ADRIAAnalysis = {url = "https://github.com/open-AIMS/ADRIA.jl", subdir = "packages/ADRIAAnalysis"}

Note: getting_started.md does not yet include these instructions — a follow-up doc PR is needed.


Deferred / known follow-ons

  • Step 9: Migrate test domain to a Julia Artifact. Deferred until a clean upload pipeline exists. Blocks @compile_workload body in both ADRIA and ADRIAAnalysis.
  • AWSS3 / libsodium_jll in load path: Still present via the Zarr dependency. Zarr v0.10 moves AWSS3 to an extension; upgrade is currently blocked by a compat constraint — investigate separately.
  • Invalidation chain: REPL, Distributions, Accessors, DifferentiationInterface still show 100% recompilation on warm load. Likely Zarr → LazyArtifacts → Pkg → REPL chain.
  • Sysimage on Linux: Requires Docker/CI environment — not verified locally.
  • Version bump: This is a breaking change; Project.toml version must be incremented before release.

Test plan

  • ] test ADRIAAnalysis passes all 48 tests
  • ] test ADRIA passes with moved symbols removed
  • using ADRIA succeeds without MLJ/SIRUS/Bootstrap in scope
  • using ADRIAAnalysis resolves all moved symbols
  • AvizExt loads and clustering/sensitivity plots render correctly
  • Sysimage build succeeds on macOS/Windows (Linux: pending Docker)
  • Docs build (make.jl) succeeds

It was barely used and the ecosystem is moving to other approaches (base @threads is pretty good now too).
Introduces the ADRIAAnalysis package under packages/ADRIAAnalysis/ and
registers it as a workspace member in the root Project.toml so that it
resolves against the shared Manifest without being a hard dependency of
ADRIA core.
Relocates data_envelopment, feature_set, intervention, rule_extraction,
scenario, sensitivity, and clustering from src/analysis/ into
packages/ADRIAAnalysis/src/ so that heavy optional dependencies (SIRUS,
Clustering, etc.) are only required when ADRIAAnalysis is explicitly loaded.
Strips exports, imports, and includes for the analysis modules now
living in ADRIAAnalysis. ADRIA core no longer carries optional
dependencies such as SIRUS, Clustering, or DEA — these are only
pulled in when ADRIAAnalysis is loaded.
AvizExt previously called ADRIA.analysis.scenario_types/rcps/clusters
and imported Rule/DEAResult which moved to ADRIAAnalysis. Fixes:
- Inline private _scenario_helpers.jl for pure scenario-grouping fns
- Relax Rule/DEAResult dispatch signatures to duck-typed AbstractVector
  and untyped parameters so AvizExt compiles without ADRIAAnalysis
Adds 48 tests covering clustering, sensitivity, scenario grouping,
rule extraction, DEA, and feature-set analysis in ADRIAAnalysis.
Removes test coverage for modules no longer present in ADRIA core.
…layout

Dockerfile and sysimage build scripts now resolve the ADRIAAnalysis
workspace member. Docs reflect the new package split and updated
getting-started instructions for the workspace-based install.
Deletes the original src/analysis/{data_envelopment,feature_set,
intervention,rule_extraction,scenario,sensitivity}.jl files now that
their content lives in packages/ADRIAAnalysis/src/. Logically part of
the migration commit.
Cleaner visual naming (and more consistent with other planned subpackages).
@Zapiano Zapiano closed this May 27, 2026
@Zapiano
Copy link
Copy Markdown
Member

Zapiano commented May 27, 2026

Closed in favor of #1100

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants