Refactor of inputs app

[tomjemmett]

Various refactors to improve the codebase and remove some of the historic code that doesn't serve a purpose any longer, and to make some strides to improve the apps responsiveness.

Remove the start() reactive

No longer serving any purpose, removed this in favour of a shiny::req(params$dataset) instead.

Altered handling of data files

Previously, we loaded all of the data files from azure inside of the server function, caching the reactive results. But, caching wasn't enabled for dev workflows, so our dev workflow was always slowed down by needing to re-download files.

Instead, we now have a function get_all_data_files() which is called in run_app if the folder inst/app/data does not exist.

This has necessitated renaming the existing data folder to inst/app/reference.

Used the same pattern for the json schema, this is downloaded by the get_all_data_files() function.

If the data files change, for dev purposes you can simply re-run the function. It will re-download and overwrite all the files.

Move all lookup files into a lookups environment

Rather than loading the lookups into reactives, then evaluating these and passing them to the modules, we now have a centralised lookups singleton.

All of the cases where we use the lookups are moved into the modules themselves, making the main server module much cleaner.

Removed redundant config values

Some options in the golem-config.yml no longer server any purpose. Removed these.

Removed caching code

No longer need to rely on shiny for caching, we persist the data on initial load. During shiny deployments, the app_data folder will not be present. It will be created the first time the app is run after any redeployment.

Improvements to schema validation

There was a potential issue in mod_run_model_remove_invalid_mitigators - if the json provided was valid, then the errors attribute can be NULL, so the subsequent dplyr code could error out.

Loading the schema can take some time, moved this into a singleton pattern so the schema only needs to be loaded once.

Switch from download.file to using httr2 to load the params file - this gets rid of some janky code to append missing final newlines

[Copilot]

Pull request overview

Refactors the Shiny “inputs” app to reduce startup latency in dev, centralize lookup loading, and simplify module wiring by removing legacy caching/reactives and moving some data/schema loading to singleton helpers.

Changes:

• Adds ADLS download helpers (download_provider_data(), get_all_data_files()) and switches runtime loading to local parquet reads via load_provider_data().
• Introduces a singleton get_lookups() environment and updates modules to consume lookups directly (instead of passing reactives around).
• Refactors schema handling to a singleton (get_params_schema()), and removes legacy caching/startup mechanisms (e.g., create_data_cache(), start() reactive, waiting list tab).

Reviewed changes

Copilot reviewed 27 out of 38 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
man/load_provider_data.Rd	Updates generated docs for load_provider_data() signature.
man/get_all_data_files.Rd	Adds generated docs for get_all_data_files().
man/download_provider_data.Rd	Adds generated docs for download_provider_data().
inst/golem-config.yml	Removes redundant slider config values from mitigator config.
inst/app/reference/waiting_list_params.csv	Adds reference CSV for waiting list multipliers.
inst/app/reference/rtt_specialties.csv	Adds RTT specialties reference data.
inst/app/reference/providers.csv	Adds providers lookup reference data.
inst/app/reference/procedures.csv	Adds procedures lookup reference data.
inst/app/reference/peers.csv	Adds peers lookup reference data.
inst/app/reference/nee_table.csv	Adds NEE lookup reference data.
inst/app/reference/ndg_variants.json	Adds NDG variants lookup reference data.
inst/app/reference/mitigator-codes.csv	Adds mitigator codes lookup reference data.
R/run_app.R	Ensures data directory exists at app startup; triggers initial download.
R/mod_waiting_list_imbalances_utils.R	Switches RTT specialties lookup access to get_lookups().
R/mod_waiting_list_imbalances_server.R	Switches waiting list multipliers to get_lookups() output.
R/mod_run_model_server.R	Removes schema text parameter and uses schema singleton.
R/mod_run_model_remove_invalid_mitigators.R	Fixes NULL errors edge case by early-return on valid JSON.
R/mod_run_model_fix_params.R	Updates calls to new schema/migitator-cleanup signatures.
R/mod_non_demographic_adjustment_server.R	Switches NDG variants loading to get_lookups().
R/mod_mitigators_summary_server.R	Switches mitigator lookup usage to get_lookups() and relies on filtered reactive data inputs.
R/mod_mitigators_server.R	Refactors module init, replaces passed-in lookups with get_lookups(), and reorganizes observers/outputs.
R/mod_inequalities_server.R	Refactors to load inequalities data internally via get_inequalities_data().
R/mod_home_server.R	Simplifies signature (providers no longer passed in).
R/mod_expat_repat_ui.R	Adds spinners around plots/map outputs.
R/mod_expat_repat_server.R	Refactors to load data internally and uses lookup singleton for providers/specialties/boundaries.
R/mod_baseline_adjustment_ui.R	Switches RTT specialties loading to get_lookups().
R/mod_baseline_adjustment_server.R	Refactors to load baseline data internally and uses lookup singleton.
R/fct_utils_lookups.R	Introduces singleton lookup loader for CSV/JSON/SF reference data.
R/fct_reduce_values.R	Removes reduce_values() helper.
R/fct_load_provider_data.R	Adds in-memory cache and helpers for reading provider parquet datasets.
R/fct_create_data_cache.R	Removes disk cache helper that configured shinyOptions(cache=...).
R/fct_azure_storage.R	Replaces load_provider_data() ADLS read with download helpers and adds bulk download function.
R/app_ui.R	Removes waiting list imbalances tab from UI.
R/app_server.R	Removes legacy caching/start reactive; rewires module initialization and data loading to new helpers.
R/ZZZ.R	Adds schema download via httr2 and a cached schema loader; removes old rtt_specialties() helper.
.gitignore	Ignores inst/app/data/ (downloaded parquet + schema).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[matt-dray]

Thanks Tom! As discussed: we'll release v5.0 first, then attend to this before it gets stale.

[StatsRhian]

This is a couple of commits behind main. Let's get it up to date and then chat it through @tomjemmett

[tomjemmett]

@StatsRhian rebased!

[tomjemmett]

not deleted, this has been moved to

nhp_inputs/R/mod_mitigators_server.R

Line 124 in 6984393

shiny::observe(

[tomjemmett]

replaced with

nhp_inputs/R/mod_mitigators_server.R

Line 15 in 6984393

default_interval <- c(0.95, 1.0)

[matt-dray]

Sorry, #688 has caused merge conflicts due to the removal of time-profile-related code. I don't think these will be painful to clean up. We can discuss in our chat in a minute.

[matt-dray]

Cheers, you walked through the latest version of this with @StatsRhian and I yesterday.

I said I would do a local test of this and we would get it deployed to dev soon after so that we can test it on the server, given the nature of some of the changes.

So far as I can tell, there are no issues when running locally.

To note: before approving this, I also did a test-run of the current dev deployments to make sure that the latest changes (time-profile removal) worked as intended and that any problems we see in dev now will be related to this PR and not the time-profile stuff.