Proof of concept: Custom Model Re-runs and Common Model Config UI

[jkislin]

Very much a proof of concept as of 4/30, but you can see the general idea:

Note that only config values passed under ops: run_custom_forecast: config: common_model_config will go to the underlying models. assets determines which models are run. I'm currently working on getting the non-common config (i.e. all the TimeseriesConfig and FusionConfig and PyrenewEConfig etc. to display as well. I need to brush up on my python OOP!

I don't have many cycles next week to work through this, so it may be a little preliminary to even submit this draft. That said, I thought it would be useful to see my approach given discussion on #1005. Others should feel free to take this branch and run with it if they think they can get to it before I can.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.49%. Comparing base (46e5f62) to head (29a8eec).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1049   +/-   ##
=======================================
  Coverage   62.49%   62.49%           
=======================================
  Files          33       33           
  Lines        2229     2229           
=======================================
  Hits         1393     1393           
  Misses        836      836           

Flag	Coverage Δ
hewr	76.98% <ø> (ø)
pipelines	60.75% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jkislin]

Working with something like this for now. I unfortunately have to pivot, but can revisit end of week:

execution:
  config:
    executor:
      multiprocess_executor:
        max_concurrent: null
        retries:
          enabled: {}
loggers: {}
ops:
  run_custom_forecast:
    config:
      asset_execution_config:
        config:
          executor:
            azure_batch_executor:
              container_kwargs:
                volumes:
                - nssp-archival-vintages:/cfa-stf-routine-forecasting/nssp-archival-vintages
                - nssp-etl:/cfa-stf-routine-forecasting/nssp-etl
                - nwss-vintages:/cfa-stf-routine-forecasting/nwss-vintages
                - prod-param-estimates:/cfa-stf-routine-forecasting/params
                - pyrenew-hew-config:/cfa-stf-routine-forecasting/config
                - pyrenew-hew-prod-output:/cfa-stf-routine-forecasting/output
                - pyrenew-test-output:/cfa-stf-routine-forecasting/test-output
                working_dir: /cfa-stf-routine-forecasting
              env_vars:
              - VIRTUAL_ENV=/cfa-stf-routine-forecasting/.venv
              image: ghcr.io/cdcgov/cfa-stf-routine-forecasting:jk-config-op
              pool_name: pyrenew-dagster-pool
              retries:
                enabled: {}
      assets:
      - timeseries_e
      - epiweekly_timeseries_e
      - pyrenew_e
      - pyrenew_h
      - pyrenew_he
      - fuse_pyrenew_e_ts
      - fuse_pyrenew_e_ts_epiweekly
      - fuse_pyrenew_he_ts
      - fuse_pyrenew_he_ts_epiweekly
      - postprocess_forecasts
      common_model_config:
        config:
          diseases:
          - COVID-19
          - Influenza
          - RSV
          exclude_last_n_days: 1
          locations:
          - US
          - AL
          - AK
          - AZ
          - AR
          - CA
          - CO
          - CT
          - DE
          - DC
          - FL
          - GA
          - HI
          - ID
          - IL
          - IN
          - IA
          - KS
          - KY
          - LA
          - ME
          - MD
          - MA
          - MI
          - MN
          - MS
          - MO
          - MT
          - NE
          - NV
          - NH
          - NJ
          - NM
          - NY
          - NC
          - ND
          - OH
          - OK
          - OR
          - PA
          - RI
          - SC
          - SD
          - TN
          - TX
          - UT
          - VT
          - VA
          - WA
          - WV
          - WI
          - WY
          n_training_days: 150
          output_basedir: test-output
      postprocess_config:
        config:
          local_copy_dir: ''
          output_basedir: test-output
          postprocess_diseases:
          - COVID-19
          - Influenza
          - RSV
          save_local_copy: false
          skip_existing: false
      timeseries_specific_config:
        config:
          n_samples: 400
resources: {}

@damonbayer let me know if there's anything else you'd like to see. Note timeseries_specific_config. Also, under the hood, we'd strip out "WY" from e models rather than do that here in config. Also note - if you want to differ config values between assets... well we already have that functionality ;)

The work I've done so far is the "frontend"; we still need to unpack this and send it to actual execution. I think we can use our old launch_asset_backfill() function as a guide.

[jkislin]

This is likely deprecated vis. https://github.com/CDCgov/cfa-dagster/pull/125/changes
Will close or adapt soon.