Add lazy option to cfa.stf.data functions#1029
Conversation
Agent-Logs-Url: https://github.com/CDCgov/cfa-stf-routine-forecasting/sessions/c2cbec45-3e19-4a8e-8191-af358483fa65 Co-authored-by: damonbayer <8095080+damonbayer@users.noreply.github.com>
lazy option to cfa.stf.data functions
|
@copilot Use the |
Agent-Logs-Url: https://github.com/CDCgov/cfa-stf-routine-forecasting/sessions/74471d02-3aef-4b10-aa90-74c986390229 Co-authored-by: damonbayer <8095080+damonbayer@users.noreply.github.com>
…hub.com/CDCgov/cfa-stf-routine-forecasting into copilot/add-lazy-option-to-cfa-stf-data
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1029 +/- ##
=======================================
Coverage 59.92% 59.92%
=======================================
Files 31 31
Lines 2041 2041
=======================================
Hits 1223 1223
Misses 818 818
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR adds an opt-in/opt-out “lazy Polars” mode to STF data-access functions so callers can choose between returning pl.DataFrame (eager) vs pl.LazyFrame (lazy) and get better type inference via overloads.
Changes:
- Added
lazy: bool = Trueparameter andLiteral-based overloads toget_nhsn_hrdandget_nssp, updating return annotations topl.DataFrame | pl.LazyFrame. - Updated NNH PMF helpers to accept
pl.LazyFrameand introduced overloads for_filter_param_estimates. - Introduced
output = "pl_lazy" if lazy else "pl"paths when loading viadatacat.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
cfa/stf/data/get_data.py |
Adds lazy option + overloads to NHSN HRD and NSSP getters and adjusts datacat loading accordingly. |
cfa/stf/data/get_nnh_pmfs.py |
Extends PMF extraction/filtering to work with LazyFrame and adds overloads for the filtering helper. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
lazy: bool = Trueparameter toget_nhsn_hrdinget_data.pylazy: bool = Trueparameter toget_nsspinget_data.pyget_nhsn_hrdandget_nssptopl.DataFrame | pl.LazyFrame@overloaddecorators toget_nhsn_hrdandget_nsspforLiteral[True]→pl.LazyFrameandLiteral[False]→pl.DataFrameinferencelazy: bool = Trueparameter to_filter_param_estimatesinget_nnh_pmfs.pylazyto_filter_param_estimatesin all three public NNH functions