Skip to content

Add lazy option to cfa.stf.data functions#1029

Merged
damonbayer merged 13 commits into
mainfrom
copilot/add-lazy-option-to-cfa-stf-data
Apr 16, 2026
Merged

Add lazy option to cfa.stf.data functions#1029
damonbayer merged 13 commits into
mainfrom
copilot/add-lazy-option-to-cfa-stf-data

Conversation

Copilot AI commented Apr 16, 2026

Copy link
Copy Markdown
Contributor
  • Add lazy: bool = True parameter to get_nhsn_hrd in get_data.py
  • Add lazy: bool = True parameter to get_nssp in get_data.py
  • Update return type annotations in get_nhsn_hrd and get_nssp to pl.DataFrame | pl.LazyFrame
  • Add @overload decorators to get_nhsn_hrd and get_nssp for Literal[True]pl.LazyFrame and Literal[False]pl.DataFrame inference
  • Add lazy: bool = True parameter to _filter_param_estimates in get_nnh_pmfs.py
  • Propagate lazy to _filter_param_estimates in all three public NNH functions
  • Run linter to verify changes

Copilot AI linked an issue Apr 16, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Add lazy option to cfa.stf.data Add lazy option to cfa.stf.data functions Apr 16, 2026
Copilot AI requested a review from damonbayer April 16, 2026 19:10
@damonbayer

damonbayer commented Apr 16, 2026

Copy link
Copy Markdown
Collaborator

@copilot Use the @overload decorator so that we can infer whether the result will be a pl.LazyFrame or pl.DataFrame.

Copilot AI commented Apr 16, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Use the @overload decorator so that we can infer whether the result will be a pl.LazyFrame or pl.DataFrame.

Done in 172da56. Added @overload signatures to both get_nhsn_hrd and get_nssplazy: Literal[True] resolves to pl.LazyFrame and lazy: Literal[False] resolves to pl.DataFrame.

@codecov

codecov Bot commented Apr 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.92%. Comparing base (3a556ef) to head (38a6690).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1029   +/-   ##
=======================================
  Coverage   59.92%   59.92%           
=======================================
  Files          31       31           
  Lines        2041     2041           
=======================================
  Hits         1223     1223           
  Misses        818      818           
Flag Coverage Δ
epiautogp 73.18% <ø> (ø)
hewr 76.98% <ø> (ø)
pipelines 56.37% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@damonbayer
damonbayer marked this pull request as ready for review April 16, 2026 20:02
@damonbayer
damonbayer requested a review from Copilot April 16, 2026 20:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an opt-in/opt-out “lazy Polars” mode to STF data-access functions so callers can choose between returning pl.DataFrame (eager) vs pl.LazyFrame (lazy) and get better type inference via overloads.

Changes:

  • Added lazy: bool = True parameter and Literal-based overloads to get_nhsn_hrd and get_nssp, updating return annotations to pl.DataFrame | pl.LazyFrame.
  • Updated NNH PMF helpers to accept pl.LazyFrame and introduced overloads for _filter_param_estimates.
  • Introduced output = "pl_lazy" if lazy else "pl" paths when loading via datacat.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File Description
cfa/stf/data/get_data.py Adds lazy option + overloads to NHSN HRD and NSSP getters and adjusts datacat loading accordingly.
cfa/stf/data/get_nnh_pmfs.py Extends PMF extraction/filtering to work with LazyFrame and adds overloads for the filtering helper.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cfa/stf/data/get_nnh_pmfs.py Outdated
Comment thread cfa/stf/data/get_nnh_pmfs.py
Comment thread cfa/stf/data/get_nnh_pmfs.py
Comment thread cfa/stf/data/get_data.py
Comment thread cfa/stf/data/get_data.py
Comment thread cfa/stf/data/get_data.py
@damonbayer
damonbayer merged commit 88c5a52 into main Apr 16, 2026
23 checks passed
@damonbayer
damonbayer deleted the copilot/add-lazy-option-to-cfa-stf-data branch April 16, 2026 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add lazy option to cfa.stf.data

4 participants