Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ The versioning pattern is `YYYY.MM.DD.micro(a/b/{none if release})

---

## [2025.11.18.0a]

### Updated

- `docs/data_user_guide.md` and added cli documentation to mkdocs nav.

## [2025.11.13.0a]

### Added
Expand Down
67 changes: 61 additions & 6 deletions docs/data_user_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,12 @@ df = datacat.private.scenarios.covid19vax_trends.load.get_dataframe(
)
```

In order to see what versions are available, use the data catalog's convenient namespace methods:
To see what versions are available, use the data catalog's convenient namespace methods:

```python
>>> from cfa.dataops import datacat
>>> # these follow hierarchical naming created using the dataset
>>> # config TOML, so extract or load are the makes assigned to
>>> # These follow hierarchical naming created using the dataset
>>> # config TOML, so extract or load are the names assigned to
>>> # raw or transformed datasets per the get_data function
>>> datacat.private.scenarios.covid19vax_trends.load.get_versions()
['2025-06-03T17-59-16',
Expand Down Expand Up @@ -94,13 +94,68 @@ vax_df = datacat.private.scenarios.covid19vax_trends.load.get_dataframe()
raw_vax = datacat.private.scenarios.covid19vax_trends.extract.get_dataframe()
```

### Seroprevalence Data
### Fetching Versions within a Range

When `get_dataframe(...)` completes successfully, it prints a short confirmation line like:

Used version(s): [...]

Examples:

```python
from cfa.dataops import datacat

# Get as polars DataFrame
sero_df = datacat.private.scenarios.seroprevalence.load.get_dataframe(output="polars")
# newest match in the range (single version)
df = datacat.private.scenarios.covid19vax_trends.load.get_dataframe(
version=">=2025-05-01,<2025-06-01"
)
```

*Console output (example):*
```
Used version(s): '2025-05-30T19-55-51'
```

```python
# oldest match in the same range (newest=False)
df_old = datacat.private.scenarios.covid19vax_trends.load.get_dataframe(
version=">=2025-05-01,<2025-06-01", newest=False
)
```

*Console output (example):*
```
Used version(s): '2025-05-30T14-50-36'
```

```python
# Fetch all matches and concatenate all tables
df_v = datacat.private.scenarios.covid19vax_trends.load.get_dataframe(
version=">=2025-05-01,<2025-06-01",
newest=None
)
```

*Console output (example):*
```
Used version(s): ['2025-05-30T19-55-51', '2025-05-30T14-50-36']
```

Use the helper `version_matcher` (from `cfa.dataops.utils`) to experiment with version boundary logic to see what matches occur prior to loading large datasets into memory.

```python
>>> from cfa.dataops.utils import version_matcher
>>> available_versions = ['1.0', '1.1', '1.2', '2.0']
>>> version_matcher('>=1.1,<2.0', available_versions)
'1.2'
>>> version_matcher('>=1.1,<2.0', available_versions, newest=False)
'1.1'
>>> version_matcher('latest', available_versions)
'2.0'
>>> version_matcher('~=1', available_versions)
'1.2'
>>> version_matcher('>=1.1,<2.0', available_versions, newest=None)
['1.2', '1.1']
```

## Common Issues
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ nav:
- data_user_guide.md
- data_developer_guide.md
- report_generation.md
- cli_tools.md

repo_url: https://github.com/cdcgov/cfa-dataops
repo_name: cfa-dataops
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "cfa.dataops"
version = "2025.11.13.0a"
version = "2025.11.18.0a"
description = "Data cataloging, ETL, modeling, verification, and validation for CFA"
authors = [
{ name = "Phil Rogers", email = "ap66@cdc.gov" },
Expand Down