Skip to content

Move Parquet saving into a separate item list collector#1127

Open
mdekstrand wants to merge 6 commits into
lenskit:mainfrom
mdekstrand:feature/result-collector
Open

Move Parquet saving into a separate item list collector#1127
mdekstrand wants to merge 6 commits into
lenskit:mainfrom
mdekstrand:feature/result-collector

Conversation

@mdekstrand

Copy link
Copy Markdown
Member

This adds ParquetItemListCollector that saves item lists as we go to a Parquet file, and does some smaller type improvements and refactors to the data code.

@mdekstrand mdekstrand added this to the 2026.3 milestone Jun 30, 2026
@mdekstrand mdekstrand added the data Data management support. label Jun 30, 2026
@mdekstrand mdekstrand self-assigned this Jun 30, 2026
@mdekstrand mdekstrand moved this from Backlog to In Progress in LensKit Development Jun 30, 2026
@codecov

codecov Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 20.20202% with 79 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.74%. Comparing base (651740a) to head (dd121cc).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
src/lenskit/data/_collection/_parquet.py 17.77% 37 Missing ⚠️
src/lenskit/batch/_runner.py 0.00% 18 Missing ⚠️
src/lenskit/batch/_results.py 0.00% 9 Missing ⚠️
src/lenskit/data/_collection/_base.py 38.46% 8 Missing ⚠️
src/lenskit/batch/_queries.py 0.00% 6 Missing ⚠️
src/lenskit/batch/__init__.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1127       +/-   ##
===========================================
- Coverage   89.55%   17.74%   -71.81%     
===========================================
  Files         256      209       -47     
  Lines       17227    14485     -2742     
===========================================
- Hits        15428     2571    -12857     
- Misses       1799    11914    +10115     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mdekstrand mdekstrand force-pushed the feature/result-collector branch from b41a6e1 to dd121cc Compare July 9, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Data management support.

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant