Skip to content

Fix #2812: Drop HDF5 support#3138

Merged
benjeffery merged 2 commits into
tskit-dev:mainfrom
hossam26644:remove-legacy-h5py
Apr 29, 2025
Merged

Fix #2812: Drop HDF5 support#3138
benjeffery merged 2 commits into
tskit-dev:mainfrom
hossam26644:remove-legacy-h5py

Conversation

@hossam26644
Copy link
Copy Markdown
Contributor

No description provided.

@hossam26644
Copy link
Copy Markdown
Contributor Author

@benjeffery please give it a look; make sure that I did not leave unnecessary tests (that do not fail)

@hossam26644 hossam26644 marked this pull request as ready for review April 10, 2025 15:04
@hossam26644
Copy link
Copy Markdown
Contributor Author

@benjeffery I also updated the license year in some files to 2025 (maybe it should have been a separate PR?)

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.58%. Comparing base (35f52cb) to head (b0f4f9a).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3138      +/-   ##
==========================================
- Coverage   89.72%   89.58%   -0.14%     
==========================================
  Files          29       28       -1     
  Lines       32225    31841     -384     
  Branches     5890     5849      -41     
==========================================
- Hits        28913    28524     -389     
- Misses       1882     1887       +5     
  Partials     1430     1430              
Flag Coverage Δ
c-tests 86.66% <ø> (ø)
lwt-tests 80.38% <ø> (ø)
python-c-tests 88.14% <ø> (-0.10%) ⬇️
python-tests 98.80% <ø> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
c/tskit/core.c 95.50% <ø> (ø)
c/tskit/core.h 100.00% <ø> (ø)
python/_tskitmodule.c 88.14% <ø> (-0.10%) ⬇️
python/tskit/__init__.py 100.00% <ø> (ø)
python/tskit/cli.py 96.66% <ø> (-0.28%) ⬇️
python/tskit/util.py 99.25% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@benjeffery benjeffery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!
Can we make the changelog line a bit more descriptive to include a mention of "legacy formats from msprime version blah"

The metadata tests shouldn't be removed, just remove the HDF5 from the name.

Copy link
Copy Markdown
Member

@jeromekelleher jeromekelleher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. We should do a search for h5py across the full repo to make sure we've cleared it out from all the requirements files etc as well.

Comment thread python/tskit/util.py Outdated
"may have been generated by msprime < 0.6.0 (June 2018) which "
"can no longer be read directly. Please convert to the new "
"kastore format using the ``tskit upgrade`` command."
"kastore format using the ``tskit upgrade`` command from tskit version 0.6.2"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we removed the "upgrade" CLI then also? I'm not sure what this would do now

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, yes. Now tskit upgrade is totally removed

Comment thread python/CHANGELOG.rst Outdated
the original schema instead (:user:`benjeffery`, :issue:`3129`, :pr:`3130`)

**Breaking Changes**
- HDF5 support is dropped (:user:`hossam26644`, :issue:`2812`, :pr:`3138`)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something a bit more descriptive might be helpful here.

@hossam26644
Copy link
Copy Markdown
Contributor Author

@benjeffery Ok, I put back the metadataroundtrip testing. It was not converting to and from HDF5 using the tskit apis; was it actually doing any conversions?

@benjeffery
Copy link
Copy Markdown
Member

@benjeffery Ok, I put back the metadataroundtrip testing. It was not converting to and from HDF5 using the tskit apis; was it actually doing any conversions?

No, the name was a relic from when the file format was HDF5 only.

@hossam26644
Copy link
Copy Markdown
Contributor Author

@benjeffery ready for another review round

@benjeffery
Copy link
Copy Markdown
Member

Almost there, we need to remove h5py from all requirements files.

@jeromekelleher
Copy link
Copy Markdown
Member

Nice, shedding a bunch of baggage here!

@benjeffery
Copy link
Copy Markdown
Member

Great! Just a squash now and we should be good to go.

@benjeffery benjeffery enabled auto-merge April 28, 2025 23:39
@benjeffery benjeffery added this pull request to the merge queue Apr 29, 2025
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 29, 2025
@benjeffery benjeffery enabled auto-merge April 29, 2025 08:42
@benjeffery benjeffery added this pull request to the merge queue Apr 29, 2025
Merged via the queue into tskit-dev:main with commit b6d7eab Apr 29, 2025
18 of 19 checks passed
@benjeffery
Copy link
Copy Markdown
Member

Woop, first PR merged with the new system. This new system gives us a rebase button on PRs too so no need to message to trigger that anymore.

@hyanwong
Copy link
Copy Markdown
Member

Also getting rid of HDF5 - hooray.

@hossam26644
Copy link
Copy Markdown
Contributor Author

🎉 🎊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants