Skip to content

Commit 535f856

Browse files
rmnldwgrmnldwg
authored
Merge pull request #3 from lycosystem/release-0.3.0
Release 0.3.0
2 parents fda8245 + 07207ef commit 535f856

5 files changed

Lines changed: 190 additions & 58 deletions

File tree

‎.github/workflows/release.yml‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
name: Build
2+
3+
on:
4+
release:
5+
types: [ created ]
6+
7+
workflow_dispatch:
8+
9+
jobs:
10+
build:
11+
name: Build package from source
12+
runs-on: ubuntu-latest
13+
14+
steps:
15+
- uses: actions/checkout@v4
16+
with:
17+
persist-credentials: false
18+
fetch-depth: 0
19+
- name: Install Python 3
20+
uses: actions/setup-python@v5
21+
with:
22+
python-version: '3.10'
23+
- name: Install build tools
24+
run: |
25+
python3 -m pip install build --user
26+
- name: Build package
27+
run: |
28+
python3 -m build
29+
- name: Upload to CI runner
30+
uses: actions/upload-artifact@v4
31+
with:
32+
name: built-package
33+
path: dist/
34+
35+
pypi-publish:
36+
name: Publish built package on PyPI
37+
runs-on: ubuntu-latest
38+
needs:
39+
- build
40+
41+
# Specifying a GitHub environment is optional, but strongly encouraged
42+
environment:
43+
name: pypi
44+
url: https://pypi.org/p/lydata
45+
permissions:
46+
# IMPORTANT: this permission is mandatory for Trusted Publishing
47+
id-token: write
48+
steps:
49+
# retrieve your distributions here
50+
- name: Download from CI runner
51+
uses: actions/download-artifact@v4
52+
with:
53+
name: built-package
54+
path: dist/
55+
- name: Publish on PyPI
56+
uses: pypa/gh-action-pypi-publish@release/v1

‎.github/workflows/testpypi.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: Test Build
2+
3+
on:
4+
pull_request:
5+
branches: [ main ]
6+
workflow_dispatch:
7+
8+
jobs:
9+
build:
10+
name: Build package from source
11+
runs-on: ubuntu-latest
12+
13+
steps:
14+
- uses: actions/checkout@v4
15+
with:
16+
persist-credentials: false
17+
fetch-depth: 0
18+
- name: Install Python 3
19+
uses: actions/setup-python@v5
20+
with:
21+
python-version: '3.10'
22+
- name: Install build tools
23+
run: |
24+
python3 -m pip install build --user
25+
- name: Build package
26+
run: |
27+
python3 -m build
28+
- name: Upload to CI runner
29+
uses: actions/upload-artifact@v4
30+
with:
31+
name: built-package
32+
path: dist/
33+
34+
testpypi-publish:
35+
name: Publish built package on TestPyPI
36+
runs-on: ubuntu-latest
37+
needs:
38+
- build
39+
40+
# Specifying a GitHub environment is optional, but strongly encouraged
41+
environment:
42+
name: testpypi
43+
url: https://test.pypi.org/p/lydata
44+
permissions:
45+
# IMPORTANT: this permission is mandatory for Trusted Publishing
46+
id-token: write
47+
steps:
48+
# retrieve your distributions here
49+
- name: Download from CI runner
50+
uses: actions/download-artifact@v4
51+
with:
52+
name: built-package
53+
path: dist/
54+
- name: Publish on PyPI
55+
uses: pypa/gh-action-pypi-publish@release/v1
56+
with:
57+
repository-url: https://test.pypi.org/legacy/

.github/workflows/tests.yaml renamed to .github/workflows/tests.yml

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ on:
1010

1111
jobs:
1212
test:
13+
name: Run tests & report coverage
1314
runs-on: ubuntu-latest
1415
permissions:
1516
pull-requests: write
@@ -41,11 +42,22 @@ jobs:
4142
run: pytest --cov=lydata --cov-config=pyproject.toml --doctest-modules src
4243
env:
4344
COVERAGE_FILE: .coverage.doctests
45+
GITHUB_TOKEN: ${{ secrets.LYCOSYSTEM_READALL }}
46+
47+
# Lastly, we can make sure the examples in the README.md are up to date.
48+
# Because doctests can occur in any text file, we can run it over those examples
49+
# as well:
50+
- name: Test README.md examples
51+
if: success() || failure()
52+
run: coverage run -m doctest README.md
53+
env:
54+
COVERAGE_FILE: .coverage.readme
55+
GITHUB_TOKEN: ${{ secrets.LYCOSYSTEM_READALL }}
4456

4557
# Lastly, we collect all files that start with `.coverage` into one file and
4658
# create a report either as a comment on the PR or in a separate branch if its
4759
# a commit to the main branch. From that branch we can put badges and coverage
48-
# reports into e.g. our main README.md
60+
# reports into e.g. our main README.md
4961
- name: Add coverage comment
5062
if: success() || failure() # run these even if previous step fails
5163
uses: py-cov-action/python-coverage-comment-action@v3

‎CHANGELOG.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,26 @@
22

33
All notable changes to this project will be documented in this file.
44

5+
## [0.3.0] - 2025-06-26
6+
7+
Th Python package `lydata` is now pulled out of the data repository [lyDATA](https://github.com/lycosystem/lydata) and will be maintained in the repository [lydata-package](https://github.com/lycosystem/lydata-package).
8+
9+
This required some changes, as the data now cannot be easily shipped with the package. Instead, all data mustalways be fetched from the remote repository.
10+
11+
### 💼 Other
12+
13+
- [**breaking**] Use github by default to fetch datasets
14+
- Fetching from GitHub now works without authentication/token, too
15+
16+
### 🧪 Testing
17+
18+
- Convert examples in `README.md` to valid doctests
19+
20+
### ⚙️ Miscellaneous Tasks
21+
22+
- Run doctest over `README.md` examples during automated tests
23+
- Add build & publish workflows (both PyPI and TestPyPI)
24+
525
## [0.2.5] - 2025-02-05
626

727
### 🐛 Bug Fixes
@@ -253,6 +273,7 @@ Initial implementation of the lyDATA library.
253273
<!-- generated by git-cliff -->
254274
<!-- markdownlint-disable-file MD024 -->
255275

276+
[0.3.0]: https://github.com/lycosystem/lydata/compare/8ae13..0.3.0
256277
[0.2.5]: https://github.com/lycosystem/lydata/compare/0.2.4..0.2.5
257278
[0.2.4]: https://github.com/lycosystem/lydata/compare/0.2.3..0.2.4
258279
[0.2.3]: https://github.com/lycosystem/lydata/compare/0.2.2..0.2.3

‎README.md‎

Lines changed: 43 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
# Python Library for Loading and Manipulating lyDATA Tables
22

3-
[![Build](https://github.com/lycosystem/lydata/actions/workflows/build.yml/badge.svg)](https://github.com/lycosystem/lydata/actions/workflows/build.yml)
4-
[![Tests](https://github.com/lycosystem/lydata/actions/workflows/tests.yml/badge.svg)](https://github.com/lycosystem/lydata/actions/workflows/tests.yml)
5-
[![Documentation Status](https://readthedocs.org/projects/lydata/badge/?version=stable)](https://lydata.readthedocs.io/en/stable/?badge=stable)
3+
[![Build](https://github.com/lycosystem/lydata-package/actions/workflows/release.yml/badge.svg)](https://github.com/lycosystem/lydata-package/actions/workflows/release.yml)
4+
[![Tests](https://github.com/lycosystem/lydata-package/actions/workflows/tests.yml/badge.svg)](https://github.com/lycosystem/lydata-package/actions/workflows/tests.yml)
5+
[![Documentation Status](https://readthedocs.org/projects/lydata/badge/?version=stable)](https://lydata.readthedocs.io/stable/?badge=stable)
6+
[![Coverage badge](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/lycosystem/lydata-package/python-coverage-comment-action-data/endpoint.json)](https://htmlpreview.github.io/?https://github.com/lycosystem/lydata-package/blob/python-coverage-comment-action-data/htmlcov/index.html)
67

78
This repository provides a Python library for loading, manipulating, and validating the datasets available on [lyDATA](https://github.com/lycosystem/lydata).
89

@@ -36,72 +37,57 @@ pip install -e .
3637
The first and most common use case would probably listing and loading the published datasets:
3738

3839
```python
39-
import lydata
40-
41-
for dataset_spec in lydata.available_datasets(
42-
year=2023, # show all datasets added in 2023
43-
use_github=True, # do not search on disk, but rather on GitHub
44-
ref="61a17e", # may be some specific hash/tag/branch
45-
):
46-
print(dataset_spec.name)
47-
48-
# output:
49-
# 2023-clb-multisite
50-
# 2023-isb-multisite
51-
52-
first_dataset = next(lydata.load_datasets(
53-
subsite="oropharynx", # merge data that include oropharyngeal tumor patients
54-
use_github=True, # again, search GitHub, not on disk (which is the default)
55-
))
56-
print(first_dataset.head())
57-
58-
# output:
59-
# patient ... positive_dissected
60-
# # ... contra
61-
# id institution sex ... III IV V
62-
# 0 P011 Centre Léon Bérard male ... 0.0 0.0 0.0
63-
# 1 P012 Centre Léon Bérard female ... 0.0 0.0 0.0
64-
# 2 P014 Centre Léon Bérard male ... 0.0 0.0 NaN
65-
# 3 P015 Centre Léon Bérard male ... 0.0 0.0 NaN
66-
# 4 P018 Centre Léon Bérard male ... NaN NaN NaN
67-
#
68-
# [5 rows x 82 columns]
40+
>>> import lydata
41+
>>> for dataset_spec in lydata.available_datasets(
42+
... year=2023, # show all datasets added in 2023
43+
... ref="61a17e", # may be some specific hash/tag/branch
44+
... ):
45+
... print(dataset_spec.name)
46+
2023-clb-multisite
47+
2023-isb-multisite
48+
49+
# return generator of datasets that include oropharyngeal tumor patients
50+
>>> first_dataset = next(lydata.load_datasets(subsite="oropharynx"))
51+
>>> print(first_dataset.head())
52+
... # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE
53+
patient ... positive_dissected
54+
# ... contra
55+
id institution sex ... III IV V
56+
0 P011 Centre Léon Bérard male ... 0.0 0.0 0.0
57+
1 P012 Centre Léon Bérard female ... 0.0 0.0 0.0
58+
2 P014 Centre Léon Bérard male ... 0.0 0.0 NaN
59+
3 P015 Centre Léon Bérard male ... 0.0 0.0 NaN
60+
4 P018 Centre Léon Bérard male ... NaN NaN NaN
61+
[5 rows x 82 columns]
62+
6963
```
7064

7165
And since the three-level header of the tables is a little unwieldy at times, we also provide some shortcodes via a custom pandas accessor. As soon as `lydata` is imported it can be used like this:
7266

7367
```python
74-
print(first_dataset.ly.age)
75-
76-
# output:
77-
# 0 67
78-
# 1 62
79-
# ..
80-
# 261 60
81-
# 262 60
82-
# Name: (patient, #, age), Length: 263, dtype: int64
68+
>>> print(first_dataset.ly.age)
69+
... # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE
70+
0 67
71+
1 62
72+
...
73+
261 60
74+
262 60
75+
Name: (patient, #, age), Length: 263, dtype: int64
76+
8377
```
8478

8579
And we have implemented `Q` and `C` objects inspired by Django that allow easier querying of the tables:
8680

8781
```python
88-
from lydata import C
82+
>>> from lydata import C
8983

9084
# select patients younger than 50 that are not HPV positive (includes NaNs)
91-
query_result = first_dataset.ly.query((C("age") < 50) & ~(C("hpv") == True))
92-
print(query_result)
93-
94-
# output:
95-
# patient ... positive_dissected
96-
# # ... contra
97-
# id institution sex age ... II III IV V
98-
# 2 P014 Centre Léon Bérard male 43 ... 1.0 0.0 0.0 NaN
99-
# 7 P024 Centre Léon Bérard male 45 ... NaN NaN NaN NaN
100-
# .. ... ... ... .. ... ... ... ... ...
101-
# 212 P270 Centre Léon Bérard male 47 ... 0.0 0.0 0.0 NaN
102-
# 217 P275 Centre Léon Bérard male 49 ... 0.0 0.0 0.0 NaN
103-
#
104-
# [13 rows x 82 columns]
85+
>>> query_result = first_dataset.ly.query((C("age") < 50) & ~(C("hpv") == True))
86+
>>> (query_result.ly.age < 50).all()
87+
np.True_
88+
>>> (query_result.ly.hpv == False).all()
89+
np.True_
90+
10591
```
10692

10793
For more details and further examples or use-cases, have a look at the [official documentation](https://lydata.readthedocs.org/)

0 commit comments

Comments
 (0)