Skip to content

Commit 7650d19

Browse files
irm-codebasepre-commit-ci[bot]sjpfenninger
authored
Feature modelblocks (#45)
* Move to modelblocks * Linting upgrades * pathvars implementation --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Stefan Pfenninger-Lee <stefan@pfenninger.org>
1 parent 8672de4 commit 7650d19

35 files changed

Lines changed: 270 additions & 778 deletions

.github/ISSUE_TEMPLATE/bug_report.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ A clear and concise description of what you expected to happen.
2424
If applicable, add screenshots to help explain your problem.
2525

2626
**Desktop (please complete the following information):**
27-
- OS: [e.g. Linux Fedora 41, Windows 10...]
27+
- OS: [e.g. Linux Fedora 43, Windows 11...]
2828
- Version [e.g. v0.1.1]
2929

3030
**Additional context**

.github/workflows/template-check-version.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,6 @@ jobs:
4949
run: |
5050
gh issue --repo ${{ github.repository }} \
5151
create --title "Template update" \
52-
--body "A new version of the template has been published. Please update this project using \`copier update --skip-answered --defaults\` and review changes. If you need help, refer to the \`clio\` documentation or reach out to the calliope project team."
52+
--body "A new version of the template has been published. Please update this project using \`copier update --skip-answered --defaults\` and review changes. If you need help, refer to the Modelblocks documentation or reach out to the project team."
5353
env:
5454
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/template-pr-ci.yml

Lines changed: 0 additions & 44 deletions
This file was deleted.

.pre-commit-config.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ repos:
1212
# Repo quality
1313
- id: check-added-large-files
1414
args: [--enforce-all]
15+
exclude: ^pixi\.lock$
1516
- id: forbid-submodules
1617
- id: check-case-conflict
1718
- id: check-illegal-windows-names
@@ -37,4 +38,4 @@ repos:
3738
- id: codespell
3839

3940
ci: # https://pre-commit.ci/
40-
autoupdate_schedule: monthly
41+
autoupdate_schedule: quarterly

AUTHORS

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,6 @@ This is the list of contributors for copyright purposes.
22

33
This does not necessarily list everyone who has contributed to this software's
44
code or documentation. For a full contributor list, see:
5-
<https://github.com/calliope-project/data-module-template/graphs/contributors>
5+
<https://github.com/modelblocks-org/data-module-template/graphs/contributors>
66

77
Ivan Ruiz Manuel, TU Delft <i.ruizmanuel@tudelft.nl>

CITATION.cff

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
# https://citation-file-format.github.io/
44
cff-version: 1.2.0
55
message: If you use this software or data produced by it, please cite it using the metadata from this file.
6-
title: clio - data module template
7-
repository: https://github.com/calliope-project/data-module-template
6+
title: Modelblocks - data module template
7+
repository: https://github.com/modelblocks-org/data-module-template
88
license: Apache-2.0
99
authors:
1010
- given-names: Ivan

README.md

Lines changed: 39 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,48 @@
1-
# Data module template
1+
# Modelblocks data module template
22

3-
A template for modular data workflows using [`snakemake`](https://snakemake.readthedocs.io/en/stable/), part of the [`clio`](https://clio.readthedocs.io/) toolset.
3+
A template for modular data workflows using [`snakemake`](https://snakemake.readthedocs.io/en/stable/) workflows, part of the [Modelblocks](https://www.modelblocks.org/) toolset.
4+
5+
> [!TIP]
6+
> Looking for general information on Modelblocks?
7+
> Check the [Modelblocks](https://www.modelblocks.org/) website and our [documentation and guidelines](https://modelblocks.readthedocs.io/en).
48
59
## Resources
610

7-
To familiarise yourself with `clio` data modules:
11+
To familiarise yourself with Modelblocks data modules:
812

913
- Check the auto-generated minimal example. You can find it in `tests/integration/Snakefile`.
10-
- Read about the `clio` approach in [our documentation](https://clio.readthedocs.io/).
1114
- Read about `snakemake` modularisation in [their documentation](https://snakemake.readthedocs.io/en/stable/snakefiles/modularization.html#modules).
1215

16+
## Features
17+
18+
- Standardised layout compliant with the [snakemake workflow catalogue's](https://snakemake.github.io/snakemake-workflow-catalog/#) listing requirements, enabling them to be automatically included in their listings once published. Read more about those requirements [here](https://snakemake.github.io/snakemake-workflow-catalog/docs/catalog.html#standardized-usage-workflows).
19+
- Standardised input-output structure across modules:
20+
- `resources/`: files needed for the module's processes.
21+
- `user/`: files that should be provided by users. Document them well!
22+
- `automatic/`: files that the module downloads or prepares in intermediate steps.
23+
- `results/`: files generated by the module's algorithms that are relevant to the user.
24+
- Pre-made integration setup for your module.
25+
- Continuous Integration (CI) settings, ready for [pre-commit.ci](https://pre-commit.ci/).
26+
- GitHub actions to automate chores during pull requests and releases.
27+
- Premade `pytest` setup.
28+
29+
> [!IMPORTANT]
30+
>
31+
> A few things to be aware of.
32+
>
33+
> - **Modules do not work like regular snakemake workflows**
34+
> - The primary way to test them should be external (calling `module:`, passing resources, and requesting results). Check the pre-made example in `tests/integration` for more info.
35+
> - Internal access (e.g., calling the `all:` rule) may not work, as the module may not have the necessary `resources/` to execute properly.
36+
> - **Please be sure to maintain the following files to ensure Modelblocks compatibility**
37+
> - `INTERFACE.yaml`: a simple description of the module's input/output structure.
38+
> - `config/config.yaml`: a basic functioning example of how to configure this module.
39+
> - `workflow/internal/config.schema.yaml`: the module's configuration schema, used by `snakemake` for [validation](https://snakemake.readthedocs.io/en/stable/snakefiles/configuration.html).
40+
> - `AUTHORS` / `CITATION.cff` / `LICENSE`: licensing and attribution of this module's code and methods.
41+
42+
1343
## How to use this template
1444

15-
We recommend using [`pixi`](https://pixi.sh/) as your package manager. Once installed, do the following:
45+
We require [`pixi`](https://pixi.sh/) as a package manager. Once installed, do the following:
1646

1747
1. Install the templater tool [`copier`](https://copier.readthedocs.io/en/stable/).
1848

@@ -25,16 +55,17 @@ We recommend using [`pixi`](https://pixi.sh/) as your package manager. Once inst
2555
We recommend you use the module name as the directory name.
2656

2757
```shell
28-
copier copy https://github.com/calliope-project/data-module-template.git ./path/to/<module_name>
58+
copier copy https://github.com/modelblocks-org/data-module-template.git ./<module_name>
2959
```
3060

31-
If your terminal does not have access to `copier` then you may need to update your `PATH` variable to include `~/.pixi/bin`.
61+
> [!TIP]
62+
> If your terminal does not have access to `copier` then you may need to update your `PATH` variable to include `~/.pixi/bin`.
3263
3364
3. Answer some questions so can we pre-fill licensing, citation files, etc...
3465
4. Initialise the `pixi` project environment of your new module.
3566

3667
```shell
37-
cd ./path/to/<module_name> # navigate to the new project
68+
cd ./<module_name> # navigate to the new project
3869
pixi install --all # install the project environment
3970
```
4071

@@ -44,30 +75,3 @@ We recommend using [`pixi`](https://pixi.sh/) as your package manager. Once inst
4475
cd tests/integration # go to the integration test...
4576
pixi run snakemake --use-conda # run it!
4677
```
47-
48-
## Features
49-
50-
- Standardised layout compliant with the [snakemake workflow catalogue's](https://snakemake.github.io/snakemake-workflow-catalog/#) listing requirements, enabling them to be automatically included in their listings once published. Read more about those requirements [here](https://snakemake.github.io/snakemake-workflow-catalog/docs/catalog.html#standardized-usage-workflows).
51-
- Standardised input/output structure across modules:
52-
- `resources/`: files needed for the module's processes.
53-
- `user/`: files that should be provided by users. Document them well!
54-
- `automatic/`: files that the module downloads or prepares in intermediate steps.
55-
- `results/`: files generated by the module's algorithms that are relevant to the user.
56-
- Pre-made integration setup for your module.
57-
- Continuous Integration (CI) settings, ready for [pre-commit.ci](https://pre-commit.ci/).
58-
- GitHub actions to automate chores during pull requests and releases.
59-
- Premade `pytest` setup.
60-
61-
> [!IMPORTANT]
62-
>
63-
> A few things to be aware of.
64-
>
65-
> - **Modules do not work like regular snakemake workflows**
66-
> - The primary way to test them should be external (calling `module:`, passing resources, and requesting results). Check the pre-made example in `tests/integration` for more info.
67-
> - Internal access (e.g., calling the `all:` rule) may not work, as the module may not have the necessary `resources/` to execute properly.
68-
> - **Please be sure to maintain the following files to ensure `clio` compatibility**
69-
> - These are:
70-
> - `INTERFACE.yaml`: a simple description of the module's input/output structure.
71-
> - `config/config.yaml`: a basic functioning example of how to configure this module.
72-
> - `workflow/internal/config.schema.yaml`: the module's configuration schema, used by `snakemake` for [validation](https://snakemake.readthedocs.io/en/stable/snakefiles/configuration.html).
73-
> - `AUTHORS` / `CITATION.cff` / `LICENSE`: licensing and attribution of this module's code and methods.

copier.yaml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,8 @@ _exclude:
2121
_skip_if_exists:
2222
# Respect user changes, except for a few standard files.
2323
- config/**
24-
- docs/**
2524
- resources/**
25+
- figures/**
2626
- results/**
2727
- tests/**
2828
- workflow/**
@@ -32,7 +32,7 @@ _skip_if_exists:
3232
- INTERFACE.yaml
3333
# Special cases: these files should not be user-modified, even though they are in user-modifiable directories
3434
- "!config/README.md"
35-
- "!tests/clio_test.py"
35+
- "!tests/integration_test.py"
3636
_templates_suffix: .jinja
3737
_answers_file: .copier-answers.yml
3838

@@ -67,12 +67,12 @@ github_org:
6767
type: str
6868
help: >-
6969
Please provide the name of the github account or organisation where
70-
this module will be created (e.g., 'calliope-project' in 'github.com/calliope-project').
70+
this module will be created (e.g., 'modelblocks-org' in 'github.com/modelblocks-org').
7171
validator: >-
7272
{% if not (github_org | regex_search('^[a-zA-Z0-9][a-zA-Z0-9-]*$')) %}
7373
"Only lowercase letters, digits, hyphens and underscores are valid."
7474
{% endif %}
75-
default: "calliope-project"
75+
default: "modelblocks-org"
7676
license:
7777
type: str
7878
help: "Please choose a license."

mypy.ini

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
[mypy]
22
disable_error_code = import-untyped
33
exclude = (^|/)\.(snakemake|pixi)(/|$)
4+
exclude_gitignore = True

0 commit comments

Comments
 (0)