Skip to content

Commit 8721c53

Browse files
committed
docs(developer_guide): refine developer guide
1 parent ed5dcaf commit 8721c53

8 files changed

Lines changed: 121 additions & 400 deletions

File tree

docs/developer_guide/ci_cd.md

Lines changed: 21 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -16,55 +16,23 @@ DeepTab uses GitHub Actions for continuous integration and delivery. All workflo
1616

1717
## ci.yml (continuous integration)
1818

19-
Runs on every push to `main` and every pull request targeting `main`. Cancels in-progress runs for the same branch via `concurrency`.
19+
Runs on every push to `main` and every pull request targeting `main`. In-progress runs for the same branch are cancelled via `concurrency`.
2020

21-
### Jobs
21+
| Job | Runner / Python | What it does | Depends on |
22+
| ----------- | --------------- | ------------------------------------------ | ---------- |
23+
| `lint` | ubuntu / 3.10 | `ruff check .` and `ruff format --check .` | - |
24+
| `typecheck` | ubuntu / 3.10 | `pyright` | - |
25+
| `build` | ubuntu / 3.10 | `poetry build` and `twine check dist/*` | - |
26+
| `tests` | full matrix | `pytest tests/ -v` | - |
27+
| `smoke` | ubuntu / 3.12 | `pytest tests/ -m smoke --tb=short` | `lint` |
28+
| `coverage` | ubuntu / 3.12 | branch coverage uploaded to Codecov | `tests` |
2229

23-
**`lint`** runs on `ubuntu-latest` / Python 3.10:
30+
The `lint`, `typecheck`, `build`, and `tests` jobs run in parallel with `fail-fast: false`, so one failing matrix cell does not cancel the rest. The `tests` matrix covers every supported Python and OS combination listed in the [Support Matrix](support_matrix.md).
2431

25-
```bash
26-
ruff check . # style and correctness
27-
ruff format --check . # formatting (no changes applied)
28-
```
29-
30-
**`typecheck`** runs on `ubuntu-latest` / Python 3.10:
31-
32-
```bash
33-
pyright
34-
```
35-
36-
**`build`** runs on `ubuntu-latest` / Python 3.10:
37-
38-
```bash
39-
poetry build
40-
twine check dist/*
41-
```
42-
43-
**`tests`** runs across a full matrix:
44-
45-
| Dimension | Values |
46-
| --------- | ------------------------------------------------- |
47-
| OS | `ubuntu-latest`, `macos-latest`, `windows-latest` |
48-
| Python | `3.10`, `3.11`, `3.12`, `3.13` |
49-
50-
```bash
51-
pytest tests/ -v
52-
```
53-
54-
**`smoke`** runs on `ubuntu-latest` / Python 3.12 after `lint` passes. It runs only the fast sanity-check tests marked with `@pytest.mark.smoke`:
55-
56-
```bash
57-
pytest tests/ -v -m smoke --tb=short
32+
```{note}
33+
The two Python versions are intentional. Compatibility-sensitive jobs (`lint`, `typecheck`, `build`) pin to the **lowest** supported version (3.10) so they catch code that relies on newer syntax or stdlib than we promise to support. The fast single-version gates (`smoke`, `coverage`) use a recent version (3.12), since they check behavior rather than compatibility. Full compatibility is verified by the `tests` matrix.
5834
```
5935

60-
**`coverage`** runs on `ubuntu-latest` / Python 3.12 after `tests` pass. It measures branch coverage and uploads the report to [Codecov](https://codecov.io/gh/OpenTabular/DeepTab):
61-
62-
```bash
63-
pytest tests/ --cov=deeptab --cov-branch --cov-report=xml:coverage.xml -q
64-
```
65-
66-
The `lint`, `typecheck`, `build`, and `tests` jobs are independent and run in parallel, with `fail-fast: false` so a failure in one matrix cell does not cancel the others. The `smoke` job depends on `lint`, and `coverage` depends on `tests`.
67-
6836
---
6937

7038
## docs.yml (documentation build)
@@ -95,42 +63,21 @@ A `workflow_dispatch`-only workflow. Builds the package with Poetry and validate
9563

9664
---
9765

98-
## publish-testpypi.yml (release candidate publishing)
66+
## Publishing (publish-testpypi.yml / publish-pypi.yml)
9967

100-
Triggered by any tag matching `v*.*.*rc*`. Uses [OIDC trusted publishing](https://docs.pypi.org/trusted-publishers/), so no `PYPI_TOKEN` secret is required.
68+
Both publish jobs use [OIDC trusted publishing](https://docs.pypi.org/trusted-publishers/), so no `PYPI_TOKEN` secret is required. Each builds with `poetry build`, publishes, then creates a GitHub release.
10169

102-
Steps:
70+
| Workflow | Tag pattern | Target | Release type |
71+
| ---------------------- | ------------------ | -------------------------------------------------- | ------------ |
72+
| `publish-testpypi.yml` | `v*.*.*rc*` | [TestPyPI](https://test.pypi.org/project/deeptab/) | Pre-release |
73+
| `publish-pypi.yml` | `v*.*.*` (no `rc`) | [PyPI](https://pypi.org/project/deeptab/) | Release |
10374

104-
1. Build the package with `poetry build`.
105-
2. Publish to [TestPyPI](https://test.pypi.org/project/deeptab/).
106-
3. Create a GitHub pre-release via `gh release create`.
107-
108-
The `pypi-publish` GitHub Environment is required; it must have the `v*rc*` tag pattern in its protection rules.
109-
110-
---
111-
112-
## publish-pypi.yml (stable release publishing)
113-
114-
Triggered by any tag matching `v*.*.*` that does **not** contain `rc` (stable only). Also uses OIDC trusted publishing.
115-
116-
Steps:
117-
118-
1. Build the package with `poetry build`.
119-
2. Publish to [PyPI](https://pypi.org/project/deeptab/).
120-
3. Create a GitHub release with auto-generated release notes.
121-
122-
See the [Release process](release.md) page for the full end-to-end procedure including when and how to push these tags.
75+
The `pypi-publish` GitHub Environment must allow the matching tag pattern in its protection rules. See the [Release process](release.md) page for when and how to push these tags.
12376

12477
---
12578

12679
## Adding or modifying a workflow
12780

12881
1. Edit the relevant YAML file in `.github/workflows/`.
129-
2. Use [act](https://github.com/nektos/act) to test locally before pushing:
130-
131-
```bash
132-
act push --job tests
133-
```
134-
135-
3. Keep job names consistent, since they are displayed in PR status checks and on the Actions tab.
136-
4. Pin third-party actions to a full commit SHA or a tagged version (e.g. `actions/checkout@v4`) and keep them up to date via `just update` (which runs `pre-commit autoupdate`).
82+
2. Keep job names consistent, since they appear in PR status checks and on the Actions tab.
83+
3. Pin third-party actions to a tagged version or full commit SHA (e.g. `actions/checkout@v4`) and keep them current via `just update` (which runs `pre-commit autoupdate`).
Lines changed: 42 additions & 121 deletions
Original file line numberDiff line numberDiff line change
@@ -1,153 +1,78 @@
11
# Contribution Guidelines
22

3-
Thank you for considering contributing to our Python package! We appreciate your time and effort in helping us improve our project. Please take a moment to review the following guidelines to ensure a smooth and efficient contribution process.
3+
Thanks for contributing to DeepTab. This page covers environment setup, the local workflow, and what a pull request needs to pass review.
44

55
## Code of Conduct
66

7-
We kindly request all contributors to adhere to our Code of Conduct when participating in this project. It outlines our expectations for respectful and inclusive behavior within the community.
7+
All contributors are expected to follow the project [Code of Conduct](https://github.com/OpenTabular/DeepTab/blob/main/CODE_OF_CONDUCT.md), which sets the standard for respectful and inclusive participation.
88

9-
## Setting Up Development Environment
9+
## Setting up the development environment
1010

11-
Before you start contributing to the project, you need to set up your development environment. This will allow you to make changes to the codebase, run tests, and build the documentation locally. The project uses `poetry` for dependency management and packaging. Along with that, `ruff` is used for source code formatting and linting.
11+
The project uses [Poetry](https://python-poetry.org/docs/) for dependency management and the [just](https://just.systems/man/en/) command runner for common tasks (`justfile` defines testing, building, and formatting).
1212

13-
To set up the development environment for this Python package, follow these steps:
13+
1. Clone the repository:
1414

15-
1. Clone the repository to your local machine using the command:
16-
17-
```
15+
```bash
1816
git clone https://github.com/OpenTabular/DeepTab
19-
2017
cd DeepTab
2118
```
2219

23-
2. Install tools required for setting up development environment:
24-
25-
- Install `poetry` for dependency management and packaging. You can install it using the following command or refer to the [official documentation](https://python-poetry.org/docs/) for more information.
26-
27-
```
28-
pip install poetry
29-
```
30-
31-
- Install `just` command runner. You can install it using the following command or refer to the [official documentation](https://just.systems/man/en/) for more information.
20+
2. Install the prerequisites: `pip install poetry` and `just` (see the [just install guide](https://just.systems/man/en/packages.html), e.g. `brew install just`).
3221

33-
`justfile` in the source directory is used to define and run common tasks like testing, building, and formatting the codebase.
34-
35-
3. In case you are able to successfully install `poetry` and `just`, you can run the following command to install the dependencies and set up the development environment:
36-
37-
```
38-
# it will install the dependencies as defined in the pyproject.toml file
39-
# it will also install the pre-commit hooks
22+
3. Install dependencies and register the pre-commit hooks:
4023

24+
```bash
4125
just install
4226
```
4327

44-
In case you are not able to install `just`, you can follow the below steps to set up the development environment:
45-
46-
```
47-
cd DeepTab
28+
Without `just`, run the same steps directly:
4829

30+
```bash
4931
poetry install
50-
5132
poetry run pre-commit install --hook-type commit-msg --hook-type pre-commit --hook-type pre-push
5233
```
5334

54-
If you need to update the documentation, please install the documentation dependencies:
35+
To work on the docs, also install the docs group with `poetry install --with docs`.
5536

56-
```bash
57-
# Recommended: install via the docs dependency group
58-
poetry install --with docs
59-
60-
# Alternative: install directly
61-
pip install -r docs/requirements_docs.txt
62-
```
63-
64-
**Note:** You can also set up a virtual environment to isolate your development environment.
65-
66-
## How to Contribute
67-
68-
1. Create a new branch from `main` for your contributions. Please use descriptive and concise branch names.
69-
2. Make your desired changes or additions to the codebase.
70-
3. Ensure that your code adheres to [PEP8](https://peps.python.org/pep-0008/) coding style guidelines.
71-
4. Write appropriate tests for your changes and verify they pass:
72-
73-
```bash
74-
just test
75-
```
37+
## How to contribute
7638

77-
5. Update the documentation and examples, if necessary.
78-
6. Build the HTML documentation and verify it works as expected:
39+
1. Branch off `main` with a short, descriptive name.
40+
2. Make your changes, keeping each pull request to a single logical focus.
41+
3. Add or update tests, and run the full check suite locally before pushing:
7942

8043
```bash
81-
just docs
44+
just test # full suite with coverage
45+
just check # lint, format, type-check, all pre-commit hooks (what CI runs)
46+
just docs # build HTML docs (warnings treated as errors)
8247
```
8348

84-
Verify the output under `docs/_build/html/`, where `index.html` is the entry point.
85-
86-
7. Run the full local check suite before pushing (lint, format, type-check, and all pre-commit hooks):
87-
88-
```bash
89-
just check
90-
```
91-
92-
If `ruff-format` modifies any files, commit those changes before pushing:
93-
94-
```bash
95-
git add -u
96-
git commit -m "style: apply ruff formatting"
97-
```
98-
99-
8. Commit your changes following the Conventional Commits specification (see below):
100-
101-
```bash
102-
just commit
103-
```
104-
105-
9. Submit a pull request from your branch to `main` in the original repository.
106-
10. Wait for the maintainers to review your pull request. Address any feedback or comments if required.
107-
11. Once approved, your changes will be merged into the main codebase.
49+
4. Commit using Conventional Commits via `just commit`. If `just check` reformats files, commit those separately with `style: apply ruff formatting`.
50+
5. Open a pull request to `main`, reference any related issues, and address review feedback until approved and merged.
10851

10952
## Pre-commit Hooks
11053

111-
This project uses [pre-commit](https://pre-commit.com/) to enforce code quality automatically. The hooks run at two stages:
54+
This project uses [pre-commit](https://pre-commit.com/) to enforce code quality automatically. `just install` registers all three hook types so each fires at the right time:
11255

113-
- **commit**: `ruff` format and lint checks, plus general file hygiene hooks (trailing whitespace, end-of-file, merge conflicts).
114-
- **push**: `pyright` type checking, which is slower and so is deferred until push.
115-
116-
A separate `commit-msg` hook validates that every commit message follows the Conventional Commits format.
117-
118-
`just install` registers all three hook types (`commit-msg`, `pre-commit`, `pre-push`) so everything fires at the right time automatically.
56+
| Stage | Hook |
57+
| ------------ | ----------------------------------------------------------------------- |
58+
| `commit-msg` | Validates the message against Conventional Commits. |
59+
| `pre-commit` | `ruff` format and lint, plus file hygiene (whitespace, EOF, conflicts). |
60+
| `pre-push` | `pyright` type checking (slower, so deferred to push). Also runs in CI. |
11961

12062
```{important}
121-
Run `just check` before opening a PR. It executes the commit and push stage hooks against every file in the repo, giving you the same signal CI will see.
63+
Run `just check` before opening a PR. It executes the commit and push stage hooks against every file, giving you the same signal CI will see.
12264
```
12365

124-
```bash
125-
# Lint and auto-fix with ruff
126-
just lint
127-
128-
# Run the ruff formatter
129-
just format
130-
131-
# Run the pyright type checker
132-
just types
133-
134-
# Run ALL hooks across ALL files (commit + push stages), equivalent to what CI checks
135-
just check
136-
```
66+
Individual recipes are available when you want to run one step:
13767

138-
If pre-commit reports files that _would be reformatted_, run `just format`, stage the changes, and commit before pushing. Formatting-only changes should be committed separately with `style: apply ruff formatting`.
139-
140-
### Type checking (pyright)
141-
142-
Type checking with `pyright` runs automatically on `git push` via the pre-push hook (registered by `just install`). It also runs in CI as the `typecheck` job in `.github/workflows/ci.yml`.
143-
144-
To run it manually at any time:
145-
146-
```bash
147-
just types
148-
```
68+
| Command | Action |
69+
| ------------- | ----------------------------------------------- |
70+
| `just lint` | Lint and auto-fix with ruff. |
71+
| `just format` | Run the ruff formatter. |
72+
| `just types` | Run the pyright type checker. |
73+
| `just check` | Run all hooks across all files (commit + push). |
14974

150-
Fix any reported errors before opening a PR.
75+
If pre-commit reports files that _would be reformatted_, run `just format` and commit the result separately with `style: apply ruff formatting`.
15176

15277
## Documentation
15378

@@ -170,20 +95,16 @@ For the end-to-end release procedure (version bump, tags, PyPI publishing) see:
17095

17196
## Submitting Contributions
17297

173-
When submitting your contributions, please ensure the following:
98+
Before requesting review, make sure your pull request:
17499

175-
- Include a clear and concise description of the changes made in your pull request.
176-
- Reference any relevant issues or feature requests in the pull request description.
177-
- Make sure your code follows the project's coding style and conventions.
178-
- Include appropriate tests that cover your changes, ensuring they pass successfully.
179-
- Update the documentation if necessary to reflect the changes made.
180-
- Ensure that your pull request has a single, logical focus.
100+
- Has a clear description and references any related issues.
101+
- Keeps a single, logical focus.
102+
- Includes passing tests and updated docs for the changes made.
181103

182104
## Issue Tracker
183105

184-
If you encounter any bugs, have feature requests, or need assistance, please visit our [Issue Tracker](https://github.com/OpenTabular/DeepTab/issues). Make sure to search for existing issues before creating a new one.
106+
Report bugs, request features, or ask for help on the [Issue Tracker](https://github.com/OpenTabular/DeepTab/issues). Search existing issues before opening a new one.
185107

186108
## License
187109

188-
By contributing to this project, you agree that your contributions will be licensed under the LICENSE of the project.
189-
Please note that the above guidelines are subject to change, and the project maintainers hold the right to reject or request modifications to any contributions. Thank you for your understanding and support in making this project better!
110+
By contributing, you agree that your contributions are licensed under the project LICENSE.

docs/developer_guide/documentation.md

Lines changed: 2 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -87,24 +87,16 @@ Sphinx raises a warning when `autodoc` documents the same symbol more than once.
8787
:noindex:
8888
```
8989

90-
## Code blocks in pages
90+
## Code blocks and diagrams
9191

92-
Use fenced code blocks with a language tag for syntax highlighting:
92+
Use fenced code blocks with a language tag for syntax highlighting (` ```python `, ` ```bash `). The `sphinxcontrib-mermaid` extension is enabled, so diagrams use a `{mermaid}` fence:
9393

9494
````markdown
9595
```python
9696
model = MLPClassifier()
9797
model.fit(X_train, y_train)
9898
```
99-
````
100-
101-
For shell commands use ` ```bash ` or ` ```sh `.
102-
103-
## Mermaid diagrams
10499

105-
The `sphinxcontrib-mermaid` extension is enabled. Use a `mermaid` code fence:
106-
107-
````markdown
108100
```{mermaid}
109101
flowchart LR
110102
A[Start] --> B[End]

0 commit comments

Comments
 (0)