Skip to content

Commit 6278277

Browse files
tobiasraabetobiasraabe
authored
Add task log capture and export (#821)
1 parent d6a5fd2 commit 6278277

32 files changed

Lines changed: 1147 additions & 149 deletions

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,3 +29,4 @@ tests/test_jupyter/*.txt
2929
.ruff_cache
3030
.venv
3131
docs/jupyter_execute
32+
.DS_Store

‎docs/AGENTS.md‎

Lines changed: 20 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,7 @@
11
# Documentation
22

3-
- Link to existing docs/API refs instead of re-explaining concepts - reduces duplication
4-
and keeps info in sync - Prevents documentation drift and outdated explanations by
5-
maintaining a single source of truth for each concept
6-
- Link to canonical docs rather than duplicating content - prevents drift and
7-
maintenance burden - Consolidating documentation into existing files with
8-
cross-references keeps information consistent and reduces the effort needed to
9-
update multiple locations when changes occur.
3+
## General
4+
105
- Document only public APIs and user-facing behavior - exclude internals, framework
116
abstractions, and implementation plumbing - Users need actionable documentation on
127
what they can use, not confusing details about internal mechanics they can't control
@@ -17,9 +12,6 @@
1712
comprehensive coverage vs. fragmented mentions - Prevents users from missing
1813
features when they approach from different contexts (CLI vs. API) and allows
1914
features to be documented holistically rather than buried in subsections.
20-
- Avoid `# ruff: noqa` or `# type: ignore` in doc examples - ensures examples stay
21-
correct and runnable - Skip directives hide bugs and type errors in documentation
22-
code that users will copy, leading to broken examples in the wild
2315
- Explicitly mark parameters/features as 'optional' in docs, even when types show it -
2416
reduces cognitive load for readers - Users shouldn't need to parse type signatures
2517
to understand optionality; explicit labels make documentation scannable and
@@ -31,3 +23,21 @@
3123
- Strip boilerplate from docs examples - show only the feature being demonstrated -
3224
Reduces cognitive load and helps readers focus on the specific API or pattern being
3325
taught without distraction from scaffolding code.
26+
27+
## Linking
28+
29+
- Link to existing docs/API refs instead of re-explaining concepts - reduces duplication
30+
and keeps info in sync - Prevents documentation drift and outdated explanations by
31+
maintaining a single source of truth for each concept
32+
- Link to canonical docs rather than duplicating content - prevents drift and
33+
maintenance burden - Consolidating documentation into existing files with
34+
cross-references keeps information consistent and reduces the effort needed to
35+
update multiple locations when changes occur.
36+
37+
## Code Examples
38+
39+
- Avoid `# ruff: noqa` or `# type: ignore` in doc examples - ensures examples stay
40+
correct and runnable - Skip directives hide bugs and type errors in documentation
41+
code that users will copy, leading to broken examples in the wild
42+
- Code file examples should have a title that shows the file name.
43+
- Important lines should be highlighted or annotated with a comment.

‎docs/source/_static/md/commands/build-arguments.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
| Argument | Description |
2-
| ------------ | ---------------------------------------------------------- |
3-
| `[PATHS]...` | Paths where pytask looks for task files and configuration. |
1+
| Argument | Description |
2+
| ----------------------- | ---------------------------------------------------------- |
3+
| <code>[PATHS]...</code> | Paths where pytask looks for task files and configuration. |

‎docs/source/_static/md/commands/build-options.md‎

Lines changed: 43 additions & 30 deletions
Large diffs are not rendered by default.

‎docs/source/_static/md/commands/clean-arguments.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
| Argument | Description |
2-
| ------------ | ---------------------------------------------------------- |
3-
| `[PATHS]...` | Paths where pytask looks for task files and configuration. |
1+
| Argument | Description |
2+
| ----------------------- | ---------------------------------------------------------- |
3+
| <code>[PATHS]...</code> | Paths where pytask looks for task files and configuration. |

‎docs/source/_static/md/commands/clean-options.md‎

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
1-
| Option | Default | Description |
2-
| -------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
3-
| `-c, --config FILE` | - | Path to configuration file. |
4-
| `-d, --directories` | `false` | Remove whole directories. |
5-
| `--database-url TEXT` | - | Url to the database. |
6-
| `-e, --exclude PATTERN` | - | A filename pattern to exclude files from the cleaning process. |
7-
| `--editor-url-scheme TEXT` | `file` | Use file, vscode, pycharm or a custom url scheme to add URLs to task ids to quickly jump to the task definition. Use no_link to disable URLs. |
8-
| `--hook-module TEXT` | - | Path to a Python module that contains hook implementations. |
9-
| `--ignore TEXT` | - | A pattern to ignore files or directories. Refer to 'pathlib.Path.match' for more info. |
10-
| `-k EXPRESSION` | - | Select tasks via expressions on task ids. |
11-
| `-m MARKER_EXPRESSION` | - | Select tasks via marker expressions. |
12-
| \`--mode \[dry-run | force | interactive\]\` |
13-
| `-q, --quiet` | `false` | Do not print the names of the removed paths. |
14-
| `--strict-markers` | `false` | Raise errors for unknown markers. |
15-
| `-h, --help` | - | Show this message and exit. |
1+
| Option | Default | Description |
2+
| ------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
3+
| <code>-c, --config FILE</code> | - | Path to configuration file. |
4+
| <code>-d, --directories</code> | <code>false</code> | Remove whole directories. |
5+
| <code>--database-url TEXT</code> | - | Url to the database. |
6+
| <code>-e, --exclude PATTERN</code> | - | A filename pattern to exclude files from the cleaning process. |
7+
| <code>--editor-url-scheme TEXT</code> | <code>file</code> | Use file, vscode, pycharm or a custom url scheme to add URLs to task ids to quickly jump to the task definition. Use no_link to disable URLs. |
8+
| <code>--hook-module TEXT</code> | - | Path to a Python module that contains hook implementations. |
9+
| <code>--ignore TEXT</code> | - | A pattern to ignore files or directories. Refer to 'pathlib.Path.match' for more info. |
10+
| <code>-k EXPRESSION</code> | - | Select tasks via expressions on task ids. |
11+
| <code>-m MARKER_EXPRESSION</code> | - | Select tasks via marker expressions. |
12+
| <code>--mode [dry-run\|force\|interactive]</code> | <code>dry-run</code> | Choose 'dry-run' to print the paths of files/directories which would be removed, 'interactive' for a confirmation prompt for every path, and 'force' to remove all unknown paths at once. |
13+
| <code>-q, --quiet</code> | <code>false</code> | Do not print the names of the removed paths. |
14+
| <code>--strict-markers</code> | <code>false</code> | Raise errors for unknown markers. |
15+
| `-h, --help` | - | Show this message and exit. |

‎docs/source/_static/md/commands/collect-arguments.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
| Argument | Description |
2-
| ------------ | ---------------------------------------------------------- |
3-
| `[PATHS]...` | Paths where pytask looks for task files and configuration. |
1+
| Argument | Description |
2+
| ----------------------- | ---------------------------------------------------------- |
3+
| <code>[PATHS]...</code> | Paths where pytask looks for task files and configuration. |

‎docs/source/_static/md/commands/collect-options.md‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
1-
| Option | Default | Description |
2-
| -------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
3-
| `-c, --config FILE` | - | Path to configuration file. |
4-
| `--database-url TEXT` | - | Url to the database. |
5-
| `--editor-url-scheme TEXT` | `file` | Use file, vscode, pycharm or a custom url scheme to add URLs to task ids to quickly jump to the task definition. Use no_link to disable URLs. |
6-
| `--hook-module TEXT` | - | Path to a Python module that contains hook implementations. |
7-
| `--ignore TEXT` | - | A pattern to ignore files or directories. Refer to 'pathlib.Path.match' for more info. |
8-
| `-k EXPRESSION` | - | Select tasks via expressions on task ids. |
9-
| `-m MARKER_EXPRESSION` | - | Select tasks via marker expressions. |
10-
| `--nodes` | `false` | Show a task's dependencies and products. |
11-
| `--strict-markers` | `false` | Raise errors for unknown markers. |
12-
| `-h, --help` | - | Show this message and exit. |
1+
| Option | Default | Description |
2+
| ------------------------------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- |
3+
| <code>-c, --config FILE</code> | - | Path to configuration file. |
4+
| <code>--database-url TEXT</code> | - | Url to the database. |
5+
| <code>--editor-url-scheme TEXT</code> | <code>file</code> | Use file, vscode, pycharm or a custom url scheme to add URLs to task ids to quickly jump to the task definition. Use no_link to disable URLs. |
6+
| <code>--hook-module TEXT</code> | - | Path to a Python module that contains hook implementations. |
7+
| <code>--ignore TEXT</code> | - | A pattern to ignore files or directories. Refer to 'pathlib.Path.match' for more info. |
8+
| <code>-k EXPRESSION</code> | - | Select tasks via expressions on task ids. |
9+
| <code>-m MARKER_EXPRESSION</code> | - | Select tasks via marker expressions. |
10+
| <code>--nodes</code> | <code>false</code> | Show a task's dependencies and products. |
11+
| <code>--strict-markers</code> | <code>false</code> | Raise errors for unknown markers. |
12+
| `-h, --help` | - | Show this message and exit. |

‎docs/source/_static/md/commands/command-list.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
| Command | Description |
2-
| ----------------------------------------- | ------------------------------------------------------------- |
3-
| [`build`](../../../commands/build.md) | Collect tasks, execute them and report the results. |
4-
| [`clean`](../../../commands/clean.md) | Clean the provided paths by removing files unknown to pytask. |
5-
| [`collect`](../../../commands/collect.md) | Collect tasks and report information about them. |
6-
| [`dag`](../../../commands/dag.md) | Create a visualization of the directed acyclic graph. |
7-
| [`markers`](../../../commands/markers.md) | Show all registered markers. |
8-
| [`profile`](../../../commands/profile.md) | Show information about resource consumption. |
1+
| Command | Description |
2+
| ----------------------- | ------------------------------------------------------------- |
3+
| [`build`](build.md) | Collect tasks, execute them and report the results. |
4+
| [`clean`](clean.md) | Clean the provided paths by removing files unknown to pytask. |
5+
| [`collect`](collect.md) | Collect tasks and report information about them. |
6+
| [`dag`](dag.md) | Create a visualization of the directed acyclic graph. |
7+
| [`markers`](markers.md) | Show all registered markers. |
8+
| [`profile`](profile.md) | Show information about resource consumption. |

‎docs/source/_static/md/commands/dag-arguments.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
| Argument | Description |
2-
| ------------ | ---------------------------------------------------------- |
3-
| `[PATHS]...` | Paths where pytask looks for task files and configuration. |
1+
| Argument | Description |
2+
| ----------------------- | ---------------------------------------------------------- |
3+
| <code>[PATHS]...</code> | Paths where pytask looks for task files and configuration. |

0 commit comments

Comments
 (0)