Skip to content

Commit 6f2db93

Browse files
authored
Remove snakemake from comparison (#994)
* Remove snakemake from comparison * more fixes * update links
1 parent 36659dc commit 6f2db93

2 files changed

Lines changed: 16 additions & 25 deletions

File tree

README.md

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -26,14 +26,14 @@ executorlib is the lightest path to take *existing* Python functions and scale t
2626
library [Executor interface](https://docs.python.org/3/library/concurrent.futures.html#executor-objects) you already
2727
know, rather than asking you to adopt a new data, actor, or workflow model.
2828

29-
| | executorlib | `concurrent.futures` | [Dask](https://www.dask.org) | [Parsl](https://parsl-project.org) | [Ray](https://www.ray.io) | [Snakemake](https://snakemake.github.io) |
30-
|---|---|---|---|---|---|---|
31-
| Drop-in `Executor` API ||| ⚠️ | | ||
32-
| Per-call resource assignment ||| ⚠️ ||||
33-
| Native HPC scheduler (SLURM/flux) ||| ⚠️ || ⚠️ ||
34-
| MPI-parallel functions ||| ⚠️ | | ⚠️ | ⚠️ |
35-
| Caching of results ||| ⚠️ ||||
36-
| Setup / learning overhead | Low | Very low | Medium | Medium | Medium | High |
29+
| | executorlib | [Concurrent futures](https://docs.python.org/3/library/concurrent.futures.html) | [Dask](https://www.dask.org) | [Parsl](https://parsl-project.org) | [Ray](https://www.ray.io) |
30+
|---|---|---|---|---|---|
31+
| Drop-in `Executor` API ||| ⚠️ | ⚠️ ||
32+
| Per-call resource assignment ||| ⚠️ |||
33+
| Native HPC scheduler (SLURM/flux) ||| ⚠️ || ⚠️ |
34+
| MPI-parallel functions ||| ⚠️ | ⚠️ | ⚠️ |
35+
| Caching of results ||| ⚠️ |||
36+
| Setup / learning overhead | Low | Very low | Medium | Medium | Medium |
3737

3838
✅ first-class · ⚠️ possible via add-on/config · ❌ not supported. See the full
3939
[comparison: when to use which](https://executorlib.readthedocs.io/en/latest/comparison.html) for honest guidance on
@@ -156,7 +156,6 @@ as hierarchical job scheduler within the allocations.
156156
* [Dask](https://executorlib.readthedocs.io/en/latest/comparison.html#dask)
157157
* [Parsl](https://executorlib.readthedocs.io/en/latest/comparison.html#parsl)
158158
* [Ray](https://executorlib.readthedocs.io/en/latest/comparison.html#ray)
159-
* [Snakemake](https://executorlib.readthedocs.io/en/latest/comparison.html#snakemake)
160159
* [Choose executorlib when](https://executorlib.readthedocs.io/en/latest/comparison.html#choose-executorlib-when)
161160
* [Single Node Executor](https://executorlib.readthedocs.io/en/latest/1-single-node.html)
162161
* [Basic Functionality](https://executorlib.readthedocs.io/en/latest/1-single-node.html#basic-functionality)

docs/comparison.md

Lines changed: 8 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -11,14 +11,14 @@ alternative is the better choice.
1111

1212
## At a glance
1313

14-
| | executorlib | `concurrent.futures` | Dask | Parsl | Ray | Snakemake |
15-
|---|---|---|---|---|---|---|
16-
| Drop-in `Executor` API ||| ⚠️ | | ||
17-
| Per-call resource assignment ||| ⚠️ ||||
18-
| Native HPC scheduler (SLURM/flux) ||| ⚠️ || ⚠️ ||
19-
| MPI-parallel functions ||| ⚠️ | | ⚠️ | ⚠️ |
20-
| Caching of results ||| ⚠️ ||||
21-
| Setup / learning overhead | Low | Very low | Medium | Medium | Medium | High |
14+
| | executorlib | Concurrent futures | Dask | Parsl | Ray |
15+
|---|---|---|---|---|---|
16+
| Drop-in `Executor` API ||| ⚠️ | ⚠️ ||
17+
| Per-call resource assignment ||| ⚠️ |||
18+
| Native HPC scheduler (SLURM/flux) ||| ⚠️ || ⚠️ |
19+
| MPI-parallel functions ||| ⚠️ | ⚠️ | ⚠️ |
20+
| Caching of results ||| ⚠️ |||
21+
| Setup / learning overhead | Low | Very low | Medium | Medium | Medium |
2222

2323
✅ first-class · ⚠️ possible via an add-on or extra configuration · ❌ not supported.
2424

@@ -58,14 +58,6 @@ scheduler integration is via cluster launchers rather than native SLURM/flux.
5858
**Use Ray instead when** you need long-lived stateful actors, an AI/ML ecosystem, or a distributed-object model — and you
5959
are willing to write code in Ray's paradigm.
6060

61-
## [Snakemake](https://snakemake.github.io)
62-
63-
Snakemake is a file-oriented workflow manager: you declare rules with inputs/outputs and it builds a dependency graph,
64-
with strong HPC support and file-based caching. It is a workflow DSL, not a drop-in way to parallelize Python functions.
65-
66-
**Use Snakemake instead when** your pipeline is naturally expressed as files transformed by rules, and you want
67-
reproducible, file-driven workflow management rather than in-process Python futures.
68-
6961
## Choose executorlib when
7062

7163
- You already have Python functions and want to scale them across HPC nodes with minimal rewriting.

0 commit comments

Comments
 (0)