docs: Expand documentation

.github/workflows/ci.yml

@@ -31,7 +31,7 @@ jobs:
           cache: true
 
       - name: Building
-        run: pixi run build-single
+        run: pixi run build-specific
 
   test:
     runs-on: ubuntu-latest

README.md

@@ -1,3 +1,34 @@
 # snakemake-plugin-catalog
 
-An automatically updated catalog of Snakemake plugins and their documentation
+An automatically updated catalog of
+[Snakemake plugins](https://snakemake.readthedocs.io/en/stable/project_info/codebase.html#plugins)
+and their documentation. See
+<https://snakemake.github.io/snakemake-plugin-catalog/> (generated from
+[this template](/source/_templates/index.rst.j2)) for a general overview of the
+catalog.
+
+## Contributing
+
+> WARNING: The `build` task is designed to run on a GitHub runner and will
+> install essentially arbitrary code on the machine it is run on. Don't run it
+> unless you understand the risk involved! The same goes for `build-specific` if
+> you don't understand the plugin you run it on.
+
+### Software environment
+
+Like the rest of the Snakemake ecosystem, the plugin catalog uses
+[Pixi](https://pixi.prefix.dev/latest/) to manage software environments and
+common development and deployment tasks. See their docs for detailed information
+on setup and task running.
+
+### Development
+
+After making changes to the code, ensure the code is consistently formatted by
+doing `pixi run style`. If you make changes to the Jinja templates used to build
+the catalog, ensure the underlying rst files follow the general Snakemake
+[documentation guidelines](https://snakemake.readthedocs.io/en/stable/project_info/contributing.html#documentation-guidelines).
+
+### Testing
+
+Currently there are no unit-tests. Checking whether the code works as expected
+is done by building individual plugin docs via the `build-specific` Pixi task.

pixi.toml

@@ -7,10 +7,19 @@ version = "0.1.0"
 
 [tasks]
 build = "sphinx-build source build"
-build-single = { cmd = "sphinx-build source build", env = { TEST_PACKAGES = "snakemake-executor-plugin-cluster-generic" } }
 apply-qc = [{ task = "style", environment = "style" }]
 qc = [{ task = "lint", environment = "style" }]
 
+[tasks.build-specific]
+description = "Build docs for individual plugins. Separate `package`s with ',' to specify multiple plugins."
+cmd = """
+export TEST_PACKAGES="{{ packages }}" && \
+sphinx-build source build
+"""
+args = [
+  { "arg" = "packages", "default" = "snakemake-executor-plugin-cluster-generic,snakemake-executor-plugin-slurm" }
+]
+
 [dependencies]
 sphinx = ">=8.2.3,<9"
 python = ">=3.11.0,<4"

source/_templates/index.rst.j2

@@ -11,6 +11,8 @@ The following illustration depicts the general architecture of Snakemake and its
 .. image:: _static/snakemake-architecture.svg
    :alt: Snakemake plugin architecture
 
+See the corresponding `section <https://snakemake.readthedocs.io/en/stable/project_info/codebase.html#plugins>`__ in the Snakemake architecture docs for more details.
+
 Contributing
 ------------
 

source/collect_plugins.py

@@ -20,7 +20,11 @@
 
 
 TEST_PACKAGES = (
-    os.environ.get("TEST_PACKAGES").split(",")
+    [
+        package.strip()
+        for package in os.environ.get("TEST_PACKAGES").split(",")
+        if package.strip()
+    ]
     if "TEST_PACKAGES" in os.environ
     else None
 )
@@ -50,6 +54,11 @@ def log(self, package: str) -> None:
 
 
 class MetadataCollector:
+    """
+    Collect metadata on a plugin `package` of a specific `plugin_type` by installing it
+    in a temporary working directory specific to each class instance.
+    """
+
     def __init__(self, package: str, plugin_type: str, version: str):
         self.envname = uuid.uuid4().hex
         self.package = package
@@ -93,6 +102,10 @@ def __enter__(self):
         )
 
         def pixi_add(args=None):
+            """
+            Add the package for which metadata is to be parsed to the temporary
+            workspace.
+            """
             args = args or []
             self._run(["pixi", "add", f"{self.package}=={self.version}"] + args)
 
@@ -154,6 +167,13 @@ def aux_info(self, metadata_collector) -> Dict[str, Any]:
         return {}
 
     def collect_plugins(self, plugins, packages, templates):
+        """
+        Collect plugins of the type of the corresponding plugin type collector class.
+        Plugins are selected from the set of ALL pypi packages by matching names to the
+        expected prefix of 'snakemake-{plugin_type}-plugin-'. For each matching package
+        metadata is then extracted, the provided `templates` are rendered using this
+        information, and the plugin name is appended to `plugins`.
+        """
         plugin_type = self.plugin_type()
         plugin_dir = Path("plugins") / plugin_type
         if plugin_dir.exists():