The runnable trees under examples/ are part of how we document the framework. Their layout and commands are listed in examples/manifest.yaml. Pytest checks that the manifest matches the tree on disk and runs selected pipelines as subprocess smoke tests.
Each top-level entry under pipelines describes one examples/<slug>/ directory that contains pipeline.py.
| Field | Meaning |
|---|---|
slug |
Directory name under examples/. |
ci_tier |
always — run on every default pytest/CI invocation (dev mode in the shipped configs). cluster_optional — prod-oriented demos; tests live under tests/integration/examples_cluster/ and follow the same collection rules as Real cluster integration tests. manual — not part of default CI; run locally when you opt in (see below). |
requirements |
Tags for humans and for cluster gating logic (yt_cluster, s3, docker_image, heavy_python, etc.). |
commands |
Non-empty list of argv lists (after python). Usually [["pipeline.py"]]. See examples/08_multiple_configs/README.md for a tree that lists extra configs so each entry point is exercised. |
If someone adds examples/<new>/pipeline.py but forgets the manifest, pytest fails with a pointer to examples/manifest.yaml.
From the repo root, with the usual dev install:
conda run -n yt-framework -- pytest tests/integration/example_pipelines/test_smoke.py -m examples --tb=shortThe same tests are collected when you run the full suite with pytest -m "not yt_cluster" (GitHub Actions uses that filter). They execute python pipeline.py (and any extra argv from the manifest) under each example directory. Timeouts are per slug so map-heavy demos get more wall clock.
Tests prepend the repository root to PYTHONPATH for each subprocess so import yt_framework works even when the active interpreter is a minimal tool venv (for example the pre-commit hook environment). They also prepend the active interpreter’s bin directory to PATH so dev-mode job shells resolve python3 to an environment that has omegaconf and the rest of the dev extra installed.
The GPU sample pulls in torch and related packages that we do not install on the default CI image. To run it locally:
export YT_FRAMEWORK_EXAMPLE_VIDEO_GPU=1
conda run -n yt-framework -- pytest tests/integration/example_pipelines/test_smoke.py::test_example_pipeline_manual_when_enabled -qWithout the env var, the test skips immediately.
Prod-only examples (06_s3_integration, 07_custom_docker) are covered in Example pipelines on a real cell.