docs: remove stale terminology and reduce duplication

KristjanESPERANTO · KristjanESPERANTO · commit 9a4bcd0b4a87 · 2026-04-16T08:47:29.000+02:00
diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md
@@ -30,7 +30,7 @@ Use the canonical helper scripts from `package.json` or call the orchestrator di
 | Inspect the pipeline      | `node --run pipeline -- list` / `describe` / `logs`                                                           | Inspect the registered stages, pipelines, and recent run records.                                     |
 | Re-run processing+publish | `node scripts/orchestrator/index.ts run full-refresh-parallel --only=parallel-processing,aggregate-catalogue` | Re-run worker analysis and publication without re-running markdown parsing manually.                  |
 
-The `parallel-processing` stage is the expensive part of the run: it clones repositories, extracts metadata and screenshots, and performs the deeper checks that produce the stage-5 payload in memory. The follow-up `aggregate-catalogue` stage turns that payload into `modules.json`, `modules.min.json`, and `stats.json`.
+The `parallel-processing` stage is the expensive part of the run: it clones repositories, extracts metadata and screenshots, and performs the deeper checks that produce the analysis payload in memory. The follow-up `aggregate-catalogue` stage turns that payload into `modules.json`, `modules.min.json`, and `stats.json`.
 
 ### Orchestrator CLI for partial runs
 
@@ -52,17 +52,7 @@ Check the [orchestrator CLI reference](pipeline/orchestrator-cli-reference.md) f
 
 ### Stage details
 
-#### Stage 1+2 – `collect-metadata/index.js`
-
-Reads the official wiki list of third-party modules and fetches metadata (stars, topics, default branch, etc.) from the hosting service (GitHub/GitLab). The stage returns an in-memory enriched payload for downstream stages and updates the API cache on disk.
-
-#### Stage 3+4+5 – `parallel-processing.js`
-
-Combines repository cloning, `package.json` enrichment, screenshot extraction, and deep analysis inside the worker pool. The stage emits the analysis payload in memory.
-
-#### Stage 6 – `aggregate-catalogue.js`
-
-Consumes the stage-5 payload and writes the published catalogue outputs (`modules.json`, `modules.min.json`, `stats.json`).
+For canonical stage definitions, responsibilities, and data flow, use [docs/architecture.md](architecture.md) as the single source of truth. This avoids drift between contributor instructions and pipeline design docs.
 
 #### `validate_release_artifacts.ts`
 
@@ -80,7 +70,7 @@ For regression testing, prefer the curated fixtures (`node --run fixtures:genera
 
 ## Release validation
 
-As of September 2025, schema validation is part of the release gate. After you regenerate the website data, run:
+Schema validation is part of the release gate. After you regenerate the website data, run:
 
 ```bash
 node --run release:validate
@@ -122,17 +112,6 @@ npm run schemas:check
 
 The bundled files live in `dist/schemas/`. They ship with the repository so `node --run release:validate` can run without extra setup.
 
-## Prerequisites & installation
-
-1. Install [Node.js](https://nodejs.org).
-2. Clone the repository:
-   ```bash
-   git clone https://github.com/MagicMirrorOrg/MagicMirror-3rd-Party-Modules
-   cd MagicMirror-3rd-Party-Modules
-   npm install
-   ```
-3. Optional: Launch the task menu with `npm start` or run `node --run all` to execute every stage sequentially (note: takes >10 minutes and >2 GB disk when cloning the full catalogue).
-
 ## Running the container locally
 
 To preview the published website bundle without running the full pipeline, use the prebuilt container image:
@@ -219,10 +198,9 @@ Golden artifacts are reference outputs stored in `fixtures/golden/` that serve a
 
 ## Helpful references
 
-- [`docs/architecture.md`](architecture.md) – current vs. target pipeline topology.
+- [`docs/architecture.md`](architecture.md) – current pipeline topology.
 - [`docs/pipeline/orchestrator-cli-reference.md`](pipeline/orchestrator-cli-reference.md) – command reference for partial runs, diagnostics, and logs.
 - [`fixtures/README.md`](../fixtures/README.md) – curated dataset and validation troubleshooting.
-- [`docs/pipeline/orchestrator-cli-reference.md`](pipeline/orchestrator-cli-reference.md) – current CLI and schema validation flow.
 
 ### Troubleshooting
 
diff --git a/docs/README.md b/docs/README.md
@@ -0,0 +1,20 @@
+# Documentation Index
+
+Use this index to find the right document quickly.
+
+## Core docs
+
+- [CONTRIBUTING.md](CONTRIBUTING.md) - contributor setup, pipeline commands, testing, and troubleshooting.
+- [architecture.md](architecture.md) - canonical pipeline topology, stage flow, and distribution touchpoints.
+- [deterministic-outputs.md](deterministic-outputs.md) - deterministic output guarantees and implementation notes.
+
+## Pipeline-specific docs
+
+- [pipeline/orchestrator-cli-reference.md](pipeline/orchestrator-cli-reference.md) - orchestrator CLI behavior, options, and run metadata.
+- [pipeline/check-modules-reference.md](pipeline/check-modules-reference.md) - check-modules rule inventory and fixture coverage.
+
+## Related areas outside docs/
+
+- [../guides/README.md](../guides/README.md) - guidance aimed at module authors (linting, README quality).
+- [../fixtures/README.md](../fixtures/README.md) - curated fixture dataset and regeneration workflow.
+- [../pipeline/workers/README.md](../pipeline/workers/README.md) - worker pool internals and runtime behavior.
diff --git a/docs/architecture.md b/docs/architecture.md
@@ -2,18 +2,18 @@
 
 Visibility into the automation that builds and publishes the third-party module catalogue helps contributors reason about changes and spot failure points early. This document summarizes the current canonical pipeline and the parts of the broader architecture that are still future-facing.
 
-## Current State (March 2026)
+## Current State (April 2026)
 
 The supported production pipeline is orchestrated via `node scripts/orchestrator/index.ts run full-refresh-parallel` (also exposed as `node --run all`). The orchestrator now drives four registered stages across three operational phases: metadata collection, parallel module processing, and publication.
 
 ### Stage Overview
 
-| Order | Stage ID                   | Key Outputs                                                               |
-| ----- | -------------------------- | ------------------------------------------------------------------------- |
-| 1     | `collect-metadata`         | in-memory metadata payload, `gitHubData.json`                             |
-| 2     | `parallel-processing`      | in-memory stage-5 payload, `modules/`, `modules_temp/`, `website/images/` |
-| 3     | `aggregate-catalogue`      | `modules.json`, `modules.min.json`, `stats.json`                          |
-| 4     | `generate-result-markdown` | `result.md`                                                               |
+| Order | Stage ID                   | Key Outputs                                                                                        |
+| ----- | -------------------------- | -------------------------------------------------------------------------------------------------- |
+| 1     | `collect-metadata`         | in-memory metadata payload, `gitHubData.json`                                                      |
+| 2     | `parallel-processing`      | in-memory analysis payload, `modules/`, `modules_temp/`, `website/images/`, `skipped_modules.json` |
+| 3     | `aggregate-catalogue`      | `modules.json`, `modules.min.json`, `stats.json`                                                   |
+| 4     | `generate-result-markdown` | `result.md`                                                                                        |
 
 ### Current Workflow Diagram
 
@@ -31,13 +31,13 @@ flowchart TB
     metadata --> parallel{{Parallel processing}}
     parallel --> clones[("modules/<br>modules_temp/")]
     parallel --> images[("website/images/")]
-    parallel --> stage5["stage-5 payload (in-memory)"]
+    parallel --> analysisPayload["analysis payload (in-memory)"]
   end
 
   subgraph Phase 3: Catalogue Aggregation
-    stage5 --> aggregate{{Aggregate catalogue}}
+    analysisPayload --> aggregate{{Aggregate catalogue}}
     aggregate --> outputs[("modules.json<br>modules.min.json<br>stats.json")]
-    stage5 --> result{{Generate result markdown}}
+    analysisPayload --> result{{Generate result markdown}}
     outputs --> result
     result --> resultMd[("result.md")]
   end
@@ -51,7 +51,7 @@ flowchart TB
 
 - **Orchestrator CLI**: Declarative stage graph with `--only/--skip` support, retries, and structured logging
 - **Worker Pool Stage**: `parallel-processing` encapsulates clone, enrich, image, and analysis work behind a single supported stage
-- **Aggregation Stage**: `aggregate-catalogue` builds published JSON artifacts from the in-memory stage-5 payload
+- **Aggregation Stage**: `aggregate-catalogue` builds published JSON artifacts from the in-memory analysis payload
 - **Schema Validation**: JSON schemas enforce contracts at the published boundaries (`modules.json`, `modules.min.json`, `stats.json`)
 - **Shared Utilities**: HTTP, Git, filesystem, and rate limiting in `scripts/shared/`
 
diff --git a/docs/pipeline/check-modules-reference.md b/docs/pipeline/check-modules-reference.md
@@ -1,16 +1,9 @@
 # Check Modules Reference
 
-_Last updated: March 19, 2026_
+_Last updated: April 16, 2026_
 
 This page consolidates the material that previously lived in early planning documents. It should stay up to date as we evolve the check-modules pipeline and the curated fixture set.
 
-## Status snapshot
-
-- ✅ TypeScript implementation is the default Stage 5 runner.
-- ✅ CLI progress indicator renders live module throughput and emits a per-run Markdown summary under `.pipeline-runs/check-modules/`.
-- ✅ Stage 5 flags modules that import any non built-in dependency without declaring it in their own `package.json` (Node built-ins and the allowlist `express`, `node_helper`, `logger` are ignored).
-- ✅ README install and update sections now require a copy-pasteable fenced command block so users can run the instructions without manual reformatting.
-
 ## Check group configuration
 
 The Stage 5 runner reads `scripts/check-modules/check-groups.config.json` to decide which groups execute. All toggles ship enabled; set individual entries to `false` to skip that category. Personal overrides belong in `check-groups.config.local.json` (same directory) so local tweaks stay out of version control.
@@ -25,12 +18,6 @@ The Stage 5 runner reads `scripts/check-modules/check-groups.config.json` to de
 
 > The runner logs whenever overrides are applied so CI output records which groups were disabled.
 
-## CLI progress & run summaries
-
-- When the stage runs interactively (TTY), it renders a single-line progress gauge on `stderr`. Disable it with `CHECK_MODULES_DISABLE_PROGRESS=1` or `CHECK_MODULES_PROGRESS=off` if you prefer verbose line-by-line logging.
-- Every execution writes a Markdown + JSON summary to `.pipeline-runs/check-modules/<run-id>/`. The Markdown combines toggle state, aggregate stats, and the top modules with issues; the JSON mirrors the key metrics for tooling.
-- Artifact links inside the summary resolve relative to the run directory, making it easy to inspect `website/result.md`, `modules.json`, and `stats.json` produced by the run.
-
 ## Rule inventory
 
 These are the rule IDs currently implemented by the TypeScript checker. Keep this table synchronized with the rule registry schema when adding or modifying checks.
@@ -157,4 +144,4 @@ The curated fixture repositories live in `fixtures/modules/`. Keep this table in
 
 ## Housekeeping
 
-- Update this page whenever you add or modify Stage 5 rules or fixtures.
+- Update this page whenever you add or modify check-modules rules or fixtures.
diff --git a/docs/pipeline/orchestrator-cli-reference.md b/docs/pipeline/orchestrator-cli-reference.md
@@ -1,6 +1,6 @@
 # Pipeline Orchestrator CLI — Reference
 
-_Last updated: 2025-09-30_
+_Last updated: 2026-04-16_
 
 A lightweight Node.js command-line interface reads the declarative stage graph, executes the pipeline end-to-end, and provides structured feedback to contributors. This document now serves as the reference for the shipped implementation.
 
@@ -98,5 +98,4 @@ The original exploration surfaced a few ideas that remain on the backlog:
 
 ## Integration with Existing Scripts
 
-- Stage runner resolves script paths from the graph and executes them raw (no refactor needed initially).
-- Shared helper for environment preparation (e.g. ensuring `modules_temp` rotation) remains a follow-up task and can live under `scripts/lib/pipeline/` when extracted.
+- All supported stages run in-process via the stage runner abstraction (`in-process-stage-runner.ts`). The stage graph declares script paths which the runner resolves and executes within the same Node process.
diff --git a/fixtures/modules/README.md b/fixtures/modules/README.md
@@ -36,6 +36,6 @@ Each fixture directory contains:
 4. Ensure every rule listed in the registry has at least one fixture (baseline or synthetic) that will trigger it. Record a short explanation in `FIXTURE.md` so future updates know the expected findings.
 5. When adding a new fixture, update `docs/pipeline/check-modules-reference.md` with the coverage matrix and, if needed, regenerate the curated dataset under `fixtures/data/`.
 
-## Next steps
+## Maintenance
 
-- Ensure contributors rerun `node scripts/fixtures/updateBaselineShas.ts` whenever we refresh the curated dataset so recorded commits stay accurate.
+- Rerun `node scripts/fixtures/updateBaselineShas.ts` whenever you refresh the curated dataset so recorded commits stay accurate.
diff --git a/module-submissions/README.md b/module-submissions/README.md
@@ -6,6 +6,8 @@ This directory contains the infrastructure for the new PR-based module submissio
 
 Replace the wiki-based module list with a modern, quality-controlled submission process using GitHub Pull Requests.
 
+> **Status:** This system is under development. The validation infrastructure and GitHub Actions workflows exist, but the submission flow has not yet been used in production. The documentation below describes the intended design.
+
 ## 📁 Directory Structure
 
 ```text
@@ -108,13 +110,13 @@ gh api orgs/MagicMirrorOrg/teams/trusted-contributors/memberships/USERNAME -X DE
 
 All submissions must conform to this schema:
 
-```json
+```jsonc
 {
   "url": "https://github.com/username/MMM-ModuleName",
   "name": "MMM-ModuleName",
   "description": "Brief description of what the module does",
   "category": "Category name",
-  "submissionType": "New Module" | "Update Existing Module",
+  "submissionType": "New Module", // or "Update Existing Module"
   "additionalInfo": "Optional additional context",
   "submittedBy": "github-username",
   "submittedAt": "2025-10-19T12:00:00Z",
diff --git a/pipeline/workers/README.md b/pipeline/workers/README.md
@@ -50,7 +50,7 @@ const results = await pool.processModules(modules, moduleConfig);
 
 The worker pool is integrated into the pipeline via:
 
-- **Script**: `scripts/parallel-processing.js`
+- **Script**: `scripts/parallel-processing.ts`
 - **Pipeline**: `full-refresh-parallel` (in `stage-graph.json`)
 
 Run with:
@@ -109,15 +109,15 @@ See [../docs/architecture.md](../../docs/architecture.md) for the canonical pipe
 │  Input: ModuleInput (from in-memory metadata payload)        │
 │                                                              │
 │  ┌──────────────────────────────────────────────────────┐   │
-│  │ Stage 3: Clone Repository                            │   │
+│  │ Phase 1: Clone Repository                            │   │
 │  │  - Check if repo is up-to-date (skip if cached)     │   │
 │  │  - Clone or update repository                        │   │
 │  │  - Move from temp to final location                  │   │
 │  └──────────────────────────────────────────────────────┘   │
 │                          │                                   │
 │                          ▼                                   │
 │  ┌──────────────────────────────────────────────────────┐   │
-│  │ Stage 4: Enrich Metadata                             │   │
+│  │ Phase 2: Enrich Metadata                             │   │
 │  │  - Load package.json                                 │   │
 │  │  - Derive tags from keywords                         │   │
 │  │  - Find and resize screenshot                        │   │
@@ -126,7 +126,7 @@ See [../docs/architecture.md](../../docs/architecture.md) for the canonical pipe
 │                          │                                   │
 │                          ▼                                   │
 │  ┌──────────────────────────────────────────────────────┐   │
-│  │ Stage 5: Analyze                                     │   │
+│  │ Phase 3: Analyze                                     │   │
 │  │  - Fast checks + deep heuristics                     │   │
 │  │  - Optional ESLint / npm-check-updates integrations  │   │
 │  │  - Dependency and README rule evaluation             │   │
