|
| 1 | +--- |
| 2 | +title: "Testing Standards" |
| 3 | +sidebar_label: "Testing" |
| 4 | +sidebar_position: 90 |
| 5 | +description: "MAPROOM fixtures, TRAILHEAD scenarios, and IIC canon — how AzureLocal repos test." |
| 6 | +--- |
| 7 | + |
| 8 | +# Testing Standards |
| 9 | + |
| 10 | +> **Canonical reference:** this document. |
| 11 | +> **Applies to:** All AzureLocal repositories that ship executable code or infrastructure. |
| 12 | +> **Last Updated:** 2026-04-12 |
| 13 | +
|
| 14 | +Testing in AzureLocal is centralized. The frameworks, schemas, and canonical test data live in [`AzureLocal/platform/testing/`](https://github.com/AzureLocal/platform/tree/main/testing) and are consumed by every product repo. This standard defines **what each repo must test** and **which framework primitives it must use** — it does not re-describe the frameworks themselves (those have their own docs under [`docs/maproom/`](https://github.com/AzureLocal/platform/tree/main/docs/maproom) and [`docs/trailhead/`](https://github.com/AzureLocal/platform/tree/main/docs/trailhead) once Phase 2 ships). |
| 15 | + |
| 16 | +!!! info "Phase status" |
| 17 | + Phase 1 (this document) establishes the **standard**. Phase 2 ships the **framework** (`AzureLocal.Maproom` PowerShell module, fixture + IIC schemas, TRAILHEAD harness). Until Phase 2 lands, repos continue to reference [`azurelocal-S2DCartographer/tests/maproom/`](https://github.com/AzureLocal/azurelocal-S2DCartographer/tree/main/tests/maproom) as the interim canonical implementation. |
| 18 | + |
| 19 | +--- |
| 20 | + |
| 21 | +## Test classification |
| 22 | + |
| 23 | +Every test in an AzureLocal repo falls into exactly one of these classes. The class determines where the test lives, what harness runs it, and what a failure means. |
| 24 | + |
| 25 | +| Class | Purpose | Location | Harness | |
| 26 | +|-------|---------|----------|---------| |
| 27 | +| **unit** | Exercise a single function/cmdlet in isolation. No external state. | `tests/unit/` | Pester 5 (PowerShell) or pytest (Python) | |
| 28 | +| **contract** | Assert the shape of a cluster / tenant / fleet against a **MAPROOM fixture**. Fixture is authoritative; code conforms to it. | `tests/maproom/fixtures/` | `AzureLocal.Maproom` | |
| 29 | +| **integration** | Exercise module + live dependency (real cluster, real Azure tenant, real AD). Requires a provisioned lab. | `tests/integration/` | Pester 5 with lab config | |
| 30 | +| **scenario** (TRAILHEAD) | Scripted end-to-end walkthrough with pass/fail gates. User-journey shaped: "user does X, expects Y". | `tests/trailhead/` | TRAILHEAD harness (Phase 2) | |
| 31 | +| **drift-audit** | Assert a live environment still matches its MAPROOM fixture after time passes. Runs scheduled, not per-commit. | `tests/drift/` | `AzureLocal.Maproom` + scheduled workflow | |
| 32 | + |
| 33 | +!!! warning "Classification under review" |
| 34 | + The five-class taxonomy above is the **current** rule. An open platform issue ([#3](https://github.com/AzureLocal/platform/issues/3)) is reviewing whether additional classes (compliance assertion, synthetic load, migration-inventory differ, repo conformance) should become first-class. Until that issue closes, treat anything outside the five classes as repo-local tooling — do not lift it into `platform/testing/` without ADR approval. |
| 35 | + |
| 36 | +--- |
| 37 | + |
| 38 | +## MAPROOM — contract testing |
| 39 | + |
| 40 | +MAPROOM fixtures are JSON documents describing the **expected shape** of a target (cluster, host pool, profile-container fleet, migration source inventory). Code-under-test reads the fixture, introspects the real target, and asserts conformance. |
| 41 | + |
| 42 | +**Required for every repo that provisions or manages infrastructure.** |
| 43 | + |
| 44 | +### Required files per repo |
| 45 | + |
| 46 | +- `tests/maproom/fixtures/<target>.json` — one fixture per logical target. Filename = target identifier. |
| 47 | +- `tests/maproom/<target>.Tests.ps1` — Pester test that loads the fixture and runs conformance assertions. |
| 48 | + |
| 49 | +### Fixture schema |
| 50 | + |
| 51 | +Fixtures MUST validate against [`platform/testing/maproom/schema/fixture.schema.json`](https://github.com/AzureLocal/platform/tree/main/testing/maproom/schema) (ships in Phase 2). Until then, use the schema derived from `azurelocal-S2DCartographer/tests/maproom/schema/` and expect to migrate on Phase 2 cutover. |
| 52 | + |
| 53 | +### IIC canon as fixture source |
| 54 | + |
| 55 | +Cluster-shape fixtures that represent the canonical **Infinite Improbability Corp** environment (see [Examples & IIC Policy](examples)) live in [`platform/testing/iic-canon/`](https://github.com/AzureLocal/platform/tree/main/testing/iic-canon). Repos reference these by path; they do not copy them locally. |
| 56 | + |
| 57 | +Canonical IIC fixtures today: |
| 58 | + |
| 59 | +- `iic-org.json` — org-level identity, tenancy, domain structure |
| 60 | +- `iic-cluster-01.json` — primary HCI cluster shape |
| 61 | +- `iic-networks.json` — network topology (VLANs, subnets, NSGs) |
| 62 | + |
| 63 | +Additional IIC canons (AVD host pools, FSLogix profile maps, Nutanix source fleets) are pending classification in issue [#3](https://github.com/AzureLocal/platform/issues/3). |
| 64 | + |
| 65 | +--- |
| 66 | + |
| 67 | +## TRAILHEAD — scenario testing |
| 68 | + |
| 69 | +TRAILHEAD scenarios are executable walkthroughs of a user journey end-to-end: **"a user does X in sequence, each step has an expected outcome, the scenario passes if every step's gate passes."** |
| 70 | + |
| 71 | +**Required for every repo that ships a user-facing workflow** (AVD logon, VM conversion run, migration rehearsal, demo deployment, training lab). |
| 72 | + |
| 73 | +### Required files per repo |
| 74 | + |
| 75 | +- `tests/trailhead/<scenario>.trailhead.ps1` — one file per scenario. Parameterized, idempotent where possible. |
| 76 | +- `tests/trailhead/expected/<scenario>.expected.json` — the pass/fail gates the scenario asserts at each step. |
| 77 | + |
| 78 | +### Naming |
| 79 | + |
| 80 | +Scenario filenames describe the journey, not the implementation: `new-avd-host-pool-deploy.trailhead.ps1`, not `test-avd-deploy.ps1`. The filename is user-documentation-quality. |
| 81 | + |
| 82 | +--- |
| 83 | + |
| 84 | +## What every repo MUST have |
| 85 | + |
| 86 | +A conforming AzureLocal repo has, at minimum: |
| 87 | + |
| 88 | +- [ ] `tests/` directory at repo root |
| 89 | +- [ ] `tests/unit/` with at least one Pester/pytest test per shipped module |
| 90 | +- [ ] `tests/maproom/fixtures/` with at least one fixture if the repo provisions or manages infrastructure |
| 91 | +- [ ] `tests/trailhead/` with at least one scenario if the repo ships a user-facing workflow |
| 92 | +- [ ] CI workflow invoking `platform/.github/workflows/run-tests.yml` (Phase 3) — executes unit + contract tests on every PR |
| 93 | +- [ ] Scheduled workflow invoking `platform/.github/workflows/drift-audit.yml` weekly for repos with live environments |
| 94 | + |
| 95 | +Repos that are purely documentation (`azurelocal-training`, `azurelocal.github.io`, `demo-repository` when it stays demo-only) are exempt from the MAPROOM/TRAILHEAD requirements but still need unit tests for any embedded scripts. |
| 96 | + |
| 97 | +--- |
| 98 | + |
| 99 | +## What does NOT belong in a product repo's `tests/` |
| 100 | + |
| 101 | +- **Copies of MAPROOM framework code.** Reference the platform module; don't vendor it. |
| 102 | +- **Copies of IIC canon fixtures.** Reference [`platform/testing/iic-canon/`](https://github.com/AzureLocal/platform/tree/main/testing/iic-canon) by path. |
| 103 | +- **Shared assertion helpers.** If two repos need the same helper, it belongs in `AzureLocal.Maproom`. Open a platform PR. |
| 104 | +- **Secrets or live-environment credentials.** Integration test config lives in the CI environment, not in-repo. |
| 105 | + |
| 106 | +--- |
| 107 | + |
| 108 | +## Enforcement |
| 109 | + |
| 110 | +- **PR gate** — `Validate Repo Structure` (Phase 0, shipped) asserts `tests/` exists and is non-empty. |
| 111 | +- **PR gate** — `Run Tests` (Phase 3, pending) executes unit + contract tests on every PR. |
| 112 | +- **Weekly gate** — `Drift Audit` (Phase 4, pending) re-runs contract tests against live environments and opens an issue on divergence. |
| 113 | +- **Onboarding gate** — `New-AzureLocalRepo.ps1` (Phase 5, pending) scaffolds the required directories so no new repo ships without them. |
| 114 | + |
| 115 | +--- |
| 116 | + |
| 117 | +## Changes to this standard |
| 118 | + |
| 119 | +Non-trivial edits require an ADR in [`decisions/`](https://github.com/AzureLocal/platform/tree/main/decisions). Current relevant ADRs: |
| 120 | + |
| 121 | +- [`0003-maproom-iic-canon.md`](https://github.com/AzureLocal/platform/blob/main/decisions/0003-maproom-iic-canon.md) — MAPROOM & IIC canon design (Proposed) |
| 122 | +- Classification rubric ADR — tracked in issue [#3](https://github.com/AzureLocal/platform/issues/3), number reserved for `0004-testing-toolset-classification.md` |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?v=4&size=40){kind=avatar}
0 commit comments