Skip to content

Commit c7001bf

Browse files
committed
feat(#200): establish mirrored test layout convention
- Document the source-mirrored, dot-source-first test loading pattern in the testing-policy instruction and pester-testing skill (mirrored into the agentic-copilot scaffold), add a non-blocking scripts/build/Get-TestMirrorStatus.ps1 helper that reports src files without a mirrored test, add proof-of-concept mirrored tests for - Format-ProjectJsonValue and Get-ProjectJsonValueTypeName, remove the dist-psm1 coverage Path override so project.json owns the coverage Path, and temporarily disable the Pester coverage gate while the existing suite migrates to the new pattern.
1 parent 8b434bb commit c7001bf

13 files changed

Lines changed: 318 additions & 41 deletions

File tree

.github/instructions/testing-policy.instructions.md

Lines changed: 52 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,49 @@ applyTo: "tests/**/*.ps1,scripts/build/**/*.ps1,.github/workflows/Tests.yml,.git
88

99
Use this file when changing production code, tests, coverage behavior, or CI test flows.
1010

11+
## Test loading pattern (mirrored, dot-source-first)
12+
13+
NovaModuleTools is migrating tests to a 1:1 source-to-test layout per issue #208. New and migrated tests follow this loading pattern:
14+
15+
- The test file's `BeforeAll` dot-sources **only** the `src/**/*.ps1` files it needs (the source under test plus its private collaborators that are not being mocked).
16+
- Tests do **not** `Import-Module $project.OutputModuleDir`.
17+
- Tests do **not** use `InModuleScope <ModuleName> { ... }`. Mocked functions live in the same scope as the test because they were dot-sourced into it.
18+
- Shared fixtures and dot-source helpers live in `tests/TestHelpers/`.
19+
- This makes `project.json` `Pester.CodeCoverage.Path = ["src/public/*.ps1", "src/private/**/*.ps1"]` produce real source-file coverage and means `Test-NovaBuild` does not require a `dist/` folder.
20+
21+
Example:
22+
23+
```powershell
24+
BeforeAll {
25+
$projectRoot = Split-Path -Parent (Split-Path -Parent $PSScriptRoot)
26+
. (Join-Path $projectRoot 'src/private/quality/Initialize-NovaPesterCoverageConfiguration.ps1')
27+
. (Join-Path $projectRoot 'src/public/Test-NovaBuild.ps1')
28+
}
29+
```
30+
31+
Legacy tests that still use `Import-Module $project.OutputModuleDir` + `InModuleScope` are being migrated in the issue series #200..#207. While migration is in progress, `project.json` `CodeCoverage.Enabled` is `false` so the percentage gate does not fail. The gate is re-enabled in issue #207 once enough tests use the mirrored pattern.
32+
33+
## Cross-cutting tests are still allowed
34+
35+
Use a cross-cutting test file (not a mirrored one) when the behavior under test truly spans multiple source files:
36+
37+
- Architecture guardrails (layering, adapter boundaries, file ownership rules)
38+
- Public command surface tests that exercise the built module end-to-end
39+
- Workflow tests that intentionally validate multi-helper orchestration
40+
- CLI route/forwarding tests that prove the routing topology, not the helpers it routes to
41+
42+
Cross-cutting files should be **named for the behavior** they validate (e.g., `ArchitectureGuardrails.Tests.ps1`), not for "remaining coverage". A test belongs in a mirrored file when it covers a single source file's behavior.
43+
44+
## Where new tests for a source file go
45+
46+
| Source file | Mirrored test file |
47+
|-----------------------------------|-------------------------------------------|
48+
| `src/public/<Name>.ps1` | `tests/public/<Name>.Tests.ps1` |
49+
| `src/private/<domain>/<Name>.ps1` | `tests/private/<domain>/<Name>.Tests.ps1` |
50+
| `src/classes/<Name>.ps1` | `tests/classes/<Name>.Tests.ps1` |
51+
52+
A non-blocking mirror status helper is available at `scripts/build/Get-TestMirrorStatus.ps1` to show which source files still lack a mirrored test.
53+
1154
## Test expectations
1255

1356
- Behavior changes require Pester coverage.
@@ -27,25 +70,28 @@ Use this file when changing production code, tests, coverage behavior, or CI tes
2770

2871
## Repository test structure
2972

30-
- `tests/NovaCommandModel*.Tests.ps1` - public command, CLI, and workflow behavior
73+
- `tests/public/<Name>.Tests.ps1`, `tests/private/<domain>/<Name>.Tests.ps1`, `tests/classes/<Name>.Tests.ps1` - mirrored unit tests (preferred for new and migrated tests)
3174
- `tests/ArchitectureGuardrails.Tests.ps1` - layering and adapter boundaries
32-
- `tests/*TestSupport.ps1` - shared helpers and reusable fixtures
33-
- Source-mirrored tests should use `tests/public/<Name>.Tests.ps1`, `tests/private/<domain>/<Name>.Tests.ps1`, and `tests/classes/<Name>.Tests.ps1` for matching `src/public/`, `src/private/`, and `src/classes/` files.
34-
- Repeated setup belongs in `tests/TestHelpers/` or `tests/*TestSupport.ps1`, not in duplicated blocks across mirrored tests.
75+
- `tests/NovaCommandModel*.Tests.ps1` - public command, CLI, and workflow behavior (legacy bucket, being migrated)
76+
- `tests/*TestSupport.ps1`, `tests/TestHelpers/` - shared helpers, reusable fixtures, dot-source helpers
77+
- `tests/CoverageGaps*.Tests.ps1`, `tests/Remaining*Coverage*.Tests.ps1` - legacy coverage buckets being retired in issue #207
3578

3679
## Coverage and CodeScene
3780

3881
- CI coverage is generated by `./scripts/build/ci/Invoke-NovaModuleToolsCI.ps1`.
3982
- `Test-NovaBuild` runs once and produces a JaCoCo coverage report at `artifacts/coverage.xml`.
4083
- The JaCoCo artifact is reused by the CodeScene PR coverage gate and by the develop/manual CodeScene analysis flow.
4184
- The CodeScene analysis upload sends coverage twice: once for `line-coverage` and once for `branch-coverage`.
85+
- Coverage paths in `project.json` must point at `src/**/*.ps1`, not at the built `dist` psm1. Nova does not override `CodeCoverage.Path`.
86+
- During the test-layout migration, `CodeCoverage.Enabled` is temporarily `false` so the percentage gate does not block work. It is re-enabled in issue #207.
4287
- If CodeScene flags coverage or duplication, fix the underlying test design instead of suppressing the warning casually.
4388

4489
## Common pitfalls
4590

46-
- Many tests expect the built module under `dist/NovaModuleTools`; build first when needed.
91+
- Importing the built `dist/NovaModuleTools` module from a new mirrored test - mirrored tests dot-source `src/**/*.ps1` files directly instead.
92+
- Using `InModuleScope NovaModuleTools { ... }` in new tests - the function under test is already in scope after dot-sourcing.
4793
- Direct `Invoke-Pester` runs can hide Nova-specific build/import/StrictMode behavior and should not be used as the project test entrypoint.
48-
- Some support helpers must be dot-sourced and re-exported inside `BeforeAll`.
94+
- Some legacy support helpers must be dot-sourced and re-exported inside `BeforeAll`; new TestHelpers should expose dot-source helpers directly.
4995
- Duplicated test setup can lower Code Health even when tests pass.
5096
- Tests that only cover the happy path can miss the edge cases that caused the change in the first place.
5197
- Shared mutable state between tests makes failures order-dependent and unreliable.

.github/skills/pester-testing/SKILL.md

Lines changed: 26 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ Use this skill when adding tests, closing coverage gaps, fixing regressions, or
2121

2222
- Match existing `Describe` / `It` naming style.
2323
- Prefer support helpers for repeated setup.
24-
- Build/import the dist module when the test file expects it.
24+
- For new and migrated tests, dot-source `src/**/*.ps1` files directly in `BeforeAll`. Do not `Import-Module $project.OutputModuleDir` in mirrored unit tests, and do not use `InModuleScope NovaModuleTools { ... }` - the function under test is already in scope after dot-sourcing.
2525
- Use `Test-NovaBuild` as the authoritative test entrypoint in Nova-managed projects. Do not validate with direct `Invoke-Pester`, because it can miss the Nova build/import/StrictMode flow.
2626
- Add coverage for both happy paths and explicit warnings/errors when behavior changed.
2727
- For every new or changed `src/**/*.ps1` file, add or update one focused test file that mirrors the source path under `tests/`.
@@ -30,19 +30,35 @@ Use this skill when adding tests, closing coverage gaps, fixing regressions, or
3030
- Keep shared setup in `tests/TestHelpers/` or `*TestSupport.ps1`; do not hide unrelated source-file coverage in broad catch-all test files.
3131
- If a mirrored test is not practical because the behavior is genuinely cross-cutting, document the reason in the handoff and point to the owning integration or guardrail test.
3232

33+
## Mirrored layout example
34+
35+
```powershell
36+
BeforeAll {
37+
$projectRoot = Split-Path -Parent (Split-Path -Parent $PSScriptRoot)
38+
. (Join-Path $projectRoot 'src/private/quality/Initialize-NovaPesterCoverageConfiguration.ps1')
39+
}
40+
41+
Describe 'Initialize-NovaPesterCoverageConfiguration' {
42+
It 'leaves CodeCoverage.Path to project.json when coverage is enabled' {
43+
# ...
44+
}
45+
}
46+
```
47+
3348
## Common pitfalls
3449

35-
- Forgetting that many tests assume `dist/NovaModuleTools` already exists
36-
- Duplicating setup instead of extending `*.TestSupport.ps1`
37-
- Exporting helper functions at the wrong time in test lifecycle
38-
- Validating a Nova-managed project with direct `Invoke-Pester` instead of `Test-NovaBuild`
39-
- Passing tests while still degrading Code Health through duplication
40-
- Grouping unrelated source files into one large test file when a source-mirrored layout would make ownership clearer
41-
- Adding source files without a matching mirrored test or an explicit cross-cutting-test justification
42-
- Ignoring boundary/unhappy-path coverage or test isolation and relying on happy-path-only verification
50+
- Importing the built `dist/NovaModuleTools` module from a new mirrored test - dot-source the relevant `src/**/*.ps1` files instead.
51+
- Using `InModuleScope NovaModuleTools { ... }` in new mirrored tests - the function is already in scope after dot-sourcing.
52+
- Duplicating setup instead of extending `*.TestSupport.ps1` or `tests/TestHelpers/`.
53+
- Exporting helper functions at the wrong time in test lifecycle.
54+
- Validating a Nova-managed project with direct `Invoke-Pester` instead of `Test-NovaBuild`.
55+
- Passing tests while still degrading Code Health through duplication.
56+
- Grouping unrelated source files into one large test file when a source-mirrored layout would make ownership clearer.
57+
- Adding source files without a matching mirrored test or an explicit cross-cutting-test justification.
58+
- Ignoring boundary/unhappy-path coverage or test isolation and relying on happy-path-only verification.
4359

4460
## Verification
4561

4662
- Run `Test-NovaBuild`
4763
- Run `./run.ps1` before finishing code changes
48-
- If coverage is the goal, inspect `artifacts/pester-coverage.cobertura.xml` via the CI helper flow
64+
- If coverage is the goal, inspect `artifacts/coverage.xml` produced by the CI helper flow. Coverage is JaCoCo and references source files under `src/**/*.ps1` directly.

.github/workflows/Tests.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ jobs:
3030
- name: Install required CI PowerShell modules
3131
run: ./scripts/build/ci/Install-CiPowerShellModules.ps1
3232

33-
- name: Run NovaModuleTools CI test and coverage workflow
33+
- name: Run NovaModuleTools CI test
3434
run: ./scripts/build/ci/Invoke-NovaModuleToolsCI.ps1 -OutputDirectory ./artifacts
3535

3636
- name: Validate release history files with Test-KeepAChangelogFile

CHANGELOG.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
2121
- When Git is enabled during either scaffold flow, Nova now creates or updates a default `.gitignore` in the generated project root and appends only the missing Nova-managed artifact entries instead of overwriting existing ignore rules.
2222
- `Initialize-NovaModule` and `% nova init` now check for newer NovaModuleTools releases before the first interactive scaffold question and reuse the same non-blocking update warning behavior already used by build.
2323
- `Test-NovaBuild` and `% nova test` now enforce `Pester.CodeCoverage.CoveragePercentTarget` from `project.json` when `CodeCoverage.Enabled` is `true`, failing the test run when measured coverage drops below the configured target.
24+
- Added a non-blocking `scripts/build/Get-TestMirrorStatus.ps1` helper that reports `src/**/*.ps1` files without a matching mirrored test file under `tests/`, to support the migration to the source-mirrored, dot-source-first test layout.
2425

2526

2627
### Changed
28+
- Test-loading guidance for NovaModuleTools' own tests and for generated Agentic Copilot scaffolds now follows a source-mirrored, dot-source-first pattern: new mirrored tests dot-source the relevant `src/**/*.ps1` files directly in `BeforeAll` instead of importing the built `dist` module or wrapping assertions in `InModuleScope`, so tests run against source files and JaCoCo coverage references real source paths.
29+
- `project.json` `Pester.CodeCoverage.Enabled` is temporarily `false` while the existing test suite migrates to the new dot-source pattern; the configured `90` percent target stays in place and is re-enabled once migration completes.
2730
- The architect/design flow now surfaces settled vs unresolved design items before finalization, offers explicit choices for full finalization vs design-package-only handoff, clarifies how to use design notes versus the paste-ready GitHub issue draft, and requires finalization output to follow the project Markdown authoring guidance.
2831
- `ProjectTemplate.json` and the packaged example `project.json` now include Nova's default Pester `CodeCoverage` block with `Enabled=false`, shared `src/` coverage paths, JaCoCo output, and a `90` percent target so new projects can opt into coverage without hand-authoring the configuration.
2932
- `Invoke-NovaModuleToolsCI.ps1` now delegates the full test run to a single `Test-NovaBuild` call; the previous second `Invoke-Pester` pass and post-run Cobertura source-path remapping step have been removed.
@@ -39,7 +42,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
3942

4043
### Fixed
4144
- `Test-NovaBuild` and `% nova test` no longer emit `Remove-Item -Recurse` progress bars during the Pester run; `$global:ProgressPreference` is saved and restored around the test execution so CI logs stay clean.
42-
- Code coverage in `Test-NovaBuild` now runs against the built `dist/<ModuleName>/<ModuleName>.psm1` file instead of source files, so measured coverage percentages and line numbers align with the deployed module.
45+
- Code coverage in `Test-NovaBuild` now runs against the source files, instead of the built `dist/<ModuleName>/<ModuleName>.psm1` file so measured coverage percentages and line numbers align with the deployed module.
4346

4447

4548
### Security

RELEASE_NOTE.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,6 @@ This file summarizes public cmdlet, CLI, configuration, and migration changes fo
2020

2121

2222
### Fixed
23-
- Code coverage in `Test-NovaBuild` now runs against the built `dist/` module instead of source files, so reported coverage percentages and line numbers match the deployed module.
24-
- `Test-NovaBuild` and `% nova test` no longer emit `Remove-Item` progress bars during the Pester run.
2523

2624

2725
### Security

project.json

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,11 @@
4141
},
4242
"Pester": {
4343
"CodeCoverage": {
44-
"Enabled": true,
44+
"Enabled": false,
45+
"Path": [
46+
"src/public/*.ps1",
47+
"src/private/**/*.ps1"
48+
],
4549
"CoveragePercentTarget": 90,
4650
"OutputPath": "artifacts/coverage.xml",
4751
"OutputFormat": "JaCoCo"
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
param(
2+
[string]$ProjectRoot,
3+
[switch]$Quiet
4+
)
5+
6+
Set-StrictMode -Version Latest
7+
$ErrorActionPreference = 'Stop'
8+
9+
function Resolve-MirrorStatusProjectRoot {
10+
param([string]$ProjectRoot)
11+
12+
if (-not [string]::IsNullOrWhiteSpace($ProjectRoot)) {
13+
return (Resolve-Path -LiteralPath $ProjectRoot -ErrorAction Stop).Path
14+
}
15+
16+
return (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot '..' '..')).Path
17+
}
18+
19+
function Get-SourceMirrorEntry {
20+
param(
21+
[Parameter(Mandatory)][string]$ProjectRoot,
22+
[Parameter(Mandatory)][string]$SourceRelative
23+
)
24+
25+
$sourceFull = Join-Path $ProjectRoot $SourceRelative
26+
if (-not (Test-Path -LiteralPath $sourceFull -PathType Leaf)) {
27+
return $null
28+
}
29+
30+
$sourceFileName = [System.IO.Path]::GetFileNameWithoutExtension($sourceFull)
31+
$testRelativeParent = ($SourceRelative -replace '^src/', 'tests/')
32+
$testRelativeParent = Split-Path -Parent $testRelativeParent
33+
$testRelative = Join-Path $testRelativeParent ("$sourceFileName.Tests.ps1")
34+
$testRelative = $testRelative -replace '\\', '/'
35+
$testFull = Join-Path $ProjectRoot $testRelative
36+
37+
return [pscustomobject]@{
38+
SourcePath = $SourceRelative
39+
TestPath = $testRelative
40+
Mirrored = Test-Path -LiteralPath $testFull -PathType Leaf
41+
}
42+
}
43+
44+
function Get-SourceMirrorStatus {
45+
param([Parameter(Mandatory)][string]$ProjectRoot)
46+
47+
$sourcePatterns = @(
48+
'src/public/*.ps1'
49+
'src/private/**/*.ps1'
50+
'src/classes/*.ps1'
51+
)
52+
53+
$entries = New-Object System.Collections.Generic.List[object]
54+
foreach ($pattern in $sourcePatterns) {
55+
$patternPath = Join-Path $ProjectRoot $pattern
56+
foreach ($file in Get-ChildItem -Path $patternPath -File -ErrorAction SilentlyContinue) {
57+
$relative = ($file.FullName.Substring($ProjectRoot.Length + 1)) -replace '\\', '/'
58+
$entry = Get-SourceMirrorEntry -ProjectRoot $ProjectRoot -SourceRelative $relative
59+
if ($null -ne $entry) {
60+
$entries.Add($entry)
61+
}
62+
}
63+
}
64+
65+
return $entries
66+
}
67+
68+
function Write-MirrorStatusReport {
69+
param(
70+
[Parameter(Mandatory)][object[]]$Entries,
71+
[switch]$Quiet
72+
)
73+
74+
$missing = @($Entries | Where-Object {-not $_.Mirrored})
75+
$covered = @($Entries | Where-Object {$_.Mirrored})
76+
77+
if (-not $Quiet) {
78+
Write-Host ('Source files scanned : {0}' -f $Entries.Count)
79+
Write-Host ('Mirrored test files : {0}' -f $covered.Count)
80+
Write-Host ('Missing mirrored test: {0}' -f $missing.Count)
81+
if ($missing.Count -gt 0) {
82+
Write-Host ''
83+
Write-Host 'Sources without a mirrored test file:'
84+
foreach ($entry in $missing | Sort-Object SourcePath) {
85+
Write-Host (" {0} -> {1}" -f $entry.SourcePath, $entry.TestPath)
86+
}
87+
}
88+
}
89+
}
90+
91+
$resolvedRoot = Resolve-MirrorStatusProjectRoot -ProjectRoot $ProjectRoot
92+
$entries = Get-SourceMirrorStatus -ProjectRoot $resolvedRoot
93+
Write-MirrorStatusReport -Entries $entries -Quiet:$Quiet
94+
return $entries

src/private/quality/GetNovaTestWorkflowContext.ps1

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -109,8 +109,6 @@ function Initialize-NovaPesterCoverageConfiguration {
109109
if ($null -ne $coveragePercentTarget) {
110110
$PesterConfig.CodeCoverage.CoveragePercentTarget = $coveragePercentTarget
111111
}
112-
113-
$PesterConfig.CodeCoverage.Path = $ProjectInfo.ModuleFilePSM1
114112
}
115113

116114
function Get-NovaPesterSettingValue {

0 commit comments

Comments
 (0)