Skip to content

Commit 331da72

Browse files
committed
feat(#194): update guidance for Test-NovaBuild usage
- Add documentation for using Test-NovaBuild as the authoritative test entrypoint - Update instructions to prohibit direct Invoke-Pester calls in Nova-managed projects - Revise various documentation files to reflect the new testing policy
1 parent 8aec25e commit 331da72

31 files changed

Lines changed: 97 additions & 38 deletions

.github/agents/powershell-developer.agent.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ Implement PowerShell command and helper changes in the NovaModuleTools style.
4545
- Preserve existing command names, warning semantics, and output shape.
4646
- Keep new or heavily changed source functions inside the warning thresholds from `.github/instructions/code-quality-matrix.instructions.md` unless the scope explicitly justifies otherwise.
4747
- Prefer `./scripts/build/Invoke-ScriptAnalyzerCI.ps1` and `./run.ps1` for normal analyzer loops; use direct `Invoke-ScriptAnalyzer` only for focused local checks that reuse the repository-approved settings.
48+
- Validate Nova-managed project tests through `Test-NovaBuild`; do not call `Invoke-Pester` directly.
4849
- When help files change, keep `docs/NovaModuleTools/en-US/*.md` valid for `Import-MarkdownCommandHelp`: use `New-MarkdownCommandHelp` for new files, `Update-MarkdownCommandHelp` after command-surface changes, and `Test-MarkdownCommandHelp` before handoff. A new public `src/public/*.ps1` file is not done until its matching help file exists.
4950

5051
## Definition of done
@@ -53,6 +54,7 @@ Implement PowerShell command and helper changes in the NovaModuleTools style.
5354
- Build output still comes from Nova-generated `dist/` files, not hand-authored module files in `src/`.
5455
- Public/private file ownership still follows the one externally called function per file rule.
5556
- Every new public entry point has its matching help file.
57+
- Project test validation ran through `Test-NovaBuild`.
5658
- Any ScriptAnalyzer findings reported by `run.ps1` or `Invoke-ScriptAnalyzerCI.ps1` are resolved.
5759
- Every changed or generated text file has been checked and ends with exactly one trailing newline and no extra blank lines at the bottom.
5860
- Docs/changelog review is complete.

.github/agents/reviewer.agent.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ Review changes for correctness, maintainability, test coverage, workflow safety,
2525
- Review changed `src/**/*.ps1` and `tests/**/*.ps1` against `.github/instructions/code-quality-matrix.instructions.md`; flag new or heavily changed code that exceeds the warning thresholds without a clear, explicit reason.
2626
- Flag public files that do not keep exactly one top-level function, and flag private files that group multiple externally called functions instead of limiting extra functions to same-file support helpers. Also flag file/function name mismatches for public commands or externally called private helpers.
2727
- Flag broad catch-all test files when focused source-mirrored tests would make ownership clearer.
28+
- Flag Nova-managed validation that bypasses `Test-NovaBuild` with direct `Invoke-Pester`.
2829
- Flag any PSScriptAnalyzer rule excludes or suppressions; the code should be fixed instead.
2930
- Flag unresolved ScriptAnalyzer findings from `run.ps1` or `Invoke-ScriptAnalyzerCI.ps1`; they should be fixed instead of deferred.
3031

.github/agents/test-engineer.agent.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,20 +37,22 @@ Improve or maintain the repository's Pester coverage, coverage-gate behavior, an
3737

3838
## Constraints
3939

40-
- Prefer targeted tests first, then the full repo quality loop.
40+
- Prefer the smallest `Test-NovaBuild` scope the project already supports, then the full repo quality loop.
4141
- Keep test files maintainable; passing tests are not enough if Code Health degrades.
4242
- Reuse existing fixture and support patterns before adding new ones.
4343
- Do not group unrelated source files into one broad test file when mirrored `tests/public`, `tests/private`, or `tests/classes` ownership is possible.
4444
- Do not introduce PowerShell 7.x-only test syntax or APIs into a project that targets `5.1` unless compatibility coverage is explicitly part of the scope.
4545
- If CodeScene flags a regression, refactor the tests or helpers instead of suppressing the finding.
4646
- Keep new or heavily changed tests inside the warning thresholds from `.github/instructions/code-quality-matrix.instructions.md` unless the scope explicitly justifies otherwise.
4747
- Use `./scripts/build/Invoke-ScriptAnalyzerCI.ps1` as the normal analyzer entrypoint for changed test/helpers, and only fall back to direct `Invoke-ScriptAnalyzer` for focused local investigation with the repository-approved settings.
48+
- Use `Test-NovaBuild` as the test entrypoint for Nova-managed projects; do not validate with direct `Invoke-Pester`.
4849

4950
## Definition of done
5051

5152
- The changed behavior is covered.
5253
- Each new or changed `src/**/*.ps1` file has a matching source-mirrored test, or the cross-cutting owner test is named explicitly.
5354
- The touched tests are readable and low-duplication.
55+
- Validation uses `Test-NovaBuild` for project test execution.
5456
- Validation and CodeScene implications are addressed.
5557
- The pre-commit CodeScene safeguard is clean before the work is treated as commit-ready when local CodeScene tooling is available.
5658

.github/copilot-instructions.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ For new or not-yet-scoped work, start with `.github/agents/architect.agent.md` a
4747
- Do not exclude or suppress PSScriptAnalyzer rules in repository analyzer helpers; fix the code that violates analyzer rules instead.
4848
- Use `.github/instructions/psscriptanalyzer.instructions.md` as the ScriptAnalyzer workflow source of truth. Prefer `./scripts/build/Invoke-ScriptAnalyzerCI.ps1` and `./run.ps1`, and only use direct `Invoke-ScriptAnalyzer` for focused local checks or deliberate analyzer-tooling changes.
4949
- Keep `run.ps1` as the local quality loop: run ScriptAnalyzer first, then `Invoke-NovaBuild`, then `Test-NovaBuild`.
50+
- Use `Test-NovaBuild` as the authoritative test entrypoint in Nova-managed projects. Do not call `Invoke-Pester` directly, because it can bypass Nova's build/import/StrictMode flow and disagree with the result users get later.
5051
- If `run.ps1` or `./scripts/build/Invoke-ScriptAnalyzerCI.ps1` reports ScriptAnalyzer findings, fix them before review, handoff, or commit. Do not treat a failing local quality loop as an acceptable stopping point.
5152
- Keep file/function ownership explicit: `src/public/` files should own exactly one top-level function each, and `src/private/` files should expose at most one externally called function per file. Additional private functions may stay only as support helpers used from the same file, and the file name should match the function that owns the file.
5253
- Use `.github/instructions/code-quality-matrix.instructions.md` as the best-effort quality matrix for `src/**/*.ps1` and `tests/**/*.ps1`; generated projects should follow that guidance through Agentic Copilot files, not by depending on a required `.codescene/code-health-rules.json`.
@@ -88,7 +89,7 @@ For new or not-yet-scoped work, start with `.github/agents/architect.agent.md` a
8889
Use the smallest validation set that proves the change, then run the repository quality loop before finishing code work:
8990

9091
- local quality loop: `pwsh -NoLogo -NoProfile -File ./run.ps1`
91-
- targeted tests: `Invoke-Pester -Path ./tests/<File>.Tests.ps1 -Output Detailed`
92+
- test validation: `Test-NovaBuild`
9293
- analyzer only: `./scripts/build/Invoke-ScriptAnalyzerCI.ps1`
9394
- CI-parity coverage flow: `./scripts/build/ci/Invoke-NovaModuleToolsCI.ps1 -OutputDirectory ./artifacts`
9495

.github/instructions/testing-policy.instructions.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,8 @@ Use this file when changing production code, tests, coverage behavior, or CI tes
1111
## Test expectations
1212

1313
- Behavior changes require Pester coverage.
14-
- Prefer targeted tests close to the changed workflow area before running the full quality loop.
14+
- Use `Test-NovaBuild` as the authoritative test entrypoint in Nova-managed projects. Do not validate with direct `Invoke-Pester`, because it can bypass Nova's build/import/StrictMode flow and disagree with what users see later.
15+
- Prefer the smallest `Test-NovaBuild` scope the project already supports before running the full quality loop.
1516
- Keep test names explicit about the behavior being proven.
1617
- Reuse `*.TestSupport.ps1` helpers where possible.
1718
- For every new or changed `src/**/*.ps1` file, add or update the matching source-mirrored `.Tests.ps1` file.
@@ -37,12 +38,13 @@ Use this file when changing production code, tests, coverage behavior, or CI tes
3738
## Common pitfalls
3839

3940
- Many tests expect the built module under `dist/NovaModuleTools`; build first when needed.
41+
- Direct `Invoke-Pester` runs can hide Nova-specific build/import/StrictMode behavior and should not be used as the project test entrypoint.
4042
- Some support helpers must be dot-sourced and re-exported inside `BeforeAll`.
4143
- Duplicated test setup can lower Code Health even when tests pass.
4244
- Docs-only changes usually do not need executable validation, but workflow/test docs must still be kept accurate.
4345

4446
## Verification
4547

46-
- Targeted Pester file(s)
48+
- `Test-NovaBuild`
4749
- `./scripts/build/Invoke-ScriptAnalyzerCI.ps1` when PowerShell code changed
4850
- `pwsh -NoLogo -NoProfile -File ./run.ps1` before completion

.github/prompts/implement-issue.prompt.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -24,12 +24,13 @@ Implement the issue in the NovaModuleTools repository using the repository-local
2424
12. Keep file/function ownership explicit: one externally called function per file, with private-file extras limited to same-file support helpers, and keep the file name aligned to the entry function.
2525
13. Before handoff, review every changed or generated text file and normalize it to exactly one trailing newline with no extra blank lines at the bottom.
2626
14. Add or update the matching source-mirrored Pester file for every changed `src/**/*.ps1` file; if the behavior is genuinely cross-cutting, document which integration/guardrail test owns it and why a mirrored unit test is not practical.
27-
15. Add or update valid PlatyPS-compatible help under `docs/NovaModuleTools/en-US/` when public commands or public classes change. Use `New-MarkdownCommandHelp` for new help, `Update-MarkdownCommandHelp` after command-surface changes, and `Test-MarkdownCommandHelp` before handoff; do not replace command help with plain Markdown prose.
28-
16. For every new public entry point, create its matching help file immediately in the same change.
29-
17. Review `README.md`, `CONTRIBUTING.md`, `CHANGELOG.md`, `RELEASE_NOTE.md`, help docs, and `docs/*.html` as applicable.
30-
18. If a commit message is requested, derive it from `$GIT_BRANCH_NAME` and the implemented change using the repository's Conventional Commit rules.
31-
19. Run the relevant validation, then summarize what changed, why, and how it was verified.
32-
20. If that summary is returned as Markdown or copy-ready UI output, format it according to the `markdown-authoring` skill (`.github/skills/markdown-authoring/SKILL.md`).
27+
15. Validate Nova-managed project tests through `Test-NovaBuild`; do not call `Invoke-Pester` directly because it can bypass Nova's build/import/StrictMode flow.
28+
16. Add or update valid PlatyPS-compatible help under `docs/NovaModuleTools/en-US/` when public commands or public classes change. Use `New-MarkdownCommandHelp` for new help, `Update-MarkdownCommandHelp` after command-surface changes, and `Test-MarkdownCommandHelp` before handoff; do not replace command help with plain Markdown prose.
29+
17. For every new public entry point, create its matching help file immediately in the same change.
30+
18. Review `README.md`, `CONTRIBUTING.md`, `CHANGELOG.md`, `RELEASE_NOTE.md`, help docs, and `docs/*.html` as applicable.
31+
19. If a commit message is requested, derive it from `$GIT_BRANCH_NAME` and the implemented change using the repository's Conventional Commit rules.
32+
20. Run the relevant validation, then summarize what changed, why, and how it was verified.
33+
21. If that summary is returned as Markdown or copy-ready UI output, format it according to the `markdown-authoring` skill (`.github/skills/markdown-authoring/SKILL.md`).
3334

3435
## Repository-specific reminders
3536

.github/prompts/improve-test-coverage.prompt.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,12 @@ Improve changed-code coverage in NovaModuleTools without lowering maintainabilit
1212
6. Follow `.github/instructions/psscriptanalyzer.instructions.md` when test code or test helpers change. Prefer `./scripts/build/Invoke-ScriptAnalyzerCI.ps1` for repo-standard analyzer runs, and use direct `Invoke-ScriptAnalyzer` only for focused local checks that reuse the repo-approved settings.
1313
7. Add the smallest test that proves the missing behavior.
1414
8. If setup is duplicated, refactor the tests before adding more assertions.
15-
9. Re-run the focused tests, then the repository quality loop if code changed.
15+
9. Re-run `Test-NovaBuild`, then the repository quality loop if code changed. Do not validate a Nova-managed project with direct `Invoke-Pester`.
1616
10. Recheck CodeScene coverage or Code Health if that was the original failure.
1717

1818
## Repository-specific reminders
1919

2020
- Many tests expect a built `dist/NovaModuleTools` module.
2121
- The CI coverage flow writes `artifacts/pester-coverage.cobertura.xml`.
22+
- Use `Test-NovaBuild` as the project test entrypoint; direct `Invoke-Pester` can miss Nova-specific strict-mode behavior.
2223
- Do not "fix" coverage by weakening assertions or suppressing CodeScene warnings.

.github/prompts/review-change.prompt.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ Review a NovaModuleTools change set with emphasis on correctness, maintainabilit
99
3. Check changed `src/**/*.ps1` and `tests/**/*.ps1` against `.github/instructions/code-quality-matrix.instructions.md`.
1010
4. Check changed PowerShell validation flow against `.github/instructions/psscriptanalyzer.instructions.md`; flag direct analyzer usage that bypasses the repository wrapper or repo-approved settings without a clear reason.
1111
5. Check changed `docs/NovaModuleTools/en-US/*.md` against `.github/instructions/platyps-help.instructions.md` when command help was added or updated; flag files that do not follow the `New-MarkdownCommandHelp` / `Update-MarkdownCommandHelp` / `Test-MarkdownCommandHelp` workflow, miss a new public entry point's matching help file, or break the required PlatyPS section structure.
12-
6. Check whether tests, docs, and changelog updates match the change.
12+
6. Check whether tests, docs, and changelog updates match the change, and flag Nova-managed validation that bypasses `Test-NovaBuild` with direct `Invoke-Pester`.
1313
7. Call out the smallest set of meaningful issues first.
1414
8. Note any missing validation or follow-up work.
1515
9. If the review is returned as Markdown or copy-ready UI text, format it according to the `markdown-authoring` skill (`.github/skills/markdown-authoring/SKILL.md`).

.github/skills/pester-testing/SKILL.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ Use this skill when adding tests, closing coverage gaps, fixing regressions, or
1313

1414
- `tests/*.Tests.ps1`
1515
- `tests/*TestSupport.ps1`
16-
- `pwsh -NoLogo -NoProfile -Command "Invoke-Pester -Path ./tests/<File>.Tests.ps1 -Output Detailed"`
16+
- `Test-NovaBuild`
1717
- `pwsh -NoLogo -NoProfile -File ./run.ps1`
1818
- `./scripts/build/ci/Invoke-NovaModuleToolsCI.ps1 -OutputDirectory ./artifacts`
1919

@@ -22,6 +22,7 @@ Use this skill when adding tests, closing coverage gaps, fixing regressions, or
2222
- Match existing `Describe` / `It` naming style.
2323
- Prefer support helpers for repeated setup.
2424
- Build/import the dist module when the test file expects it.
25+
- Use `Test-NovaBuild` as the authoritative test entrypoint in Nova-managed projects. Do not validate with direct `Invoke-Pester`, because it can miss the Nova build/import/StrictMode flow.
2526
- Add coverage for both happy paths and explicit warnings/errors when behavior changed.
2627
- For every new or changed `src/**/*.ps1` file, add or update one focused test file that mirrors the source path under `tests/`.
2728
- Keep test files and helpers compatible with `project.json` `Manifest.PowerShellHostVersion`; if a project targets `5.1`, do not introduce PowerShell 7.x-only syntax, cmdlets, parameters, or APIs in the tests.
@@ -34,13 +35,14 @@ Use this skill when adding tests, closing coverage gaps, fixing regressions, or
3435
- Forgetting that many tests assume `dist/NovaModuleTools` already exists
3536
- Duplicating setup instead of extending `*.TestSupport.ps1`
3637
- Exporting helper functions at the wrong time in test lifecycle
38+
- Validating a Nova-managed project with direct `Invoke-Pester` instead of `Test-NovaBuild`
3739
- Passing tests while still degrading Code Health through duplication
3840
- Grouping unrelated source files into one large test file when a source-mirrored layout would make ownership clearer
3941
- Adding source files without a matching mirrored test or an explicit cross-cutting-test justification
4042
- Ignoring the test-code matrix and letting new or heavily changed tests grow beyond the warning thresholds without justification
4143

4244
## Verification
4345

44-
- Run the touched test file(s) directly first
46+
- Run `Test-NovaBuild`
4547
- Run `./run.ps1` before finishing code changes
4648
- If coverage is the goal, inspect `artifacts/pester-coverage.cobertura.xml` via the CI helper flow

.github/skills/powershell-module-development/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,6 @@ Use this skill when changing public commands, private helpers, CLI routing suppo
6060

6161
## Verification
6262

63-
- Targeted Pester file(s) for the changed behavior
63+
- `Test-NovaBuild` for the changed behavior
6464
- `tests/ArchitectureGuardrails.Tests.ps1` implications checked
6565
- `pwsh -NoLogo -NoProfile -File ./run.ps1`

0 commit comments

Comments
 (0)