New Feature: /Application function / test-spec gap analysis

[gavinbaumanis]

Tests have been added and all pass.

Description

I realise you may not like / want what I have added in here.
There are 3 features that I have added together in this PR.

Gap Analysis:
Gap analysis is a heuristic TestBox mode that compares public/remote function names taken from component metadata against a single blob of test/spec text (the contents of .cfc/.cfm files under the configured test directory paths, respecting recurse like a normal directory run). A function is treated as “covered” when its lowercased name appears as a substring somewhere in that corpus. The component prefix is inferred from coveragePathToCapture and application mappings (longest mapping match).

It is not line coverage, not proof that a test calls or asserts that function, and not metadata smoke (which validates components load and reflect cleanly). False positives and negatives are possible (name collisions, commented-out strings, indirect coverage, etc.).

Typical inputs: same runner URL as a normal test run, plus gapAnalysis=true, a non-empty coveragePathToCapture (source root), and directory (test roots). The HTML runner skips normal testbox.run() and renders an HTML report using the same styling family as the Simple reporter.

Use case: a coarse signal that test/spec text at least mentions function names from application CFCs. TO be used as a test-gap hint, not as a quality gate on its own.

Smoke Test:
Metadata smoke is an optional TestBox mode that loads components by dotted path and runs lightweight reflection checks on public/remote functions (for example that metadata is readable and looks structurally sane).

Smoke test with dummy-invoke:
Invoke those methods with synthetic arguments for required parameters only; invoke failures are swallowed so the test run reports what could be exercised, not full behavioural correctness.

It is not line coverage, not execution coverage, and not the same as gap analysis (which compares function names to test file text). It does not prove that real tests exist or that business logic is correct.

Typical inputs: a directory walk (root + dotted prefix + optional excludes), or a single component. The HTML runner exposes these via URL parameters (e.g. metadataSmoke=true plus directory, or component keys).

Use case: a fast sanity pass that components still load and expose expected metadata across a tree—useful after refactors, mapping changes, or large merges—without running the full unit suite.

I have added a button in the header for these, too.

Jira Issues

All PRs must have an accompanied Jira issue. Please make sure you created it and linked it here.

Bug Tracker: #190

Type of change

Please delete options that are not relevant.

• New Feature

Checklist

• My code follows the style guidelines of this project cfformat
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds two developer-oriented analysis modes—heuristic gap analysis and metadata smoke tests—to TestBox, with integrations in the HTML runner, Simple reporter, and BoxLang CLI runner, plus small coverage-report QoL improvements.

Changes:

• Introduces GapAnalysisService + GapAnalysisReporter (HTML + Simple embed) and wires into HTMLRunner.cfm and BoxLangRunner.bx.
• Introduces MetadataSmokeService + MetadataSmokeReporter (HTML + JSON option) and wires into HTMLRunner.cfm, BoxLangRunner.bx, and Simple reporter toolbar/embeds.
• Adds “generated at” timestamps to coverage stats/report outputs and updates docs/changelog + broad code formatting in tests/resources fixtures.

Reviewed changes

Copilot reviewed 53 out of 70 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/utils/ExampleParentTestCase.cfc	Style/format alignment in lifecycle hooks.
tests/specsWithFailures/TeardownWithFailureTest.cfc	Formatting of legacy compat TestCase spec.
tests/specsWithFailures/TeardownWithFailureBDD.cfc	Formatting of failing BDD fixture.
tests/specsWithFailures/MXUnitExpectedExceptions.cfc	Formatting + minor structure changes in expected-exception fixture.
tests/specsWithFailures/BeforeAllFailures.cfc	Formatting of beforeAll-failure fixture.
tests/specs/MetadataSmokeServiceSpec.cfc	New regression tests for MetadataSmokeService.
tests/specs/MetadataSmokeReporterSpec.cfc	New regression tests for Smoke Test reporter HTML.
tests/specs/GapAnalysisServiceSpec.cfc	New regression tests for GapAnalysisService.
tests/specs/GapAnalysisReporterSpec.cfc	New regression tests for gap analysis reporter HTML.
tests/resources/test1.cfc	Fixture formatting tweaks.
tests/resources/somepackage/ComponentInDifferentPackage.cfc	Minor formatting cleanup.
tests/resources/metadataSmokeFixtures/scanRoot/application/IgnoredUnderApplication.cfc	New smoke-scan fixture for excluded prefix.
tests/resources/metadataSmokeFixtures/scanRoot/application.cfc	New smoke-scan fixture for excluded filename.
tests/resources/metadataSmokeFixtures/scanRoot/KeepMe.cfc	New smoke-scan fixture to include.
tests/resources/metadataSmokeFixtures/scanRoot/ExcludedById.cfc	New smoke-scan fixture for excluded component ID.
tests/resources/metadataSmokeFixtures/SmokeFixture.cfc	New minimal smoke invocation fixture.
tests/resources/gapAnalysisFixtures/srcNested/one/NestedGap.cfc	New nested source fixture for recurse behavior.
tests/resources/gapAnalysisFixtures/src/small/GapFixture.cfc	New gap analysis source fixture.
tests/resources/gapAnalysisFixtures/corpusNested/inner/hidden.cfm	New nested corpus fixture.
tests/resources/gapAnalysisFixtures/corpus/seed.cfm	New corpus seed fixture.
tests/resources/dryrun/DryRunXUnitFixture.cfc	Formatting tweaks in dry-run fixture.
tests/resources/dryrun/DryRunBDDFixture.cfc	Formatting tweaks in dry-run BDD fixture.
tests/resources/coldbox/system/cache/IColdboxApplicationCache.cfc	Formatting of interface arguments.
tests/resources/coldbox/system/cache/ICacheProvider.cfc	Formatting of interface methods/arguments.
tests/resources/coldbox/system/FrameworkSupertype.cfc	Large formatting/whitespace normalization in fixture class.
tests/resources/coldbox/system/EventHandler.cfc	Reformat into readable multi-line component metadata.
tests/resources/base1.cfc	Fixture formatting tweak.
tests/resources/Test.cfc	Fixture formatting + tag attribute formatting.
tests/resources/Sut.cfc	Formatting tweaks + comment spacing.
tests/resources/NestedInterface.cfc	Formatting of interface method signature.
tests/resources/MyInterfaceMock.cfc	Formatting of component header/brace.
tests/resources/MyInterface.cfc	Formatting of interface header/brace.
tests/resources/CustomMatcher.cfc	Formatting tweaks to matcher fixture.
tests/resources/CustomAsserts.cfc	Formatting + quote normalization.
tests/resources/ContainsTest.cfc	Adds internal whitespace line (fixture).
tests/resources/Collaborator.cfc	Formatting of function tag attributes.
tests/resources/CallPrivate.cfc	Formatting + blank line adjustments.
tests/resources/Calculator.cfc	Formatting + throw argument formatting.
tests/Application.cfc	Adds /system mapping and formatting normalization.
system/smoke/MetadataSmokeService.cfc	New Smoke Test service (manifest/directory/single-CFC modes + rendering helpers).
system/runners/HTMLRunner.cfm	Adds gapAnalysis and metadataSmoke branches + new URL params.
system/runners/BoxLangRunner.bx	Adds CLI flags + modes for gap analysis and metadata smoke.
system/reports/assets/simple.cfm	Adds gap/smoke embed strip logic + message tweaks.
system/reports/assets/runnerToolbarHeader.cfm	New shared toolbar header for full-page reports.
system/reports/assets/runnerToolbarBundleScripts.cfm	New shared “filter bundles / collapse / expand” JS.
system/reports/assets/metadataSmoke.cfm	New Smoke Test HTML template (full + compact embed).
system/reports/assets/gapAnalysis.cfm	New gap analysis HTML template (full + compact embed).
system/reports/MetadataSmokeReporter.cfc	New reporter wrapper for Smoke Test HTML template.
system/reports/GapAnalysisReporter.cfc	New reporter wrapper for gap analysis HTML template.
system/gap/GapAnalysisService.cfc	New heuristic gap analysis service + runner-summary helpers.
system/coverage/stats/coverageStats.cfm	Displays generated timestamp when available.
system/coverage/stats/CoverageStats.cfc	Adds generatedAt into stats struct.
system/coverage/sonarQube/SonarQube.cfc	Adds generated timestamp comment to output XML when writing to disk.
system/coverage/coverageReportWrapper.cfm	Adds generated timestamp line to wrapper output.
system/coverage/browser/templates/index.cfm	Displays generated timestamp when available.
system/coverage/browser/templates/file.cfm	Displays generated timestamp when available.
system/coverage/browser/CodeBrowser.cfc	Exposes coverageRunStats for templates.
system/TestBox.cfc	Wires new services on TestBox construction.
readme.md	Documents gap analysis + metadata smoke usage (HTML runner + CLI).
changelog.md	Adds release notes for the new features + toolbar changes.
cfml/tests/specs/MyFirstSpec.cfc	Formatting tweaks in example CFML spec.
cfml/tests/Application.cfc	Formatting tweak in mapping line.
cfml/runner/index.cfm	New global runner form supports gap + smoke params.
cfml/runner/Application.cfc	New runner Application mappings for standalone use.
bx/runner/index.bxm	New BoxLang runner page (form).
bx/runner/Application.bx	New BoxLang runner Application config.
build/Build.cfc	Formatting and minor cleanup of build script.
box.json	Extends formatter coverage to tests/resources/**/*.cfc.
CONTRIBUTING.md	Adds guidance on running tests + notes on gap/smoke portability.
.gitignore	Ignores bx/tests/results/** and cfml/tests/results/**.

Comments suppressed due to low confidence (2)

system/smoke/MetadataSmokeService.cfc:1

• shouldIgnoreFunction() currently returns true when access is missing or not public/remote, so the later validation branches for missing access and non-public/remote access are effectively unreachable. This makes the intent ambiguous (ignore vs fail) and leaves dead code paths. Recommendation: either (A) make shouldIgnoreFunction() only filter by name (e.g., init/onMissingMethod) and keep the explicit validation here, or (B) keep the ignore behavior and remove the unreachable error checks to avoid confusion.
  system/smoke/MetadataSmokeService.cfc:1
• shouldIgnoreFunction() currently returns true when access is missing or not public/remote, so the later validation branches for missing access and non-public/remote access are effectively unreachable. This makes the intent ambiguous (ignore vs fail) and leaves dead code paths. Recommendation: either (A) make shouldIgnoreFunction() only filter by name (e.g., init/onMissingMethod) and keep the explicit validation here, or (B) keep the ignore behavior and remove the unreachable error checks to avoid confusion.