[tests] Structure agent-driven PHPUnit JSON results (#329)

* [tests] Structure agent-driven PHPUnit JSON results

* Update wiki submodule pointer for PR #329

* [tests] Wrap structured PHPUnit output in command JSON

* [tests] Suppress extra JSON progress records

* [console] Decode nested structured command output

* Update wiki submodule pointer for PR #329

* [console] Suppress structured progress preambles

* [console] Normalize Rector changed file lists

* [bootstrap] Normalize CLI autoload bootstrap

* [console] Ignore warning preambles before structured JSON

* [console] Centralize implicit JSON output detection

* [container] Centralize trait-only service resolution

* Update wiki submodule pointer for PR #329

* [console] Align structured output detection

* Update wiki submodule pointer for PR #329

* [console] Normalize structured command output

* [tests] Stabilize structured reporter expectations

* Update wiki submodule pointer for PR #329

* [console] Centralize non-terminal command logging

* [container] Remove redundant DevTools accessors

* Update wiki submodule pointer for PR #329

* [console] Remove intermediateInfo helper

* [console] Remove unused notice helper

* Refine structured command logging

* Update wiki submodule pointer for PR #329

* Fix coverage metadata for command provider test

* Preserve structured PHPUnit raw output

* Fix coverage metadata for command tests

* Simplify structured command logging

* Remove redundant service provider alias

* Update wiki submodule pointer for PR #329

* Use DevTools test double in command provider test

* Use PHPUnit lifecycle attributes for container reset

* Refresh README and testing docs

* Refresh changelog entry for structured output work

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Author: coisa

.github/wiki

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- Keep structured DevTools command output parseable and agent-friendly by exposing the PHPUnit agent-reporter payload under nested `output`, suppressing intermediary progress logs in JSON modes, and normalizing orchestrated subprocess payloads before the final JSON envelope is emitted (#248)
 - Register ``ergebnis/phpunit-agent-reporter`` in the packaged ``phpunit.xml`` so AI agents receive compact PHPUnit JSON summaries without changing consumer overrides manually (#327)
 - Let `wiki`, `docs`, `tests`, `metrics`, and `reports` skip gracefully for guide-only repositories while keeping wiki/report workflows and published preview links aligned with the artifacts that were actually generated (#325)
 

CHANGELOG.md

@@ -35,6 +35,8 @@ across Fast Forward libraries.
 - Synchronizes packaged skills and project-agent prompts into consumer
   `.agents/skills` and `.agents/agents` directories using safe link-based
   updates
+- Supports guide-only and automation-only repositories by skipping PHPUnit or
+  wiki generation gracefully when no runnable PHP surface exists
 - Works both as a Composer plugin and as a local binary
 - Preserves local overrides through consumer-first configuration resolution
 
@@ -71,6 +73,8 @@ You can also run individual commands for specific development tasks:
 ```bash
 # Run PHPUnit tests
 composer dev-tools tests
+composer dev-tools tests --json
+composer dev-tools tests --pretty-json
 
 # Analyze missing, unused, misplaced, and outdated Composer dependencies
 composer dependencies
@@ -120,6 +124,7 @@ composer phpdoc
 
 # Generate HTML API documentation using phpDocumentor
 composer docs
+composer docs --source=docs/user-guide
 
 # Generate Markdown documentation for the wiki
 composer wiki
@@ -180,6 +185,14 @@ The `metrics` command ships with `phpmetrics/phpmetrics` as a direct
 dependency of `fast-forward/dev-tools`, so consumer repositories can generate
 metrics reports without extra setup.
 
+Guide-only repositories and workflow-only repositories can still use the
+packaged command surface. When no runnable PHPUnit surface exists, `tests`
+returns a controlled warning instead of failing. When a repository ships
+guides without PSR-4 source paths, `docs` builds the guide site without trying
+to synthesize API pages. When `.github/wiki` is absent and no wiki has been
+initialized yet, `wiki` now skips generation with a warning instead of failing
+the whole automation run.
+
 The changelog commands manage Keep a Changelog 1.1.0 files without requiring
 extra tooling in the consumer repository. `changelog:entry` bootstraps a
 missing changelog file on first use, `changelog:check` enforces meaningful
@@ -209,16 +222,35 @@ automatically when the runtime is detected as agent-driven. For
 prints raw values so release workflows can keep capturing semantic versions
 and piping rendered release notes directly into GitHub releases.
 
+`--pretty-json` intentionally remains valid JSON. DevTools does not inject ANSI
+escape sequences into that mode today because preserving a parseable payload
+takes precedence over terminal-only color. Where orchestrated tools can expose
+structured subprocess results safely, DevTools prefers adding stable fields to
+the JSON context rather than coloring otherwise strict JSON output.
+
+For the `tests` command, structured runs now capture the bundled PHPUnit
+agent-reporter payload inside `context.output` while preserving the normal
+DevTools JSON envelope. That means the top-level command still emits one final
+document, and consumers can inspect stable nested keys such as
+`context.output.result`, `context.output.summary`, optional
+`context.output.details`, and `context.output.coverage` when minimum-coverage
+validation is active.
+
 Progress output is disabled by default on the commands that support transient
 rendering, and `--progress` re-enables it for human-readable terminal runs.
 When `--json` or `--pretty-json` is active on commands that orchestrate other
 tools, DevTools keeps progress suppressed, forwards JSON flags where the
 underlying tool supports structured output, and otherwise falls back to
-quieter subprocess modes so the captured payload stays machine-readable. In
-GitHub Actions, queued subprocess output is grouped into collapsible sections,
-and logged failures emit native workflow error annotations, including file and
-line metadata when commands provide it. The packaged tests, reports, wiki, and
-changelog workflows also append concise Markdown outcomes to
+quieter subprocess modes so the captured payload stays machine-readable. The
+`tests` command now captures the bundled PHPUnit agent-reporter payload in
+structured runs and stores it in `context.output`, preserving `result`,
+`summary`, optional `details`, and a `raw_output` fallback when coverage or
+other PHPUnit text is emitted before the final reporter JSON. In GitHub
+Actions, queued subprocess output is
+grouped into collapsible sections, and logged failures emit native workflow
+error annotations, including file and line metadata when commands provide it.
+The packaged tests, reports, wiki, and changelog workflows also append concise
+Markdown outcomes to
 `GITHUB_STEP_SUMMARY` so maintainers can scan versions, URLs, preview refs,
 verification status, and release results without expanding full logs. This
 repository also keeps a bounded retry workflow that reruns failed jobs once
@@ -324,8 +356,8 @@ authoring.
 - `Composer Plugin` - `FastForward\DevTools\Composer\Plugin` exposes the
   packaged command set to Composer and runs `dev-tools:sync` after install and
   update.
-- `DevTools Container` - `FastForward\DevTools\Console\DevTools::create()`
-  builds a shared container from `DevToolsServiceProvider`, which wires
+- `DevTools Container` - `FastForward\DevTools\Container\ContainerFactory::get(FastForward\DevTools\Console\DevTools::class)`
+  resolves the shared application from `DevToolsServiceProvider`, which wires
   process execution, filesystem access, changelog services, Git helpers,
   diffing, reporting, and template loading.
 - `Generic Link Synchronization` -

README.md

@@ -20,10 +20,18 @@
 namespace FastForward\DevTools;
 
 use FastForward\DevTools\Console\DevTools;
+use FastForward\DevTools\Container\ContainerFactory;
 
-$projectVendorAutoload = \dirname(__DIR__, 4) . '/vendor/autoload.php';
-$pluginVendorAutoload = \dirname(__DIR__) . '/vendor/autoload.php';
+$autoloadCandidates = [\dirname(__DIR__, 4) . '/vendor/autoload.php', \dirname(__DIR__) . '/vendor/autoload.php'];
 
-require_once file_exists($projectVendorAutoload) ? $projectVendorAutoload : $pluginVendorAutoload;
+foreach ($autoloadCandidates as $autoloadCandidate) {
+    if (is_file($autoloadCandidate)) {
+        require_once $autoloadCandidate;
 
-DevTools::create()->run();
+        exit(ContainerFactory::get(DevTools::class)->run());
+    }
+}
+
+fprintf(\STDERR, "Could not locate Composer autoload.php for fast-forward/dev-tools.\n");
+
+exit(1);

bin/dev-tools.php

@@ -44,8 +44,8 @@
         "container-interop/service-provider": "^0.4.1",
         "dg/bypass-finals": "^1.9",
         "ergebnis/agent-detector": "^1.1",
-        "ergebnis/phpunit-agent-reporter": "^0.3",
         "ergebnis/composer-normalize": "^2.51",
+        "ergebnis/phpunit-agent-reporter": "^0.3",
         "ergebnis/rector-rules": "^1.18",
         "fakerphp/faker": "^1.24",
         "fast-forward/phpdoc-bootstrap-template": "^2.0",

composer.json

@@ -19,6 +19,9 @@ Runtime Chain
    builds a summary notification and sends it when the run finishes.
 5. ``Ergebnis\PHPUnit\AgentReporter\Extension`` replaces PHPUnit's normal
    output with a compact JSON report when an agent runtime is detected.
+6. In structured DevTools runs, ``tests`` forces the same reporter path for
+   the PHPUnit subprocess so the final nested payload remains deterministic
+   even when the surrounding process would not naturally look agent-driven.
 
 Why This Helps Consumer Projects
 --------------------------------
@@ -29,6 +32,8 @@ Why This Helps Consumer Projects
   scrollback;
 - agent-driven runs consume far less terminal context while still keeping
   failure details and PHPUnit exit semantics intact;
+- DevTools can preserve a single top-level JSON document while nesting the
+  compact PHPUnit summary under ``context.output`` for bot-friendly parsing;
 - event counts are available to the notification layer without adding ad-hoc
   test code.
 

docs/advanced/phpunit-extension.rst

@@ -12,8 +12,9 @@ Startup Chain
    ``vendor/autoload.php`` and falls back to the package autoloader.
 3. ``bin/dev-tools.php`` starts ``FastForward\DevTools\Console\DevTools`` and appends
    ``--no-plugins``.
-4. ``FastForward\DevTools\Console\DevTools::create()`` builds the shared
-   container from ``FastForward\DevTools\ServiceProvider\DevToolsServiceProvider``.
+4. ``FastForward\DevTools\Container\ContainerFactory::get(FastForward\DevTools\Console\DevTools::class)``
+   resolves the shared application container from
+   ``FastForward\DevTools\ServiceProvider\DevToolsServiceProvider``.
 5. ``FastForward\DevTools\Console\CommandLoader\DevToolsCommandLoader``
    lazily discovers ``#[AsCommand]`` classes from the command namespace.
 6. ``FastForward\DevTools\Composer\Capability\DevToolsCommandProvider`` later

docs/api/composer-integration.rst

@@ -103,9 +103,14 @@ Exit Codes
 Behavior
 ---------
 
-- ``docs/`` must exist unless you pass another ``--source`` directory.
+- The default ``docs`` source directory is optional. When it is absent, the
+  command still generates API pages if PSR-4 source paths are available.
+- A custom explicit ``--source`` path must exist; otherwise the command fails
+  fast.
 - API pages are built from the PSR-4 paths declared in ``composer.json``.
 - Guide pages are built from the selected source directory.
+- Repositories without PSR-4 source paths can still generate a guides-only
+  site from the selected source directory.
 - Cache stays enabled by default; omit both flags to keep the command default,
   pass ``--cache`` to force it on, and pass ``--no-cache`` to force it off.
 - When ``--cache-dir`` is omitted, phpDocumentor keeps its default cache

docs/commands/docs.rst

@@ -123,6 +123,9 @@ Behavior
 - When ``--json`` or ``--pretty-json`` is active, it forwards JSON mode to the
   ``docs``, ``tests``, and ``metrics`` subprocesses and suppresses transient
   progress output where those tools support it.
+- Nested ``docs`` and ``tests`` stages can now skip gracefully with warnings in
+  guide-only or automation-only repositories, while ``reports`` still returns a
+  successful aggregate result for the stages that were actually generated.
 - Passes ``--junit <coverage>/junit.xml`` to the metrics step.
 - Used by the ``standards`` command as the final phase.
 - This is the reporting stage used by GitHub Pages.

docs/commands/reports.rst

@@ -71,7 +71,8 @@ Options
 ``--pretty-json``
    Emit the same structured payload with indentation for terminal inspection.
    This also suppresses PHPUnit progress output automatically so the JSON
-   payload is not polluted by transient progress rendering.
+   payload is not polluted by transient progress rendering. The output remains
+   valid JSON and intentionally does not include ANSI color escapes.
 
 Examples
 --------
@@ -112,6 +113,13 @@ Run with minimum coverage enforcement:
 
    composer tests --min-coverage=80
 
+Run with structured JSON output:
+
+.. code-block:: bash
+
+   composer tests --json
+   composer tests --pretty-json
+
 Run without cache:
 
 .. code-block:: bash
@@ -149,6 +157,10 @@ Behavior
 ---------
 
 - Local ``phpunit.xml`` is preferred over the packaged default.
+- When the default ``tests`` path is absent and the project exposes no
+  testable PHP source files, the command exits successfully with a warning
+  instead of failing the whole automation flow.
+- A custom explicit tests path that does not exist still fails fast.
 - Coverage filters are automatically applied to all PSR-4 paths from composer.json.
 - Multiple coverage formats are generated: HTML, Testdox HTML, Clover XML, and PHP.
 - Cache stays enabled by default; omit both flags to keep the command default,
@@ -160,5 +172,25 @@ Behavior
 - progress output is disabled by default.
 - ``--json`` and ``--pretty-json`` keep progress output disabled so the
   structured payload stays clean, even when ``--progress`` is provided.
+- in agent-driven runs outside the Composer test suite, the command also
+  switches to the same structured capture mode automatically.
+- when structured capture is active and PHPUnit emits agent-reporter JSON, the
+  command stores that payload inside ``output`` while keeping the standard
+  DevTools JSON envelope. ``--json`` and ``--pretty-json`` therefore expose
+  the same structured result, with formatting as the only difference.
+- the command forces the bundled PHPUnit agent-reporter extension for
+  structured runs so the nested payload stays compact and stable even when the
+  surrounding runtime would not normally look like an agent.
+- in structured mode, the command suppresses intermediary ``Running...`` log
+  records so the output stream contains a single final JSON document.
+- when structured capture is active but PHPUnit does not emit parseable JSON,
+  the command preserves the raw subprocess text inside ``output.raw_output``
+  instead of dropping it.
+- when coverage generation or other PHPUnit text appears before the final
+  reporter payload, the command preserves that prelude in
+  ``output.raw_output`` while keeping the main JSON result parseable.
+- when ``--min-coverage`` is used in structured mode, the command appends a
+  ``coverage`` object under ``output`` and flips ``output.result`` to
+  ``failure`` if the threshold is not met.
 - The command fails if minimum coverage is not met (when ``--min-coverage`` is set).
 - The packaged configuration registers the DevTools PHPUnit extension.