Reports/Ctrf: add CTRF reporter

[Luc45]

Description

Adds a new ctrf report which emits results in the Common Test Report Format (CTRF), an open JSON standard for test results. The format is consumed by a growing ecosystem of CI/CD integrations (GitHub Actions summaries, dashboards, etc.), letting PHPCS lint runs feed into the same reporting pipelines as test runs.

JUnit already serves a similar purpose, but JUnit XML is older, has looser semantics, and isn't as well-supported by modern CI surfaces. CTRF is a tighter JSON schema that's validatable end-to-end.

Minimal example

Given this fixture (example.php):

<?php

namespace Vendor\App;

class Greeter
{
    public function hello($name) {
        return "hi $name";
    }
}

Running bin/phpcs --no-colors --basepath=. --standard=PSR12 --report=ctrf example.php produces (verbatim):

{
    "reportFormat": "CTRF",
    "specVersion": "1.0.0",
    "reportId": "c236c666-f286-457f-82da-b5074eaec7ec",
    "timestamp": "2026-04-28T13:26:26Z",
    "generatedBy": "PHP_CodeSniffer 4.0.2",
    "results": {
        "tool": {
            "name": "PHP_CodeSniffer",
            "version": "4.0.2"
        },
        "summary": {
            "tests": 1,
            "passed": 0,
            "failed": 1,
            "skipped": 0,
            "pending": 0,
            "other": 0,
            "suites": 1,
            "start": 1777382786761,
            "stop": 1777382786761,
            "duration": 0,
            "extra": {
                "fixable": 1
            }
        },
        "tests": [
            {
                "name": "Squiz.Functions.MultiLineFunctionDeclaration.BraceOnSameLine at example.php (7:34)",
                "status": "failed",
                "duration": 0,
                "start": 1777382786761,
                "stop": 1777382786761,
                "message": "Opening brace should be on a new line",
                "filePath": "example.php",
                "line": 7,
                "suite": [
                    "example.php"
                ],
                "rawStatus": "ERROR",
                "type": "lint",
                "tags": [
                    "ERROR",
                    "fixable"
                ],
                "extra": {
                    "column": 34,
                    "severity": 5,
                    "fixable": true,
                    "source": "Squiz.Functions.MultiLineFunctionDeclaration.BraceOnSameLine"
                }
            }
        ]
    }
}

How violations map to CTRF

Each sniff violation produces one CTRF test entry (mirroring the JUnit reporter's per-violation pattern):

• ERROR → status: "failed"
• WARNING → status: "other" (CTRF's catch-all for non-pass/non-fail outcomes; the ERROR/WARNING distinction is preserved in rawStatus, tags, and extra.severity)
• Clean file → one status: "passed" test entry, so the report contains the full set of files that were processed

PHPCS-native metadata (sniff source, severity, fixability, column) is preserved via CTRF's rawStatus, tags, and extra extension fields. The output passes npx --package=ctrf-cli@0.0.5 -- ctrf-cli validate <file>.

Use --report=ctrf for stdout, or --report-ctrf=/path/to/file.json to write to a file. Composes with other reporters (--report=ctrf,summary).

Manual testing

Beyond the unit and end-to-end tests added in this PR, I validated the reporter against a real-world codebase: WooCommerce's plugins/woocommerce/src/Internal/ directory (443 PHP files, ~10MB of source) using the PSR12 standard. The same scan was run with --report=json and --report=ctrf, and in both serial and parallel (--parallel=8) modes.

Metric	Serial	Parallel (8 workers)
Time	13.2s	2.8s (~4.7× speedup)
Memory peak	581 MB	431 MB
Output size	106 MB	106 MB
Tests emitted	136,749	136,749
Schema valid (ctrf-cli validate + validate-strict)	✓	✓

Equivalence checks performed:

1. Schema — both outputs pass ctrf-cli validate AND ctrf-cli validate-strict (the strict variant enforces additionalProperties: false, so no extra fields anywhere). Verified on both serial and parallel outputs.
2. Self-consistency — summary.tests equals the actual tests array length in both runs (136,749).
3. Serial ↔ parallel equivalence — identical sets of test entries (matched on (status, name, line, rawStatus, source)); identical per-file violation counts. The fork-merge pattern preserves every entry without drops or duplicates.
4. Cross-reporter consistency vs. the JSON reporter on the same scan:
   • JSON: 134,138 errors + 2,611 warnings = 136,749 violations
   • CTRF: 134,138 failed + 2,611 other = 136,749 tests ✓
   • Per-file violation counts match for 443/443 files
   • Fixable count matches (131,224 in both)

Edge cases also verified:

• Filenames with backslashes, quotes, or newlines (handled via json_encode).
• Invalid UTF-8 bytes in messages — explicitly guarded via JSON_INVALID_UTF8_SUBSTITUTE to prevent silent test drops; covered by testInvalidUtf8DoesNotDropTest.
• Composite report mode (--report=ctrf,summary).
• File output mode (--report-ctrf=/path).

Suggested changelog entry

Added: New ctrf report which emits results in the Common Test Report Format, an open JSON standard for test results. Each violation becomes one CTRF test entry (errors as failed, warnings as other). Use --report=ctrf for stdout output, or --report-ctrf=/path/to/file.json to write to a file.

Related issues/external references

CTRF spec: https://ctrf.io/

Types of changes

• New feature (non-breaking change which adds functionality)

PR checklist

• I have checked there is no other PR open for the same change.
• I have read the Contribution Guidelines.
• I grant the project the right to include and distribute the code under the BSD-3-Clause license (and I have the right to grant these rights).
• I have added tests to cover my changes (10 unit tests + 6 end-to-end bash tests, including parallel-mode equivalence).
• I have verified that the code complies with the projects coding standards.
• [Required for new sniffs] I have added XML documentation for the sniff. (N/A — this is a reporter, not a sniff.)
• I have opened a sister-PR in the documentation repository to update the Wiki. (Happy to open one if maintainers want — wanted to check direction first.)

[jrfnl]

@Luc45 I appreciate what you are trying to do, but AI usage is not acceptable for this repo.

[Luc45]

@Luc45 I appreciate what you are trying to do, but AI usage is not acceptable for this repo.

I've manually reviewed and verified every single line written here, and tested it exhaustively against real, large codebases, both serial mode and parallel, and I'm very confident in this implementation. Apart from the tools that I used to write the code, is there any concern with the implementation?

[Luc45]

I can also push this as a separate packagist library if this is a hard rule. But honestly, I think AI is just one more tool in the stack - it depends on how conscious you are while using it.

[Luc45]

I've published it as a standalone package for now https://github.com/Luc45/phpcs-ctrf-reporter

Let me know if you'd like to integrate it into PHPCS Core and I'll gladly do it. Feel free to close this PR or move forward with merging it, your call.

Thanks for all your work, Juliette!

[Luc45]

I'll be closing this PR now, pending your decision.

[jrfnl]

@Luc45 No-AI use is a hard rule, which is not open for discussion. There are too much ethical problems, legal risks and threats to the longevity of OSS, that accepting any AI-generated code would be an existential threat to any OSS codebase and I see it as my duty as custodian of this project to protect the users of this package against that.

Happy to see that the report will be available to users via the package you published, but due to the use of AI, I cannot accept it in PHPCS itself.

Thank you for contributing to the larger ecosystem by publishing it separately.