Add CTRF export format for reports (admin + WP-CLI)

[davidperezgar]

What?

Closes #1270

Add CTRF export support for Plugin Check reports in both admin exports and WP-CLI output.

Why?

Plugin Check already supports CSV/JSON/Markdown exports, but lacks a standardized test report format for CI tooling and platforms that consume CTRF.

This PR adds first-class CTRF output so results can be integrated with the CTRF ecosystem.

How?

• Added CTRF format support to the exporter:
  • New Results_Exporter::FORMAT_CTRF constant.
  • Added CTRF payload generation with required schema fields:
    • reportFormat: "CTRF"
    • specVersion: "1.0.0"
    • results.tool.name: "plugin-check"
    • results.summary + results.tests[]
  • Refactored CTRF generation into helper methods to satisfy PHPMD complexity rules.
  • Updated CTRF export filename extension to .ctrf.json.
• Added admin export support:
  • Added Export CTRF localized string.
  • Added CTRF export button in admin JS (format: ctrf).
• Added CLI support:
  • Added ctrf and strict-ctrf to accepted --format options.
  • For both formats, output is strict JSON CTRF (no FILE: headers), while preserving existing filtering behavior.
• Updated documentation:
  • docs/CLI.md now documents ctrf and strict-ctrf and includes CLI example.
  • readme.txt changelog updated.
• Added tests:
  • New PHPUnit tests for CTRF export payload and empty-results behavior.
  • Extended Behat scenario to verify --format=ctrf and --format=strict-ctrf.

Testing Instructions

1. Run coding standards checks:

   composer phpmd
   ./vendor/bin/phpcs includes/Utilities/Results_Exporter.php tests/phpunit/tests/Utilities/Results_Exporter_Tests.php

2. Verify CLI docs include the new formats:

   rg -n "ctrf|strict-ctrf" docs/CLI.md includes/CLI/Plugin_Check_Command.php

3. In wp-admin Plugin Check screen, run checks and confirm export buttons include Export CTRF.
4. Test CLI output formats:

   wp plugin check <plugin> --format=ctrf
   wp plugin check <plugin> --format=strict-ctrf

   Confirm output is valid JSON and contains "reportFormat": "CTRF".
5. Confirm exported CTRF filename ends with .ctrf.json.

AI Usage Disclosure

• This PR was created without the help of AI tools
• This PR includes AI-assisted code or content

If AI tools were used, please describe how they were used:
Used Codex/ChatGPT to draft and implement code changes, tests, and documentation updates based on issue requirements.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: davidperezgar <davidperez@git.wordpress.org>
Co-authored-by: Luc45 <lucasbustamante@git.wordpress.org>
Co-authored-by: ernilambar <nilambar@git.wordpress.org>
Co-authored-by: frantorres <frantorres@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[Luc45]

Howdy David, I passed the results through a CTRF validator using ctrf-cli and it failed one small thing:

1. Check out this PR in wp-env, install a plugin with findings (e.g. wp plugin install hello-dolly --activate), then generate a CTRF report:

   wp plugin check hello-dolly --format=ctrf > report.json
   

2. Validate it:

   npx --package=ctrf-cli@0.0.5 -- ctrf-cli validate report.json
   

3. Output:

   ✗ report.json failed validation:
     - /results/tests/0: must NOT have additional properties
     - /results/tests/1: must NOT have additional properties
     ...
   

The culprit is tests[].labels — it isn't in the CTRF spec, and the test-item schema is additionalProperties: false. CTRF reserves extra (object) as the official extension point ("Extension point for arbitrary metadata"), so renaming labels → extra in Results_Exporter::build_ctrf_test() (plus the matching assertions in Results_Exporter_Tests.php) should be enough. I confirmed locally — after that rename both validate and validate-strict pass clean.

[Luc45]

I do wonder, though, that ideally this should be a PR to PHPCS.

For now, I'm converting PHPCS to CTRF in my tool, so Plugin Check being able to generate CTRF isn't a blocker for me.

[Luc45]

I've opened a PR to PHPCS: PHPCSStandards/PHP_CodeSniffer#1420

If they merge this, we could update the PHPCS binary here and use the upstream CTRF reporter instead (since I'll be using CTRF for other PHPCS checks other than plugin check, too)

[davidperezgar]

We could make the conversion in this repository.

[Luc45]

Yes - so, as discussed in DM, I've pushed CTRF reporter for PHPCS as a library: https://github.com/Luc45/phpcs-ctrf-reporter

Your call if you want to use it or integrate something in PCP directly.

[Luc45]

As part of validating that reporter, I used it against WooCommerce and compared the integrity of the CTRF output against a native PHPCS output and they match, both in serial mode or in parallel, passing strict validation by ctrf-cli.

This, together with the unit/integration tests it has, gives me confidence that it's in a good shape.

[ernilambar]

I tested the PR locally and confirm that replacing "labels" to "extra" will generate valid CTRF report.