Skip to content

Fix Abilities Explorer schema constraint validation#612

Open
ekamran wants to merge 2 commits into
WordPress:developfrom
ekamran:shifteq/fix-abilities-explorer-schema-validation
Open

Fix Abilities Explorer schema constraint validation#612
ekamran wants to merge 2 commits into
WordPress:developfrom
ekamran:shifteq/fix-abilities-explorer-schema-validation

Conversation

@ekamran
Copy link
Copy Markdown
Contributor

@ekamran ekamran commented May 22, 2026

What?

Fixes the Abilities Explorer test runner so the Validate Input button checks JSON schema constraints, not only basic field types.

This updates validation for:

  • minimum
  • maximum
  • enum
  • stricter number / integer handling

Why?

Some ability input schemas include constraints that were displayed in the UI but not enforced by the validator.

For example, the Content Classification ability shows:

"max_suggestions": {
  "type": "integer",
  "minimum": 1,
  "maximum": 10
}

Before this change, entering "max_suggestions": 11 still showed the input as valid.

How?

  • Updates the PHP validation helper used by the Abilities Explorer AJAX handler.
  • Updates the client-side validator used by the visible Validate Input button.
  • Adds focused PHP coverage for integer, min/max, and enum validation.
  • Adds E2E coverage for the visible Content Classification validation flow.

Testing Instructions

  1. Enable AI.
  2. Enable the Abilities Explorer experiment.
  3. Enable the Content Classification experiment.
  4. Go to Tools > Abilities Explorer.
  5. Open the test runner for ai/content-classification.
  6. Set max_suggestions to 11.
  7. Click Validate Input.
  8. Confirm the UI shows a validation error.
  9. Change max_suggestions to 5.
  10. Click Validate Input again.
  11. Confirm the UI shows the input is valid.

Automated checks run locally:

npm run lint:php -- includes/Experiments/Abilities_Explorer/Ability_Handler.php tests/Integration/Includes/Experiments/Abilities_Explorer/Ability_HandlerTest.php
npm run test:php -- --filter Ability_HandlerTest
npm run lint:php:stan -- includes/Experiments/Abilities_Explorer/Ability_Handler.php tests/Integration/Includes/Experiments/Abilities_Explorer/Ability_HandlerTest.php
npm run lint:js -- src/experiments/abilities-explorer/index.js tests/e2e/specs/experiments/abilities-explorer.spec.js
npm run test:e2e -- tests/e2e/specs/experiments/abilities-explorer.spec.js
npm run build
git diff --check

Screenshots/Screencast

Before:
Before

After:
After

Use of AI Tools

AI assistance: Yes

Tool(s): ChatGPT / Codex

Used for: Repository review, reproduction planning, implementation guidance, test updates, and local verification. I reviewed the changes, tested the behavior locally, and take responsibility for the final submission.

Open WordPress Playground Preview

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 22, 2026

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: ekamran <ekamran@git.wordpress.org>
Co-authored-by: dkotter <dkotter@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.08%. Comparing base (3624c91) to head (0a4294b).
⚠️ Report is 9 commits behind head on develop.

Additional details and impacted files
@@              Coverage Diff              @@
##             develop     #612      +/-   ##
=============================================
+ Coverage      73.18%   74.08%   +0.90%     
- Complexity      1731     1752      +21     
=============================================
  Files             85       85              
  Lines           7473     7514      +41     
=============================================
+ Hits            5469     5567      +98     
+ Misses          2004     1947      -57     
Flag Coverage Δ
unit 74.08% <100.00%> (+0.90%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dkotter dkotter added this to the 1.1.0 milestone May 26, 2026
const isValid = this.validateType(
value,
propSchema.type
if ( ! ( propName in input ) ) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So there is now a client-side Abilities API. I don't know off the top of my head but wondering if that can be used to validate things here rather than us maintaining our own validation?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review. I updated the PHP side to move the property-specific validation into a dedicated validate_property() helper, while keeping validate_type() focused on type checks. That keeps the behavior and existing messages the same, but makes validate_input() easier to follow.

I also updated the new test docblocks from @since n.e.x.t to @since x.x.x.

For the client-side Abilities API question: I checked this locally. Importing @wordpress/abilities here caused the built ai_abilities_explorer script to depend on the wp-abilities script handle, but that handle is not registered on this classic admin page. E2E caught that as a WordPress script dependency notice. Because of that, I left the existing local JS validation in place for now instead of adding script-loading changes to this PR.

Local checks run:

  • npm run test:php -- --filter Ability_HandlerTest
  • npm run lint:php -- includes/Experiments/Abilities_Explorer/Ability_Handler.php tests/Integration/Includes/Experiments/Abilities_Explorer/Ability_HandlerTest.php
  • npm run lint:php:stan -- includes/Experiments/Abilities_Explorer/Ability_Handler.php tests/Integration/Includes/Experiments/Abilities_Explorer/Ability_HandlerTest.php
  • npm run build
  • npm run test:e2e -- tests/e2e/specs/experiments/abilities-explorer.spec.js

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also considered another approach: making the Validate Input button call a validate-only AJAX endpoint and reuse Ability_Handler::validate_input() as the single source of truth.

That would avoid duplicating schema validation rules in JS, and it would also avoid importing @wordpress/abilities, so it would not introduce the wp-abilities script dependency issue on this classic admin page.

The tradeoff is that it expands the scope a bit: it would add a new AJAX action, require the usual nonce/capability handling, and make the validation button async with loading/error states. So I kept this PR smaller by extending the existing client-side validation and aligning the PHP validation path.

Happy to switch to the server-backed validation approach if you think that is cleaner for this screen.

Comment thread tests/Integration/Includes/Experiments/Abilities_Explorer/Ability_HandlerTest.php Outdated
Comment thread includes/Experiments/Abilities_Explorer/Ability_Handler.php Outdated
@dkotter dkotter modified the milestones: 1.0.1, 1.1.0 May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants