Skip to content

Commit 679dc46

Browse files
jpuzz0jpuzz0
authored
feat(PF-4339,PF-4346,PF-4343): add scanner skill evals (#133)
## Summary - Add eval configs and test cases for 3 scanner skills (Batch 1 of PF-4230 eval coverage plan) - **pf-raw-colors-scan**: 3 cases — raw hex/rgb/hsl detection, token declaration exception, already-tokenized negative control - **pf-import-checker**: 3 cases — invalid charts import, invalid chatbot import, valid PF6 imports negative control - **pf-class-migration-scanner**: 3 cases — legacy versioned classes, legacy token patterns, valid PF6 code negative control - All judges use deterministic `check` type (Python regex/string matching) - Negative-control judges check for absence of violation markers (Kent's testing feedback applied) Closes: PF-4339 Closes: PF-4346 Closes: PF-4343
1 parent 56cda6c commit 679dc46

35 files changed

Lines changed: 630 additions & 20 deletions

File tree

‎CONTRIBUTING-SKILLS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ Every skill or agent must live in a plugin. Pick the one that matches your skill
6767
| **code-review** | Review code for quality | Does this help me review code for quality? | |
6868
| **design-audit** | Validate existing code/designs against PF standards | Does this check whether existing code or designs follow PF standards? | `pf-compliance-checker`, `pf-design-token-check`, `pf-figma-icon-finder` |
6969
| **design-guide** | Choose the right PF components and patterns when building | Does this help me choose the right PF components and patterns when building? | `pf-ai-experience-patterns`, `pf-design-mode` |
70-
| **migration** | Upgrade PatternFly versions | Does this help me upgrade PF versions? | `pf-class-migration-scanner` |
70+
| **migration** | Upgrade PatternFly versions | Does this help me upgrade PF versions? | `pf-class-migration-scanner`, `pf-react-breaking-changes` |
7171
| **patternfly-mcp** | Connect AI tools to PatternFly documentation and component data | | |
7272
| **pf-workshop** | Team tools and skill incubation | Is this a team workflow tool, or a new skill that isn't ready for a consumer plugin yet? | `analytics-repo-pruning`, `css-var-analyzer`, `duplicate-epic` |
7373
| **react** | Develop and test React components | Does this help me write or test a React component? | `pf-component-structure`, `pf-design-comments`, `pf-github-pages-deploy` |

‎PLUGINS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,7 @@ PatternFly team tools and skill incubation — issue triage, release management,
100100
| `icon-finder` | Find Red Hat Design System icons (@rhds/icons) by keyword or use case with visual previews. |
101101
| `pf-analyze-modifiers` | Analyze PatternFly modifier class (pf-m-*) usage across SCSS files and generate usage reports. |
102102
| `pf-bug-triage` | Triage PatternFly bug reports — assess completeness, suggest fixes, identify affected components, and recommend assignees. |
103-
| `pf-content-review` | Audit and rewrite content to match PatternFly and Red Hat voice and tone standards. |
103+
| `pf-content-review` | Review content against PatternFly and Red Hat voice and tone standards. |
104104
| `pf-create-issue` | Create well-structured GitHub issues for PatternFly repositories with templates, follow-up tracking, and duplicate detection. |
105105
| `pf-org-version-update` | Update patternfly-org for a new PatternFly release — resolve versions, update package.json and versions.json, and provide build steps. |
106106
| `pf-tokens` | Build CSS design tokens for PatternFly core and copy them to the PatternFly repository. |

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
[![License](https://img.shields.io/github/license/patternfly/ai-helpers)](./LICENSE)
44
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](./CONTRIBUTING.md)
55
[![Plugins](https://img.shields.io/badge/plugins-8-blueviolet)](./PLUGINS.md)
6-
[![Skills](https://img.shields.io/badge/skills-31-blue)](./PLUGINS.md)
6+
[![Skills](https://img.shields.io/badge/skills-32-blue)](./PLUGINS.md)
77

88
AI coding helpers for [PatternFly](https://www.patternfly.org/) development. This repository provides plugins and documentation to help AI tools generate accurate, best-practice PatternFly applications.
99

‎eval/pf-class-migration-scanner/cases/legacy-token-patterns/_legacy-overrides.scss‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
.app-sidebar {
2+
background-color: var(--pf-global--BackgroundColor--dark-300);
3+
color: var(--pf-global--Color--light-100);
4+
border-right: 1px solid var(--pf-global--BorderColor--100);
5+
}
6+
7+
.app-header {
8+
font-size: var(--pf-v6--global--FontSize--xl);
9+
color: var(--pf-v6--global--Color--brand--default);
10+
padding: var(--pf-v6--global--spacer--md);
11+
}
12+
13+
.app-alert {
14+
background-color: var(--pf-global--danger-color--100);
15+
color: var(--pf-global--Color--light-100);
16+
}

‎eval/pf-class-migration-scanner/cases/legacy-token-patterns/annotations.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
expect_violations: true
2+
violation_type: legacy-tokens
3+
legacy_patterns:
4+
- "--pf-global--"
5+
- "--pf-v6--"

‎eval/pf-class-migration-scanner/cases/legacy-token-patterns/input.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
prompt: "Check this SCSS file for deprecated PatternFly token patterns that need to migrate to semantic tokens."
2+
fixture: _legacy-overrides.scss

‎eval/pf-class-migration-scanner/cases/legacy-versioned-classes/LegacyNavbar.tsx‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
import React from 'react';
2+
3+
interface NavbarProps {
4+
items: { label: string; href: string; isActive?: boolean }[];
5+
onLogout: () => void;
6+
}
7+
8+
export const LegacyNavbar: React.FC<NavbarProps> = ({ items, onLogout }) => (
9+
<div className="pf-c-page">
10+
<nav className="pf-v5-c-nav" aria-label="Main navigation">
11+
<ul className="pf-v5-c-nav__list">
12+
{items.map((item) => (
13+
<li
14+
key={item.href}
15+
className={`pf-v5-c-nav__item ${item.isActive ? 'pf-v5-c-nav__item--current' : ''}`}
16+
>
17+
<a href={item.href} className="pf-v5-c-nav__link">
18+
{item.label}
19+
</a>
20+
</li>
21+
))}
22+
</ul>
23+
</nav>
24+
<div className="pf-u-mt-md pf-u-text-align-right">
25+
<button
26+
className="pf-v5-c-button pf-m-secondary"
27+
onClick={onLogout}
28+
type="button"
29+
>
30+
Log out
31+
</button>
32+
</div>
33+
</div>
34+
);

‎eval/pf-class-migration-scanner/cases/legacy-versioned-classes/annotations.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
expect_violations: true
2+
violation_type: legacy-classes
3+
legacy_patterns:
4+
- "pf-v5-c-button"
5+
- "pf-v5-c-nav"
6+
- "pf-c-page"
7+
- "pf-u-mt-md"

‎eval/pf-class-migration-scanner/cases/legacy-versioned-classes/input.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
prompt: "Scan this React component for legacy PatternFly CSS classes that need to be migrated to PF6."
2+
fixture: LegacyNavbar.tsx

‎eval/pf-class-migration-scanner/cases/valid-pf6-code/ModernCard.tsx‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
import React from 'react';
2+
import {
3+
Card,
4+
CardBody,
5+
CardTitle,
6+
CardFooter,
7+
Button,
8+
Label,
9+
} from '@patternfly/react-core';
10+
11+
interface ModernCardProps {
12+
title: string;
13+
description: string;
14+
status: 'active' | 'inactive';
15+
onAction: () => void;
16+
}
17+
18+
export const ModernCard: React.FC<ModernCardProps> = ({
19+
title,
20+
description,
21+
status,
22+
onAction,
23+
}) => (
24+
<Card>
25+
<CardTitle>
26+
{title}
27+
<Label
28+
color={status === 'active' ? 'green' : 'grey'}
29+
className="pf-v6-u-ml-sm"
30+
>
31+
{status}
32+
</Label>
33+
</CardTitle>
34+
<CardBody>{description}</CardBody>
35+
<CardFooter>
36+
<Button variant="primary" onClick={onAction}>
37+
View Details
38+
</Button>
39+
</CardFooter>
40+
</Card>
41+
);

0 commit comments

Comments
 (0)