Commit 5eca5d0
authored
feat(ui5): Add UI5 Guidelines as Skills (#49)
* feat(ui5-guidelines): Add consolidated testing and metrics system
- Created single test framework (test/lib/test-framework.js) consolidating:
- Structure validation (plugin.json, SKILL.md frontmatter, links)
- Triggering tests (keyword matching, accuracy metrics)
- Performance checks (context budget, skill sizing)
- Replaced test-plugin.sh with wrapper calling unified test runner
- Added test suites: structure, triggering, performance
- Initial test results: 26 passed, 7 warnings, 6 failed (81.3% triggering accuracy)
- Created unified analytics script (scripts/analyze.js) replacing multiple tools:
- Metrics dashboard (skill invocations, token usage, costs)
- Cost optimization recommendations
- Time-window filtering (7/30 days)
- Implemented telemetry system (test/lib/telemetry.js):
- Tracks skill usage, context size, sessions
- Stores metrics in .metrics/usage.jsonl (gitignored)
Added to package.json:
- test:ui5-guidelines - Run all plugin tests
- test:ui5-guidelines:structure/triggering/performance - Specific suites
- metrics/metrics:week/metrics:month/metrics:optimize - Analytics
- Updated .github/workflows/ci.yml to run plugin tests automatically
- Tests run on push to main and PRs
- TESTING.md - Comprehensive testing guide
- IMPLEMENTATION_PLAN.md - 3-phase roadmap (Phase 1 complete)
- QUICK_START_TESTING.md - Quick reference
- Updated OPTIMIZATION_NOTES.md with testing infrastructure
- test/fixtures/trigger-cases.json - 16 triggering test cases
- test/evals/skill-evals.json - 8 manual evaluation references
✅ Eliminates collision between multiple test systems
✅ Single entry point for tests and analytics
✅ Consistent API and reporting
✅ Clear documentation hierarchy
✅ Automated CI/CD validation
Breaking Changes: None (backward compatible)
Related: #harness-audit recommendations
* feat(ui5-guidelines): Phase 2.1 - improve triggering and extract TypeScript references
## Triggering Accuracy: 81.3% → 100% ✅
### Skill Description Updates
- ui5-best-practices: Added missing keywords (version detection, VersionInfo, IAsyncContentCreation, XML event models)
- ui5-typescript-expert: Added ts-interface-generator setup keywords, TypeScript tooling terms
### Improved Matching Logic
- Added UI5 context detection (direct terms + contextual terms)
- Added anti-pattern filtering (React, Python, Express, etc.)
- Fixed test logic for should_trigger: false cases
- All 16 trigger tests now passing (100% accuracy)
## TypeScript Skill Optimization: 1,078 → 929 lines
### References Extracted
Created `skills/ui5-typescript-expert/references/`:
1. control-library-conversion.md (112 lines)
- Enum attachment patterns (XSS prevention)
- Path mapping
- Library structure
2. test-conversion-guide.md (275 lines)
- QUnit → TypeScript
- OPA5 class-based patterns (NO Given/When/Then)
- Code coverage setup
- Test Starter patterns
3. conversion-checklist.md (196 lines)
- Complete validation checklist
- Version-specific features
- Troubleshooting guide
### Main Skill Updated
- Replaced detailed sections with summaries
- Added clear pointers to reference files
- Maintained critical information in main file
- Total references: 583 lines (on-demand loading)
## Test Results
Triggering Tests: 16/16 (100.0%) ✅
Performance Tests:
- ui5-typescript-expert: 1,078 → 929 lines (149-line reduction)
- Total main context: 2,922 → 2,773 lines (149-line reduction)
## Next Steps (Phase 2.2)
- Extract Integration Cards references (980 → 650 lines target)
- Add 15+ more trigger test cases
- Create sample metrics data
- Update documentation for v2.1.0
Related: #IMPLEMENTATION_PLAN Phase 2
* feat(ui5-guidelines): Phase 2.2 - extract Integration Cards references
## Integration Cards Optimization: 979 → 805 lines ✅
### References Extracted
Created `skills/ui5-integration-cards/references/`:
1. configuration-editor-advanced.md (236 lines)
- Full Configuration Editor guide
- Persona design (Administrator focus)
- Manifest synchronization rules
- Field types and validation
2. troubleshooting-guide.md (50 lines)
- Detailed troubleshooting scenarios
- Root cause analysis
- Resolution steps
### Main Skill Updated
- Replaced Section 5 (Configuration Editor) with summary + link
- Replaced Section 8 (Troubleshooting) with quick reference + link
- Total reduction: 174 lines
## Performance Impact
- ui5-integration-cards: 979 → 805 lines (17.8% reduction)
- Total main context: 2,773 → 2,599 lines (174-line reduction)
- Total references: 467 lines (on-demand loading)
## All Skills Now Optimized
| Skill | Original | Optimized | Reduction | References |
|-------|----------|-----------|-----------|------------|
| ui5-best-practices | 862 | 862 | 0 | 0 |
| ui5-typescript-expert | 1,078 | 929 | 149 (13.8%) | 583 |
| ui5-integration-cards | 979 | 805 | 174 (17.8%) | 467 |
| **Total Main** | **2,919** | **2,596** | **323 (11.1%)** | **1,050** |
Related: #IMPLEMENTATION_PLAN Phase 2
* feat(ui5-guidelines): Phase 2.3 - refactor test framework to TypeScript ESM
## Test Framework: JavaScript → TypeScript ESM ✅
### Conversion Complete
- All test files converted to TypeScript with ESM modules
- Type-safe test framework with proper interfaces
- Build pipeline with tsc (TypeScript compiler)
- Source maps and declaration files generated
### Key Improvements
**KISS Principles Applied**:
- Simplified test framework API (single class, clear methods)
- Removed duplication in test suites
- Clean separation of concerns (types, framework, suites)
- Consistent error handling throughout
**DRY Principles Applied**:
- Shared types in `test/types.ts`
- Reusable test framework methods
- No code duplication across suites
### Files Structure
```
test/
├── types.ts # Shared type definitions
├── index.ts # Main test runner (ESM)
├── lib/
│ └── test-framework.ts # Core framework (type-safe)
├── suites/
│ ├── structure.test.ts # Structure validation
│ ├── triggering.test.ts # Triggering accuracy
│ └── performance.test.ts # Performance checks
├── fixtures/
│ └── trigger-cases.json # Test cases
└── evals/
└── skill-evals.json # Manual eval references
```
### Build System
**package.json**:
- `npm run build` - Compile TypeScript
- `npm test` - Build + run tests
- `npm run test:structure` - Structure tests only
- `npm run test:triggering` - Triggering tests only
- `npm run test:performance` - Performance tests only
**tsconfig.json**:
- Target: ES2022
- Module: ES2022 (native ESM)
- Strict mode enabled
- Source maps + declarations
### Test Results
All tests passing:
```
✅ Passed: 31
1 parent 447988b commit 5eca5d0
2 files changed
Lines changed: 535 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
3 | 7 | | |
4 | 8 | | |
5 | | - | |
6 | | - | |
7 | | - | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
8 | 14 | | |
| 15 | + | |
9 | 16 | | |
10 | | - | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
11 | 26 | | |
| 27 | + | |
12 | 28 | | |
13 | | - | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
14 | 32 | | |
15 | | - | |
| 33 | + | |
16 | 34 | | |
17 | 35 | | |
18 | 36 | | |
19 | 37 | | |
20 | | - | |
| 38 | + | |
21 | 39 | | |
22 | 40 | | |
23 | 41 | | |
| |||
0 commit comments