Skip to content

Commit 138a77f

Browse files
codeaholicguycodeaholicguy
committed
Add skill find plan
1 parent 80afaf9 commit 138a77f

5 files changed

Lines changed: 429 additions & 0 deletions

File tree

‎docs/ai/design/feature-skill-find.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
---
2+
phase: design
3+
title: System Design & Architecture
4+
description: Define the technical architecture, components, and data models
5+
---
6+
7+
# System Design & Architecture
8+
9+
## Architecture Overview
10+
**What is the high-level system structure?**
11+
12+
- Include a mermaid diagram that captures the main components and their relationships. Example:
13+
```mermaid
14+
graph TD
15+
User -->|CLI| SkillFind
16+
SkillFind --> RegistryList
17+
SkillFind --> IndexStore
18+
SkillFind --> IndexBuilder
19+
IndexBuilder -->|Fetch metadata| RegistrySources
20+
RegistrySources -->|skills/| IndexBuilder
21+
```
22+
- Key components and their responsibilities
23+
- SkillFind: CLI command `skill find` orchestrates index checks and search.
24+
- RegistryList: reads `skills/registry.json` to discover registries.
25+
- IndexBuilder: builds or updates a local index without full clones.
26+
- IndexStore: local cache file (json) with searchable entries and metadata.
27+
- RegistrySources: Git/HTTP access to registry metadata or skill folder list.
28+
- Technology stack choices and rationale
29+
- Node/TypeScript to align with existing CLI.
30+
- Git commands or HTTP fetch for low-cost metadata access.
31+
- Local cache under user cache dir to avoid repo writes.
32+
33+
## Data Models
34+
**What data do we need to manage?**
35+
36+
- Core entities and their relationships
37+
- Registry: `{ name, url, branch?, skillsPath }`
38+
- SkillEntry: `{ name, registry, path, description, lastIndexed }`
39+
- IndexMeta: `{ version, createdAt, updatedAt, ttlSeconds, registriesHash, registryHeads? }`
40+
- Data schemas/structures
41+
- `index.json` with `{ meta, skills: SkillEntry[] }`
42+
- Data flow between components
43+
- RegistryList -> IndexBuilder (registry config)
44+
- IndexBuilder -> IndexStore (write updated index)
45+
- SkillFind -> IndexStore (read and search)
46+
47+
## API Design
48+
**How do components communicate?**
49+
50+
- External APIs (if applicable)
51+
- GitHub REST tree API to list `skills/` paths without cloning.
52+
- Git `ls-remote` to detect head changes for refresh decisions.
53+
- Internal interfaces
54+
- `getRegistries(): Registry[]`
55+
- `ensureIndex(registries): Index` (checks TTL, registry list hash, optional head hash)
56+
- `searchIndex(index, keyword): SkillEntry[]`
57+
- Request/response formats
58+
- CLI: `npx ai-devkit skill find <keyword>`
59+
- Output: table or list with `skillName` and `description`
60+
- Authentication/authorization approach
61+
- Use standard Git auth (SSH/HTTPS tokens) if private registries appear.
62+
63+
## Component Breakdown
64+
**What are the major building blocks?**
65+
66+
- Frontend components (if applicable)
67+
- CLI output formatting only (no UI).
68+
- Backend services/modules
69+
- `skill-find` command handler.
70+
- `registry-reader` to parse registry list.
71+
- `index-builder` to assemble skill entries.
72+
- `search` utility for keyword matching.
73+
- Database/storage layer
74+
- Local cache file in user cache dir (json).
75+
- Third-party integrations
76+
- GitHub REST API for tree listing and raw file fetch.
77+
- Git CLI for optional head hash checks.
78+
79+
## Design Decisions
80+
**Why did we choose this approach?**
81+
82+
- Key architectural decisions and trade-offs
83+
- Use local index to avoid repeated network calls per search.
84+
- Prefer metadata fetch over full repo clone for speed and bandwidth.
85+
- TTL-based refresh plus manual `--refresh` option for freshness control.
86+
- Optionally compare remote head hashes to detect changes cheaply.
87+
- Store index at `~/.ai-devkit/skills.json`.
88+
- Alternatives considered
89+
- Git sparse checkout of `skills/` only (still needs Git and network).
90+
- Remote centralized index service (fast, but adds infra and availability).
91+
- On-demand repo scan per search (simple but slow and wasteful).
92+
- Patterns and principles applied
93+
- Cache with explicit invalidation (TTL + registry list hash).
94+
- Fail-soft behavior (use stale index if refresh fails).
95+
96+
## Non-Functional Requirements
97+
**How should the system perform?**
98+
99+
- Performance targets
100+
- Search completes in < 2s with a warm index.
101+
- Index refresh completes in < 30s for typical registry sizes.
102+
- Scalability considerations
103+
- Index size grows linearly with skills; support thousands of entries.
104+
- Avoid full clones to keep bandwidth stable as registries grow.
105+
- Security requirements
106+
- Avoid executing registry code; only read metadata paths.
107+
- Do not store credentials in index.
108+
- Reliability/availability needs
109+
- Use last-known index if refresh fails.
110+
- Provide clear error messaging for unreachable registries.
111+

‎docs/ai/implementation/feature-skill-find.md‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
---
2+
phase: implementation
3+
title: Implementation Guide
4+
description: Technical implementation notes, patterns, and code guidelines
5+
---
6+
7+
# Implementation Guide
8+
9+
## Development Setup
10+
**How do we get started?**
11+
12+
- Prerequisites and dependencies
13+
- Node.js, npm, Git (or HTTP access to registries)
14+
- Environment setup steps
15+
- `npm install`
16+
- Configuration needed
17+
- Registry list at `skills/registry.json`
18+
- Local cache at `~/.ai-devkit/skills.json`
19+
20+
## Code Structure
21+
**How is the code organized?**
22+
23+
- Directory structure
24+
- CLI command: `packages/...` (skill command module)
25+
- Index utilities: `packages/.../skills/index`
26+
- Module organization
27+
- `registry-reader`, `index-builder`, `search`, `output-format`
28+
- Naming conventions
29+
- `skill-find` for command, `skill-index` for cache utilities
30+
31+
## Implementation Notes
32+
**Key technical details to remember:**
33+
34+
### Core Features
35+
- Feature 1: Implement `skill find <keyword>` command entry and parsing
36+
- Feature 2: Build/update local index with TTL and `--refresh`
37+
- Feature 2a: Fetch `SKILL.md` from each skill folder for description
38+
- Feature 3: Keyword search across index entries and output results
39+
40+
### Patterns & Best Practices
41+
- Prefer small pure functions for search and filtering logic
42+
- Keep IO (Git/HTTP) isolated from search logic
43+
- Fail-soft on refresh: use stale index if update fails
44+
45+
## Integration Points
46+
**How do pieces connect?**
47+
48+
- API integration details
49+
- Git `ls-remote` or sparse checkout to read `skills/` entries
50+
- Database connections
51+
- Local cache file under user cache dir
52+
- Third-party service setup
53+
- Optional registry-hosted `skills/index.json`
54+
55+
## Error Handling
56+
**How do we handle failures?**
57+
58+
- Error handling strategy
59+
- Return non-zero exit for fatal failures
60+
- Show warnings for partial registry failures
61+
- Logging approach
62+
- Use existing CLI logger for warnings/errors
63+
- Retry/fallback mechanisms
64+
- Use stale index if refresh fails
65+
66+
## Performance Considerations
67+
**How do we keep it fast?**
68+
69+
- Optimization strategies
70+
- Cache index and avoid repeated network calls
71+
- Caching approach
72+
- TTL-based refresh with manual override
73+
- Optional async rebuild after returning cached results
74+
- Query optimization
75+
- Pre-normalize keywords and skill names to lowercase
76+
- Resource management
77+
- Avoid full clones; read only `skills/` metadata
78+
79+
## Security Notes
80+
**What security measures are in place?**
81+
82+
- Authentication/authorization
83+
- Respect Git auth for private registries
84+
- Input validation
85+
- Validate keyword length and sanitize output
86+
- Data encryption
87+
- Not required for local index
88+
- Secrets management
89+
- Never store tokens in index file
90+

‎docs/ai/planning/feature-skill-find.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
---
2+
phase: planning
3+
title: Project Planning & Task Breakdown
4+
description: Break down work into actionable tasks and estimate timeline
5+
---
6+
7+
# Project Planning & Task Breakdown
8+
9+
## Milestones
10+
**What are the major checkpoints?**
11+
12+
- [ ] Milestone 1: Requirements, design, and plan finalized
13+
- [ ] Milestone 2: Indexing and search implementation complete
14+
- [ ] Milestone 3: Tests, docs, and verification complete
15+
16+
## Task Breakdown
17+
**What specific work needs to be done?**
18+
19+
### Phase 1: Foundation
20+
- [ ] Task 1.1: Locate registry config and current skill commands
21+
- [ ] Task 1.2: Define index storage location and schema
22+
- [ ] Task 1.3: Add CLI command entry and help text
23+
24+
### Phase 2: Core Features
25+
- [ ] Task 2.1: Implement registry metadata fetch (Git/HTTP)
26+
- [ ] Task 2.2: Build index from registry skill folders
27+
- [ ] Task 2.3: Implement keyword search and output formatting
28+
- [ ] Task 2.4: Add refresh triggers (TTL, `--refresh`)
29+
30+
### Phase 3: Integration & Polish
31+
- [ ] Task 3.1: Error handling and offline behavior
32+
- [ ] Task 3.2: Add docs and examples to CLI help
33+
- [ ] Task 3.3: Write unit/integration tests
34+
35+
## Dependencies
36+
**What needs to happen in what order?**
37+
38+
- Index schema defined before index builder and search.
39+
- Registry access strategy finalized before implementation.
40+
- Tests depend on stable CLI outputs and fixtures.
41+
- External dependency: access to registry repos or hosted manifests.
42+
43+
## Timeline & Estimates
44+
**When will things be done?**
45+
46+
- Phase 1: 0.5-1 day
47+
- Phase 2: 1-2 days
48+
- Phase 3: 1 day
49+
- Buffer: 0.5 day for unknowns
50+
51+
## Risks & Mitigation
52+
**What could go wrong?**
53+
54+
- Technical risks
55+
- Registry access blocked (auth, rate limits)
56+
- Index refresh too slow for large registries
57+
- Resource risks
58+
- Limited time for testing edge cases
59+
- Dependency risks
60+
- Registry does not expose metadata
61+
- Mitigation strategies
62+
- Cache and use stale index on failure
63+
- Add `--refresh` and `--ttl` options
64+
- Support optional registry-provided index file
65+
66+
## Resources Needed
67+
**What do we need to succeed?**
68+
69+
- Team members and roles
70+
- CLI maintainer, reviewer
71+
- Tools and services
72+
- Git CLI, HTTP client
73+
- Infrastructure
74+
- Local cache directory
75+
- Documentation/knowledge
76+
- Registry format and sample repos
77+

‎docs/ai/requirements/feature-skill-find.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
---
2+
phase: requirements
3+
title: Requirements & Problem Understanding
4+
description: Clarify the problem space, gather requirements, and define success criteria
5+
---
6+
7+
# Requirements & Problem Understanding
8+
9+
## Problem Statement
10+
**What problem are we solving?**
11+
12+
- Users cannot quickly discover available skills across registries by keyword.
13+
- Skill registries are distributed across multiple repos, each with a `skills/` folder.
14+
- Current search requires cloning or scanning full registries, which is slow and wasteful.
15+
16+
## Goals & Objectives
17+
**What do we want to achieve?**
18+
19+
- Provide `npx ai-devkit skill find <keyword>` to list matching skills.
20+
- Make search fast without cloning full registries.
21+
- Keep results reasonably fresh via a lightweight index update trigger.
22+
- Keep output useful for install (skill name + short description).
23+
- Non-goals: downloading or installing skills in this feature; ranking by popularity.
24+
25+
## User Stories & Use Cases
26+
**How will users interact with the solution?**
27+
28+
- As a user, I want to search by keyword so that I can find relevant skills to install.
29+
- As a user, I want results to include the registry and skill path so I can install it.
30+
- As a user, I want search to be fast even with many registries.
31+
- As a maintainer, I want indexing to avoid full repo clones.
32+
- Edge cases: no matches, multiple registries with same skill name, offline mode.
33+
34+
## Success Criteria
35+
**How will we know when we're done?**
36+
37+
- `npx ai-devkit skill find typescript` returns matching skills within a few seconds.
38+
- Index update does not clone full registries for typical usage.
39+
- Search works across all registries in `skills/registry.json`.
40+
- Clear output format and exit codes for no matches or errors.
41+
42+
## Constraints & Assumptions
43+
**What limitations do we need to work within?**
44+
45+
- Registries are Git repos with a `skills/` directory containing skill folders.
46+
- Users may have limited bandwidth; avoid large downloads.
47+
- Assume Git is available or use HTTP fetch if possible.
48+
- Assume registries can be accessed without authentication (or handle auth errors).
49+
50+
## Questions & Open Items
51+
**What do we still need to clarify?**
52+
53+
- Do registries expose a manifest or index we can fetch without Git?
54+
- Description is required; fetched from `SKILL.md` inside each skill folder.
55+
- Index refresh should offer options and allow async rebuild; start with a simpler sync path.
56+
- Local index stored at `~/.ai-devkit/skills.json`.
57+

0 commit comments

Comments
 (0)