Skip to content

Commit 4b0b63b

Browse files
authored
Skill engine: weighted scoring with negative patterns and context guards (#12)
* Skill engine: weighted scoring, negative patterns, context guards, max 3 cap Replace binary keyword matching with confidence-scored suggestions. Phrases carry weights (0.0-1.0), negative patterns suppress false positives, and context guards require confirming words for low-confidence matches. Results ranked by score and capped at 3 per prompt. * Fix CodeRabbit review: lowercase negative pattern, add missing docstring - "DockerClient" → "dockerclient" in docker skill's negative list (mixed-case never matched against lowered input) - Add docstring to main() to meet 80% coverage threshold --------- Co-authored-by: AnExiledDev <AnExiledDev@users.noreply.github.com>
1 parent 03a1dcf commit 4b0b63b

File tree

3 files changed

+484
-293
lines changed

3 files changed

+484
-293
lines changed

.devcontainer/CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,13 @@
1919

2020
### Changed
2121

22+
#### Skill Engine: Auto-Suggestion
23+
- **Weighted scoring** — Skill suggestion phrases now carry confidence weights (0.0–1.0) instead of binary match/no-match. Specific phrases like "build a fastapi app" score 1.0; ambiguous phrases like "start building" score 0.2
24+
- **Negative patterns** — Skills can define substrings that instantly disqualify them. Prevents `fastapi` from triggering when discussing `pydantic-ai`, and `docker` from triggering for `docker-py` prompts
25+
- **Context guards** — Low-confidence matches (score < 0.6) require a confirming context word elsewhere in the prompt. "health check" only suggests `docker` if "docker", "container", or "compose" also appears
26+
- **Ranked results, capped at 3** — Suggestions are sorted by score (then priority tier), and only the top 3 are returned. Eliminates 6+ skill suggestion floods
27+
- **Priority tiers** — Explicit commands (priority 10) outrank technology skills (7), which outrank patterns (5) and generic skills (3) when scores tie
28+
2229
#### Claude Code Installation
2330
- **Claude Code now installs as a native binary** — uses Anthropic's official installer (`https://claude.ai/install.sh`) via new `./features/claude-code-native` feature, replacing the npm-based `ghcr.io/anthropics/devcontainer-features/claude-code:1.0.5`
2431
- **In-session auto-updater now works without root** — native binary at `~/.local/bin/claude` is owned by the container user, so `claude update` succeeds without permission issues

.devcontainer/plugins/devs-marketplace/plugins/skill-engine/README.md

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -39,11 +39,14 @@ Two capabilities:
3939

4040
### Auto-Suggestion
4141

42-
The `skill-suggester.py` hook matches user prompts against keyword maps defined in each skill. When a match is found, it injects a suggestion via `additionalContext` so Claude knows a relevant skill is available.
42+
The `skill-suggester.py` hook scores user prompts against keyword maps for each skill using weighted matching. Suggestions are ranked by confidence and capped at **3 skills maximum** per prompt.
4343

44-
Keywords are defined per skill as:
45-
- **Phrases** — Multi-word patterns (e.g., "docker compose", "REST API")
46-
- **Terms** — Single keywords (e.g., "FastAPI", "pytest")
44+
Each skill defines:
45+
- **Phrases**`(substring, weight)` tuples. Weight 0.0–1.0 reflects specificity (e.g., `("build a fastapi app", 1.0)` vs `("pydantic model", 0.3)`)
46+
- **Terms** — Whole-word regex patterns, all scored at 0.6
47+
- **Negative patterns** — Substrings that instantly disqualify a skill (e.g., `"pydanticai"` suppresses `fastapi`)
48+
- **Context guards** — Required co-occurring words for low-confidence matches. When the best score is below 0.6, at least one guard word must appear in the prompt or the match is dropped
49+
- **Priority** — Integer tie-breaker (10 = commands, 7 = tech, 5 = patterns, 3 = generic)
4750

4851
## How It Works
4952

@@ -56,9 +59,12 @@ User submits a prompt
5659
|
5760
+-> skill-suggester.py
5861
|
59-
+-> Scan prompt against all skill keyword maps
60-
+-> Match found? -> Inject skill suggestion as additionalContext
61-
+-> No match? -> Silent (no output)
62+
+-> Check negative patterns (instant disqualify)
63+
+-> Score phrases (best weight) and terms (0.6)
64+
+-> Enforce context guards on low-confidence matches
65+
+-> Rank by score desc, priority desc
66+
+-> Return top 3 as additionalContext
67+
+-> No matches above threshold? -> Silent (no output)
6268
```
6369

6470
### Skill Structure
@@ -127,7 +133,7 @@ skill-engine/
127133
+-- hooks/
128134
| +-- hooks.json # UserPromptSubmit hook registration
129135
+-- scripts/
130-
| +-- skill-suggester.py # Keyword-based skill auto-suggestion
136+
| +-- skill-suggester.py # Weighted scoring skill auto-suggestion
131137
+-- skills/
132138
| +-- api-design/ # 22 skill directories
133139
| +-- ast-grep-patterns/

0 commit comments

Comments
 (0)