Skip to content

Commit 14dcca8

Browse files
committed
Create a skill to author HIPs
Signed-off-by: George Jenkins <gvjenkins@gmail.com>
1 parent 82cca09 commit 14dcca8

24 files changed

Lines changed: 2964 additions & 0 deletions

File tree

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
{
2+
"skill_name": "hip-author",
3+
"evals": [
4+
{
5+
"id": 0,
6+
"name": "feature-hip",
7+
"prompt": "I want to write a HIP for adding native support for chart signing using Sigstore/cosign instead of the current PGP-based provenance system. The current .prov file approach is clunky and almost nobody uses it because PGP key management is painful. Cosign is already the standard in the container ecosystem and it would make chart verification way more accessible.",
8+
"expected_output": "A well-structured HIP document with all required sections, proper YAML preamble with hip: 9999, type: feature, status: draft. Should address backwards compatibility with existing .prov files.",
9+
"files": []
10+
},
11+
{
12+
"id": 1,
13+
"name": "process-hip",
14+
"prompt": "I think we should create a formal process for how Helm handles security embargoes and coordinated disclosure with downstream distributors. Right now the SECURITY.md covers reporting but there's no documented process for how we coordinate with distro maintainers who package Helm before a CVE goes public.",
15+
"expected_output": "A well-structured HIP document with type: process, addressing the coordination gap in security disclosure. Should reference existing SECURITY.md and governance docs.",
16+
"files": []
17+
},
18+
{
19+
"id": 2,
20+
"name": "vague-idea",
21+
"prompt": "I have a half-baked idea - what if Helm charts could somehow declare their resource requirements or dependencies on cluster capabilities? Like, a chart could say 'I need an ingress controller' or 'I need at least 3 nodes' and helm install would check that before deploying. I'm not sure if this is even feasible or if it's too broad.",
22+
"expected_output": "Should help refine the vague idea into something concrete before producing any HIP text. Should probe scope, suggest narrowing, and only proceed to drafting after the idea is focused.",
23+
"files": []
24+
}
25+
]
26+
}
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
{
2+
"eval_id": 0,
3+
"eval_name": "feature-hip",
4+
"prompt": "I want to write a HIP for adding native support for chart signing using Sigstore/cosign instead of the current PGP-based provenance system. The current .prov file approach is clunky and almost nobody uses it because PGP key management is painful. Cosign is already the standard in the container ecosystem and it would make chart verification way more accessible.",
5+
"assertions": [
6+
{
7+
"name": "valid_yaml_preamble",
8+
"description": "HIP starts with valid YAML frontmatter containing all required fields (hip, title, authors, created, type, status)",
9+
"type": "structural"
10+
},
11+
{
12+
"name": "placeholder_hip_number",
13+
"description": "Uses hip: 9999 (or quoted '9999') as placeholder, not an invented number",
14+
"type": "structural"
15+
},
16+
{
17+
"name": "type_is_feature",
18+
"description": "type field is set to 'feature'",
19+
"type": "structural"
20+
},
21+
{
22+
"name": "status_is_draft",
23+
"description": "status field is set to 'draft'",
24+
"type": "structural"
25+
},
26+
{
27+
"name": "all_required_sections",
28+
"description": "Contains all required sections: Abstract, Motivation, Rationale, Specification, Backwards compatibility, Security implications, How to teach this, Reference implementation, Rejected ideas, Open issues, References",
29+
"type": "structural"
30+
},
31+
{
32+
"name": "abstract_concise",
33+
"description": "Abstract is roughly 100-300 words (not a full page, not a single sentence)",
34+
"type": "quality"
35+
},
36+
{
37+
"name": "backwards_compat_addresses_prov",
38+
"description": "Backwards compatibility section explicitly addresses migration from existing .prov/PGP system",
39+
"type": "content"
40+
},
41+
{
42+
"name": "specification_has_details",
43+
"description": "Specification section contains concrete technical details (CLI flags, file formats, or workflow descriptions), not just hand-waving",
44+
"type": "content"
45+
}
46+
]
47+
}
Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
{
2+
"eval_name": "feature-hip",
3+
"variant": "with_skill",
4+
"expectations": [
5+
{
6+
"text": "valid_yaml_preamble",
7+
"passed": true,
8+
"evidence": "Found valid YAML frontmatter"
9+
},
10+
{
11+
"text": "placeholder_hip_number",
12+
"passed": true,
13+
"evidence": "Found hip: 9999 placeholder"
14+
},
15+
{
16+
"text": "status_is_draft",
17+
"passed": true,
18+
"evidence": "status is draft"
19+
},
20+
{
21+
"text": "all_required_sections",
22+
"passed": true,
23+
"evidence": "All 11 required sections present"
24+
},
25+
{
26+
"text": "abstract_concise",
27+
"passed": true,
28+
"evidence": "Abstract is 145 words (target: 100-300)"
29+
},
30+
{
31+
"text": "type_is_feature",
32+
"passed": true,
33+
"evidence": "type is feature"
34+
},
35+
{
36+
"text": "backwards_compat_addresses_prov",
37+
"passed": true,
38+
"evidence": "Backwards compat section references .prov/PGP"
39+
},
40+
{
41+
"text": "specification_has_details",
42+
"passed": true,
43+
"evidence": "Specification is 733 words, has code examples"
44+
}
45+
]
46+
}

0 commit comments

Comments
 (0)