Skip to content

Commit cc01af1

Browse files
committed
feat: add config.yaml and platform parameter docs for Aegis skill system
- Create docs/skill-params.md documenting platform env vars (AEGIS_GATEWAY_URL, AEGIS_VLM_URL, AEGIS_SKILL_ID, AEGIS_SKILL_PARAMS, AEGIS_PORTS) and the config.yaml format for user-configurable params - Create config.yaml for home-security-benchmark with mode (llm/vlm/full) and noOpen user params — no platform params (Aegis auto-injects those) - Update run-benchmark.cjs to read AEGIS_SKILL_PARAMS: merge skillParams.noOpen into NO_OPEN, add TEST_MODE for suite filtering (llm skips VLM Scene Analysis, vlm keeps only VLM suites, full runs all) - Update SKILL.md with User Configuration section referencing config.yaml and linking to docs/skill-params.md
1 parent a6fc750 commit cc01af1

4 files changed

Lines changed: 153 additions & 6 deletions

File tree

docs/skill-params.md

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# Aegis Skill Platform Parameters
2+
3+
Aegis automatically injects these environment variables to every skill process. Skills should **not** ask users to configure these — they are provided by the platform.
4+
5+
## Platform Parameters (auto-injected)
6+
7+
| Env Var | Type | Description |
8+
|---------|------|-------------|
9+
| `AEGIS_GATEWAY_URL` | `string` | LLM gateway endpoint (e.g. `http://localhost:5407`). Proxies to whatever LLM provider the user has configured (OpenAI, Anthropic, local). Skills should use this for all LLM calls — it handles auth, routing, and model selection. |
10+
| `AEGIS_VLM_URL` | `string` | Local VLM (Vision Language Model) server endpoint (e.g. `http://localhost:5405`). Available when the user has a local VLM running. |
11+
| `AEGIS_SKILL_ID` | `string` | The skill's unique identifier (e.g. `home-security-benchmark`). |
12+
| `AEGIS_SKILL_PARAMS` | `JSON string` | User-configured parameters from `config.yaml` (see below). |
13+
| `AEGIS_PORTS` | `JSON string` | All Aegis service ports as JSON. Use the URL vars above instead of parsing this directly. |
14+
15+
## User Parameters (from config.yaml)
16+
17+
Skills can define user-configurable parameters in a `config.yaml` file alongside `SKILL.md`. Aegis parses this at install time and renders a config panel in the UI. User values are passed as JSON via `AEGIS_SKILL_PARAMS`.
18+
19+
### config.yaml Format
20+
21+
```yaml
22+
params:
23+
- key: mode
24+
label: Test Mode
25+
type: select
26+
options: [option1, option2, option3]
27+
default: option1
28+
description: "Human-readable description shown in the config panel"
29+
30+
- key: verbose
31+
label: Verbose Output
32+
type: boolean
33+
default: false
34+
description: "Enable detailed logging"
35+
36+
- key: threshold
37+
label: Confidence Threshold
38+
type: number
39+
default: 0.7
40+
description: "Minimum confidence score (0.0–1.0)"
41+
42+
- key: apiEndpoint
43+
label: Custom API Endpoint
44+
type: string
45+
default: ""
46+
description: "Optional override for external API"
47+
```
48+
49+
Supported types: `string`, `boolean`, `select`, `number`
50+
51+
### Reading config.yaml in Your Skill
52+
53+
```javascript
54+
// Node.js — parse AEGIS_SKILL_PARAMS
55+
let skillParams = {};
56+
try { skillParams = JSON.parse(process.env.AEGIS_SKILL_PARAMS || '{}'); } catch {}
57+
58+
const mode = skillParams.mode || 'default';
59+
const verbose = skillParams.verbose || false;
60+
```
61+
62+
```python
63+
# Python — parse AEGIS_SKILL_PARAMS
64+
import os, json
65+
66+
skill_params = json.loads(os.environ.get('AEGIS_SKILL_PARAMS', '{}'))
67+
mode = skill_params.get('mode', 'default')
68+
verbose = skill_params.get('verbose', False)
69+
```
70+
71+
### Precedence
72+
73+
```
74+
CLI flags > AEGIS_SKILL_PARAMS > Platform env vars > Defaults
75+
```
76+
77+
When a skill supports both CLI arguments and `AEGIS_SKILL_PARAMS`, CLI flags should take priority. Platform-injected env vars (like `AEGIS_GATEWAY_URL`) are always available regardless of `config.yaml`.
78+
79+
## Gateway as Proxy
80+
81+
The gateway (`AEGIS_GATEWAY_URL`) is an OpenAI-compatible proxy. Skills call it like any OpenAI endpoint — the gateway handles:
82+
83+
- **API key management** — user configures keys in Aegis settings
84+
- **Provider routing** — OpenAI, Anthropic, local models
85+
- **Model selection** — user picks model in Aegis UI
86+
87+
Skills should **not** need raw API keys. If a skill needs direct provider access in the future, Aegis will expose additional env vars (`AEGIS_LLM_API_KEY`, `AEGIS_LLM_PROVIDER`, etc.) — but this is not yet implemented.
88+
89+
### Example: Calling the Gateway
90+
91+
```javascript
92+
const gatewayUrl = process.env.AEGIS_GATEWAY_URL || 'http://localhost:5407';
93+
94+
const response = await fetch(`${gatewayUrl}/v1/chat/completions`, {
95+
method: 'POST',
96+
headers: { 'Content-Type': 'application/json' },
97+
body: JSON.stringify({
98+
messages: [{ role: 'user', content: 'Hello' }],
99+
stream: false,
100+
}),
101+
});
102+
```
103+
104+
No API key header needed — the gateway injects it.

skills/analysis/home-security-benchmark/SKILL.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,17 @@ node scripts/run-benchmark.cjs --no-open
5959

6060
> **Note**: URLs should be base URLs (e.g. `http://localhost:5405`). The benchmark appends `/v1/chat/completions` automatically. Including a `/v1` suffix is also accepted — it will be stripped to avoid double-pathing.
6161
62+
### User Configuration (config.yaml)
63+
64+
This skill includes a [`config.yaml`](config.yaml) that defines user-configurable parameters. Aegis parses this at install time and renders a config panel in the UI. Values are delivered via `AEGIS_SKILL_PARAMS`.
65+
66+
| Parameter | Type | Default | Description |
67+
|-----------|------|---------|-------------|
68+
| `mode` | select | `llm` | Which suites to run: `llm` (96 tests), `vlm` (35 tests), or `full` (131 tests) |
69+
| `noOpen` | boolean | `false` | Skip auto-opening the HTML report in browser |
70+
71+
Platform parameters like `AEGIS_GATEWAY_URL` and `AEGIS_VLM_URL` are auto-injected by Aegis — they are **not** in `config.yaml`. See [Aegis Skill Platform Parameters](../../../docs/skill-params.md) for the full platform contract.
72+
6273
### CLI Arguments (standalone fallback)
6374

6475
| Argument | Default | Description |
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
params:
2+
- key: mode
3+
label: Test Mode
4+
type: select
5+
options: [llm, vlm, full]
6+
default: llm
7+
description: "Which test suites to run: llm-only, vlm-only, or full"
8+
9+
- key: noOpen
10+
label: Don't auto-open report
11+
type: boolean
12+
default: false
13+
description: Skip opening the HTML report in browser after completion

skills/analysis/home-security-benchmark/scripts/run-benchmark.cjs

Lines changed: 25 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -73,18 +73,19 @@ Tests: 131 total (96 LLM + 35 VLM) across 16 suites
7373
process.exit(0);
7474
}
7575

76+
// Parse skill parameters if running as Aegis skill
77+
let skillParams = {};
78+
try { skillParams = JSON.parse(process.env.AEGIS_SKILL_PARAMS || '{}'); } catch { }
79+
7680
// Aegis provides config via env vars; CLI args are fallback for standalone
7781
const GATEWAY_URL = process.env.AEGIS_GATEWAY_URL || getArg('gateway', 'http://localhost:5407');
7882
const VLM_URL = process.env.AEGIS_VLM_URL || getArg('vlm', '');
7983
const RESULTS_DIR = getArg('out', path.join(os.homedir(), '.aegis-ai', 'benchmarks'));
80-
const NO_OPEN = args.includes('--no-open');
84+
const IS_SKILL_MODE = !!process.env.AEGIS_SKILL_ID;
85+
const NO_OPEN = args.includes('--no-open') || skillParams.noOpen || false;
86+
const TEST_MODE = skillParams.mode || 'full';
8187
const TIMEOUT_MS = 30000;
8288
const FIXTURES_DIR = path.join(__dirname, '..', 'fixtures');
83-
const IS_SKILL_MODE = !!process.env.AEGIS_SKILL_ID;
84-
85-
// Parse skill parameters if running as Aegis skill
86-
let skillParams = {};
87-
try { skillParams = JSON.parse(process.env.AEGIS_SKILL_PARAMS || '{}'); } catch { }
8889

8990
// ─── Skill Protocol: JSON lines on stdout, human text on stderr ──────────────
9091

@@ -1706,6 +1707,24 @@ async function main() {
17061707
// Emit ready event (Aegis listens for this)
17071708
emit({ event: 'ready', model: results.model.name, system: results.system.cpu });
17081709

1710+
// Filter suites by test mode (from AEGIS_SKILL_PARAMS or default 'full')
1711+
if (TEST_MODE !== 'full') {
1712+
const isVlmSuite = (name) => name.includes('VLM Scene') || name.includes('📸');
1713+
const originalCount = suites.length;
1714+
if (TEST_MODE === 'llm') {
1715+
// Remove VLM image-analysis suites (VLM-to-Alert Triage stays — it's LLM-based text triage)
1716+
for (let i = suites.length - 1; i >= 0; i--) {
1717+
if (isVlmSuite(suites[i].name)) suites.splice(i, 1);
1718+
}
1719+
} else if (TEST_MODE === 'vlm') {
1720+
// Keep only VLM image-analysis suites (requires VLM URL)
1721+
for (let i = suites.length - 1; i >= 0; i--) {
1722+
if (!isVlmSuite(suites[i].name)) suites.splice(i, 1);
1723+
}
1724+
}
1725+
log(` Filter: ${TEST_MODE} mode → ${suites.length}/${originalCount} suites selected`);
1726+
}
1727+
17091728
const suiteStart = Date.now();
17101729
await runSuites();
17111730
results.totals.timeMs = Date.now() - suiteStart;

0 commit comments

Comments
 (0)