Skip to content

Commit dea5322

Browse files
test(readiness): cover provider agent network matrix
1 parent 621efd1 commit dea5322

5 files changed

Lines changed: 352 additions & 3 deletions

File tree

.beads/issues.jsonl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1119,7 +1119,7 @@
11191119
{"id":"bd-xvvd","title":"Add 'Type yes' prompt reassurance and explanation","description":"# Task: Add 'Type yes' Prompt Reassurance\n\n## Parent Epic\n[EPIC] First SSH Connection Experience (agentic_coding_flywheel_setup-q7r3)\n\n## Description\nThe SSH host authenticity warning is TERRIFYING to beginners. It uses security language (\"fingerprint\", \"Are you sure?\") that sounds like a virus warning. Users need reassurance that this is normal.\n\n## Implementation Details\nLocation: apps/web/app/wizard/ssh-connect/page.tsx\n\nAdd OutputPreview showing the scary prompt with explanation:\n\n\\`\\`\\`tsx\n<div className=\"space-y-3\">\n <h3 className=\"font-semibold\">You'll see a security message — that's normal!</h3>\n <OutputPreview title=\"This message looks scary but is safe:\">\n <p className=\"text-muted-foreground\">The authenticity of host '45.123.67.89' can't be established.</p>\n <p className=\"text-muted-foreground\">ED25519 key fingerprint is SHA256:xYz123...</p>\n <p className=\"text-amber-500\">Are you sure you want to continue connecting (yes/no)?</p>\n </OutputPreview>\n \n <AlertCard variant=\"info\" icon={Shield}>\n <strong>This is normal!</strong> Your computer is saying: \"I've never talked to this\n server before. Do you trust it?\" Since you just created this VPS, the answer is yes.\n <br /><br />\n Type <kbd className=\"rounded bg-muted px-1.5 py-0.5 font-mono text-xs\">yes</kbd>\n (the full word, not just \"y\") and press Enter.\n </AlertCard>\n</div>\n\\`\\`\\`\n\n## Acceptance Criteria\n- [ ] Shows the actual prompt users will see\n- [ ] Explains why it appears in simple terms\n- [ ] Reassures that it's expected and safe\n- [ ] Emphasizes typing \"yes\" not just \"y\"\n- [ ] Uses calming visual style (info/green, not warning/red)\n\n## UI/UX Notes\n- This MUST appear before the SSH command, not buried in SimplerGuide\n- Use a Shield icon to reinforce safety\n- The explanation should feel like a friendly heads-up, not a warning\n- Consider adding: \"You'll only see this once per VPS\"","status":"closed","priority":0,"issue_type":"task","created_at":"2025-12-22T18:41:14.730547Z","updated_at":"2025-12-22T19:02:13.053967Z","closed_at":"2025-12-22T19:02:13.053967Z","close_reason":"Already covered: ssh-connect/page.tsx Step 5 explains 'Type yes' prompt in detail","source_repo":".","compaction_level":0,"original_size":0}
11201120
{"id":"bd-xwp3s","title":"Create redacted swarm support bundle report index","description":"## Background\nSupport bundles now collect swarm timeline diagnostics, status, and provenance. Users still need a browsable report index that explains what was captured, what was redacted, and which files matter first.\n\n## Goal\nGenerate a small redacted Markdown or HTML index inside the support bundle that links swarm status, timeline, simulation/rehearsal artifacts, provenance, and manifest diagnostics.\n\n## Acceptance Criteria\n- Report clearly labels sensitive redactions and degraded probes.\n- Report links only files present in the bundle and handles missing optional artifacts gracefully.\n- Report generation is read-only within the bundle workspace, performs no network/process execution, and fails closed for unknown sensitive fields.\n- Tests cover full bundle, minimal bundle, redacted path/command snippets, malformed optional JSON, and unknown-sensitive-field handling.\n- No large binary artifacts are introduced.","status":"closed","priority":2,"issue_type":"feature","estimated_minutes":100,"created_at":"2026-05-08T09:47:20.170750060Z","created_by":"ubuntu","updated_at":"2026-05-08T12:19:15.402701948Z","closed_at":"2026-05-08T12:19:15.402428676Z","close_reason":"Added redacted support-report.md index for support bundles with present-file links, degraded JSON labeling, sensitive-field name-only review, and focused unit/VM coverage.","source_repo":".","compaction_level":0,"original_size":0,"labels":["observability","support","swarm"]}
11211121
{"id":"bd-xy325","title":"Add supply-chain governance tests and CI artifacts","description":"## What\nAdd governance test coverage and CI artifacts for the supply-chain and drift epic.\n\n## Why\nThe governance work is a release safety net. Tests need to prove that stale generated files, stale checksums, and mismatched docs are caught before maintainers tag or publish a release.\n\n## Coverage Requirements\n- Manifest drift mismatches across generated scripts, website metadata, docs, onboarding lessons, and doctor checks.\n- Verified installer checksum candidate behavior: timestamp-only diff, target-only diff, unrelated diff.\n- Release-doctor JSON and human output.\n- Opt-in network checks mocked or fixture-driven for deterministic CI.\n\n## Risks\nAvoid network-dependent flakes in default CI. Live upstream probes should be opt-in or gracefully skipped with explicit status.\n\n## Logging\nTests should print the drift class, expected owner surface, and candidate artifact path.\n\n## Acceptance Criteria\n- ShellCheck/Bun gates are wired only for the surfaces actually touched.\n- `bv --robot-insights` remains cycle-free after dependency changes.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-05-08T13:51:59.897627990Z","created_by":"ubuntu","updated_at":"2026-05-08T19:12:47.531005577Z","closed_at":"2026-05-08T19:12:47.530610718Z","close_reason":"Added governance test coverage for release-doctor checksum candidates, manifest drift surfaces, stack provenance, and wired CI artifacts","source_repo":".","compaction_level":0,"original_size":0,"labels":["checksums","ci","idea-wizard","manifest","next-wave","security","tests"],"dependencies":[{"issue_id":"bd-xy325","depends_on_id":"bd-qsljs","type":"blocks","created_at":"2026-05-08T13:54:18.774589981Z","created_by":"ubuntu","metadata":"{}","thread_id":""},{"issue_id":"bd-xy325","depends_on_id":"bd-s984n","type":"blocks","created_at":"2026-05-08T13:54:13.229504627Z","created_by":"ubuntu","metadata":"{}","thread_id":""}]}
1122-
{"id":"bd-xyycz","title":"Add provider agent and network readiness test matrix","description":"## What\nAdd readiness e2e and fixture tests for provider, agent account, and offline/cache preflights.\n\n## Why\nReadiness guidance spans web UI, CLI output, network checks, and auth-state detection. Tests need to prevent contradictory recommendations or credential leaks across those surfaces.\n\n## Coverage Requirements\n- Web wizard provider readiness scenarios.\n- CLI JSON/human output for supported, unknown, and unsafe provider choices.\n- Agent auth audit fixtures with fake configs and no secret leakage.\n- Network preflight timeout/offline/reachable fixtures.\n\n## Risks\nDo not rely on live provider pricing or real third-party auth state in default tests. Use fixtures for deterministic results and keep live checks opt-in.\n\n## Logging\nTests should log readiness category, selected recommendation, and artifact path. They must not log credentials.\n\n## Acceptance Criteria\n- Bun type-check/lint/build gates are run for web changes.\n- Bash syntax and ShellCheck pass for script changes.","status":"open","priority":2,"issue_type":"task","created_at":"2026-05-08T13:52:54.942407823Z","created_by":"ubuntu","updated_at":"2026-05-08T14:02:28.436459828Z","source_repo":".","compaction_level":0,"original_size":0,"labels":["agent-readiness","e2e","idea-wizard","next-wave","provider","tests","unit-tests"],"dependencies":[{"issue_id":"bd-xyycz","depends_on_id":"bd-in4xd","type":"blocks","created_at":"2026-05-08T13:55:18.783151496Z","created_by":"ubuntu","metadata":"{}","thread_id":""},{"issue_id":"bd-xyycz","depends_on_id":"bd-oaita","type":"blocks","created_at":"2026-05-08T13:55:24.175144226Z","created_by":"ubuntu","metadata":"{}","thread_id":""}]}
1122+
{"id":"bd-xyycz","title":"Add provider agent and network readiness test matrix","description":"## What\nAdd readiness e2e and fixture tests for provider, agent account, and offline/cache preflights.\n\n## Why\nReadiness guidance spans web UI, CLI output, network checks, and auth-state detection. Tests need to prevent contradictory recommendations or credential leaks across those surfaces.\n\n## Coverage Requirements\n- Web wizard provider readiness scenarios.\n- CLI JSON/human output for supported, unknown, and unsafe provider choices.\n- Agent auth audit fixtures with fake configs and no secret leakage.\n- Network preflight timeout/offline/reachable fixtures.\n\n## Risks\nDo not rely on live provider pricing or real third-party auth state in default tests. Use fixtures for deterministic results and keep live checks opt-in.\n\n## Logging\nTests should log readiness category, selected recommendation, and artifact path. They must not log credentials.\n\n## Acceptance Criteria\n- Bun type-check/lint/build gates are run for web changes.\n- Bash syntax and ShellCheck pass for script changes.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-05-08T13:52:54.942407823Z","created_by":"ubuntu","updated_at":"2026-05-08T19:33:51.333790764Z","closed_at":"2026-05-08T19:33:51.333551586Z","close_reason":"Added provider, agent-auth, and network readiness matrix coverage","source_repo":".","compaction_level":0,"original_size":0,"labels":["agent-readiness","e2e","idea-wizard","next-wave","provider","tests","unit-tests"],"dependencies":[{"issue_id":"bd-xyycz","depends_on_id":"bd-in4xd","type":"blocks","created_at":"2026-05-08T13:55:18.783151496Z","created_by":"ubuntu","metadata":"{}","thread_id":""},{"issue_id":"bd-xyycz","depends_on_id":"bd-oaita","type":"blocks","created_at":"2026-05-08T13:55:24.175144226Z","created_by":"ubuntu","metadata":"{}","thread_id":""}]}
11231123
{"id":"bd-y0d","title":"Docs reference missing VM test script","description":"AGENTS.md and README.md reference tests/vm/test_install_ubuntu.sh, but only tests/vm/.gitkeep exists. This confuses contributors and breaks documented commands.","acceptance_criteria":"Either:\n- Add tests/vm/test_install_ubuntu.sh that actually runs a meaningful installer test, OR\n- Update AGENTS.md and README.md to remove/replace the reference.","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-12-20T19:48:12.189137Z","updated_at":"2025-12-20T19:52:41.740620Z","closed_at":"2025-12-20T19:52:41.740620Z","close_reason":"Added missing tests/vm/test_install_ubuntu.sh (Docker-based integration test) and updated README/AGENTS installer testing docs.","source_repo":".","compaction_level":0,"original_size":0}
11241124
{"id":"bd-y0g9","title":"Bug: Beads daemon clobbers .beads/issues.jsonl on feature branches","description":"Observed bd daemon auto-import pulling sync branch (configured as main) and overwriting .beads/issues.jsonl in the working tree even when current branch HEAD contains newer committed beads. This causes persistent dirty working trees and blocks git pull/rebase.\\n\\nRepro: start bd daemon, create/commit a beads change on a non-main local branch that is ahead of origin/main, then any JSONL write triggers daemon to pull sync branch main and rewrite .beads/issues.jsonl back to origin/main version.\\n\\nFix: disable auto-start-daemon by default for this repo and document safe usage (run daemon only on main, or use --local).","status":"closed","priority":1,"issue_type":"bug","created_at":"2025-12-25T15:06:18.006234Z","updated_at":"2025-12-25T15:07:44.877861Z","closed_at":"2025-12-25T15:07:44.877861Z","close_reason":"Disable auto-start daemon in repo config and document safe daemon usage","source_repo":".","compaction_level":0,"original_size":0}
11251125
{"id":"bd-y4gh","title":"Add RU to commands.ts","description":"# Task: Add RU to Commands Reference\n\n## Location\nFile: apps/web/lib/commands.ts\nInsert after: Line ~235 (after 'am' Agent Mail entry, within 'stack' category)\n\n## What commands.ts Is\nThis file defines all CLI commands shown on /learn/commands page.\nIt's the quick reference users consult to remember command syntax.\n\n## Exact Code to Add\n\n```typescript\n{\n name: 'ru',\n fullName: 'Repo Updater',\n description: 'Multi-repo sync with AI-driven commit automation.',\n category: 'stack',\n example: 'ru sync -j4',\n aliases: ['repo-updater'],\n docsUrl: '/learn/ru',\n},\n```\n\n## Field Explanations\n- name: Short command name (what user types)\n- fullName: Human-readable name\n- description: One-line description\n- category: 'stack' (Dicklesworthstone tools)\n- example: Most common usage\n- aliases: Alternative names (optional)\n- docsUrl: Link to detailed lesson\n\n## Verification\n- /learn/commands page lists ru under 'Stack' category\n- Clicking ru shows example and links to docs","status":"closed","priority":1,"issue_type":"task","created_at":"2026-01-11T04:01:26.445084225Z","created_by":"ubuntu","updated_at":"2026-01-11T04:21:43.789847253Z","closed_at":"2026-01-11T04:21:43.789847253Z","close_reason":"Added RU command entry to commands.ts in stack category","source_repo":".","compaction_level":0,"original_size":0,"dependencies":[{"issue_id":"bd-y4gh","depends_on_id":"bd-pkvn","type":"blocks","created_at":"2026-01-11T04:05:32.027631263Z","created_by":"ubuntu","metadata":"","thread_id":""}]}

apps/web/e2e/wizard-flow.spec.ts

Lines changed: 65 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { test, expect, Page } from "@playwright/test";
2-
import { readFile } from "node:fs/promises";
2+
import { readFile, writeFile } from "node:fs/promises";
33

44
/**
55
* Standard timeouts for different scenarios.
@@ -256,6 +256,70 @@ test.describe("Wizard Flow", () => {
256256
await expect(summary).toContainText("listed 48/64 GB VPS plans are undersized");
257257
});
258258

259+
test("should surface provider readiness states for supported, unknown, and unsafe choices", async ({ page }, testInfo) => {
260+
await setupWizardState(page, { os: "mac", completedSteps: [1, 2, 3] });
261+
await page.goto("/wizard/rent-vps");
262+
await page.waitForLoadState("domcontentloaded");
263+
264+
const readiness = page.getByTestId("provider-readiness-check");
265+
const providerSelect = readiness.getByLabel("Provider");
266+
const ubuntuSelect = readiness.getByLabel("Ubuntu image");
267+
const artifactPath = testInfo.outputPath("provider-readiness-matrix.json");
268+
const matrixLog: Array<{
269+
readinessCategory: string;
270+
selectedRecommendation: string;
271+
expectedLabel: string;
272+
artifactPath: string;
273+
}> = [];
274+
275+
const recordState = async (
276+
readinessCategory: string,
277+
selectedRecommendation: string,
278+
expectedLabel: string,
279+
expectedSummary: RegExp | string
280+
) => {
281+
matrixLog.push({
282+
readinessCategory,
283+
selectedRecommendation,
284+
expectedLabel,
285+
artifactPath,
286+
});
287+
await expect(readiness).toContainText(expectedLabel);
288+
await expect(readiness).toContainText(expectedSummary);
289+
};
290+
291+
await recordState(
292+
"supported",
293+
"Contabo Cloud VPS 50",
294+
"Supported",
295+
"Ready for the selected target."
296+
);
297+
298+
await providerSelect.selectOption("other");
299+
await recordState(
300+
"unknown",
301+
"manual spec comparison",
302+
"Unknown",
303+
"Not in the ACFS provider table; compare the specs manually."
304+
);
305+
306+
await providerSelect.selectOption("ovh");
307+
await expect(readiness.getByLabel("Plan")).toHaveValue("VPS-5");
308+
await ubuntuSelect.selectOption("20.04");
309+
await recordState(
310+
"unsafe",
311+
"choose Ubuntu 24.04+ before checkout",
312+
"Unsupported",
313+
/Ubuntu 20\.04 is below the ACFS minimum/
314+
);
315+
316+
await writeFile(artifactPath, JSON.stringify(matrixLog, null, 2));
317+
await testInfo.attach("provider-readiness-matrix", {
318+
path: artifactPath,
319+
contentType: "application/json",
320+
});
321+
});
322+
259323
test("should complete step 5: Create VPS with IP address", async ({ page }) => {
260324
// Set up prerequisite state
261325
await page.goto("/wizard/os-selection");

apps/web/lib/vpsProviders.test.ts

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,25 @@ import {
33
calculateRequiredSpecs,
44
getWorkloadProfile,
55
validateVPSReadiness,
6+
type VPSReadinessCheckId,
7+
type VPSReadinessInput,
68
type VPSReadinessResult,
9+
type VPSReadinessStatus,
710
} from "./vpsProviders";
811

912
function checkStatus(result: VPSReadinessResult, id: string) {
1013
return result.checks.find((check) => check.id === id)?.status;
1114
}
1215

16+
type ProviderReadinessScenario = {
17+
category: string;
18+
selectedRecommendation: string;
19+
artifactPath: string;
20+
input: VPSReadinessInput;
21+
expectedStatus: VPSReadinessStatus;
22+
expectedChecks: Partial<Record<VPSReadinessCheckId, VPSReadinessStatus>>;
23+
};
24+
1325
describe("VPS capacity sizing", () => {
1426
test("keeps the wizard calculator aligned with the standard ACFS profile", () => {
1527
const standard = getWorkloadProfile("standard");
@@ -29,6 +41,86 @@ describe("VPS capacity sizing", () => {
2941
});
3042

3143
describe("validateVPSReadiness", () => {
44+
test("covers supported, unknown, and unsafe provider choices as a readiness matrix", () => {
45+
const scenarios: ProviderReadinessScenario[] = [
46+
{
47+
category: "supported",
48+
selectedRecommendation: "Contabo Cloud VPS 50",
49+
artifactPath: "apps/web/lib/vpsProviders.test.ts#provider-readiness-matrix",
50+
input: {
51+
providerId: "contabo",
52+
planName: "Cloud VPS 50",
53+
ubuntuVersion: "25.10",
54+
region: "us",
55+
targetAgents: 10,
56+
workloadId: "standard",
57+
},
58+
expectedStatus: "supported",
59+
expectedChecks: {
60+
provider: "supported",
61+
plan: "supported",
62+
os: "supported",
63+
region: "supported",
64+
capacity: "supported",
65+
},
66+
},
67+
{
68+
category: "unknown",
69+
selectedRecommendation: "manual spec comparison",
70+
artifactPath: "apps/web/lib/vpsProviders.test.ts#provider-readiness-matrix",
71+
input: {
72+
providerId: "other",
73+
planName: "custom plan",
74+
ubuntuVersion: "25.10",
75+
region: "not-listed",
76+
targetAgents: 10,
77+
workloadId: "standard",
78+
},
79+
expectedStatus: "unknown",
80+
expectedChecks: {
81+
provider: "unknown",
82+
plan: "unknown",
83+
os: "unknown",
84+
region: "unknown",
85+
},
86+
},
87+
{
88+
category: "unsafe",
89+
selectedRecommendation: "choose Ubuntu 24.04+ and a larger host",
90+
artifactPath: "apps/web/lib/vpsProviders.test.ts#provider-readiness-matrix",
91+
input: {
92+
providerId: "ovh",
93+
planName: "VPS-4",
94+
ubuntuVersion: "20.04",
95+
region: "us-east",
96+
targetAgents: 25,
97+
workloadId: "heavy",
98+
},
99+
expectedStatus: "unsupported",
100+
expectedChecks: {
101+
provider: "supported",
102+
plan: "supported",
103+
os: "unsupported",
104+
region: "supported",
105+
capacity: "unsupported",
106+
},
107+
},
108+
];
109+
110+
for (const scenario of scenarios) {
111+
const result = validateVPSReadiness(scenario.input);
112+
113+
expect(result.status).toBe(scenario.expectedStatus);
114+
for (const [checkId, status] of Object.entries(scenario.expectedChecks) as Array<
115+
[VPSReadinessCheckId, VPSReadinessStatus]
116+
>) {
117+
expect(checkStatus(result, checkId)).toBe(status);
118+
}
119+
expect(scenario.artifactPath).toContain("vpsProviders.test.ts");
120+
expect(scenario.selectedRecommendation.length).toBeGreaterThan(0);
121+
}
122+
});
123+
32124
test("supports a recommended provider plan, Ubuntu image, region, and target", () => {
33125
const result = validateVPSReadiness({
34126
providerId: "contabo",

0 commit comments

Comments
 (0)