|
| 1 | +# Operator Testing Guide |
| 2 | + |
| 3 | +Use this guide to verify PromptImprover from a clean Windows operator session. |
| 4 | + |
| 5 | +## 1. Enter The Active Package |
| 6 | + |
| 7 | +```powershell |
| 8 | +cd C:\PersonalRepo\portfolio\Promptimprover\universal-refiner |
| 9 | +``` |
| 10 | + |
| 11 | +## 2. Run The Full Release Gate |
| 12 | + |
| 13 | +```powershell |
| 14 | +npm.cmd run release:verify |
| 15 | +``` |
| 16 | + |
| 17 | +Expected result: |
| 18 | + |
| 19 | +- TypeScript build passes. |
| 20 | +- Vitest coverage reports 100% statements, branches, functions, and lines. |
| 21 | +- MCP tool acceptance passes. |
| 22 | +- Semantic fallback acceptance passes. |
| 23 | +- Tracked-turn acceptance links a `prm_...` prompt ID in SQLite. |
| 24 | +- EventStore stress, abrupt recovery, and soak pass. |
| 25 | +- Production and full dependency audits report zero high-or-higher vulnerabilities. |
| 26 | +- Secret scan passes. |
| 27 | +- Package dry-run passes. |
| 28 | +- `acceptance:package-runtime` installs the packed tarball into a temporary global prefix and serves `/api/health`. |
| 29 | + |
| 30 | +## 3. Check Global MCP Registration |
| 31 | + |
| 32 | +```powershell |
| 33 | +powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\register-global.ps1 -Check -ProfileRoot C:\Users\KimHarjamaki -CodexHome C:\codex-home |
| 34 | +``` |
| 35 | + |
| 36 | +Expected result: |
| 37 | + |
| 38 | +```text |
| 39 | +OK Codex |
| 40 | +OK Claude Code MCP |
| 41 | +OK Claude Code hooks |
| 42 | +OK Gemini |
| 43 | +``` |
| 44 | + |
| 45 | +If any row reports `DRIFT`, run `-Apply` only after confirming the drift is expected: |
| 46 | + |
| 47 | +```powershell |
| 48 | +powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\register-global.ps1 -Apply -ProfileRoot C:\Users\KimHarjamaki -CodexHome C:\codex-home |
| 49 | +``` |
| 50 | + |
| 51 | +## 4. List MCP Servers In Each CLI |
| 52 | + |
| 53 | +Use the canonical Windows profile when testing global CLI configuration. This avoids the accented-profile drift that can make a CLI read stale user-level settings: |
| 54 | + |
| 55 | +```powershell |
| 56 | +$env:USERPROFILE = 'C:\Users\KimHarjamaki' |
| 57 | +$env:HOME = 'C:\Users\KimHarjamaki' |
| 58 | +$env:AZURE_CONFIG_DIR = 'C:\Users\KimHarjamaki\.azure' |
| 59 | +``` |
| 60 | + |
| 61 | +```powershell |
| 62 | +codex.cmd mcp list | Select-String -Pattern 'prompt-refiner|obsidian|Connected|enabled' |
| 63 | +claude.cmd mcp list | Select-String -Pattern 'prompt-refiner|obsidian|Connected|Configured' |
| 64 | +gemini.cmd mcp list | Select-String -Pattern 'prompt-refiner|obsidian|Connected|Configured' |
| 65 | +``` |
| 66 | + |
| 67 | +Expected result: |
| 68 | + |
| 69 | +- Codex lists `prompt-refiner` and `obsidian` as enabled. |
| 70 | +- Claude lists `prompt-refiner` and `obsidian` as connected. |
| 71 | +- Gemini lists `prompt-refiner` and `obsidian` as connected. |
| 72 | + |
| 73 | +Codex may show `Unsupported` in the status column for stdio MCP entries. Treat the registration doctor as the authoritative config drift check. |
| 74 | + |
| 75 | +On Windows, Obsidian MCP must be registered with `npx.cmd`, not `npx`. `npx` can resolve to the PowerShell shim (`npx.ps1`) and fail under the default execution policy. |
| 76 | + |
| 77 | +Gemini can report MCP servers as disabled when the current folder is untrusted. Start Gemini from the repo once with `--skip-trust` or accept the workspace trust prompt. Gemini also requires its own auth setup (`GEMINI_API_KEY`, Vertex AI, or Gemini Code Assist) before a trusted full CLI session can start. |
| 78 | + |
| 79 | +## 5. Verify Dashboard Runtime Health |
| 80 | + |
| 81 | +Start or restart the local background runtime: |
| 82 | + |
| 83 | +```powershell |
| 84 | +$repo = 'C:\PersonalRepo\portfolio\Promptimprover\universal-refiner' |
| 85 | +Get-CimInstance Win32_Process | |
| 86 | + Where-Object { $_.Name -eq 'node.exe' -and ($_.CommandLine -match 'universal-refiner.*dist/src/index.js' -or $_.CommandLine -match 'universal-refiner.*dist\\src\\index.js') } | |
| 87 | + ForEach-Object { Stop-Process -Id $_.ProcessId -Force } |
| 88 | +
|
| 89 | +$env:PROMPT_REFINER_BACKGROUND = 'true' |
| 90 | +Start-Process -WindowStyle Hidden -FilePath node -ArgumentList (Join-Path $repo 'dist\src\index.js') -WorkingDirectory $repo |
| 91 | +``` |
| 92 | + |
| 93 | +Check health: |
| 94 | + |
| 95 | +```powershell |
| 96 | +Invoke-RestMethod http://127.0.0.1:3000/api/health | ConvertTo-Json -Depth 6 |
| 97 | +``` |
| 98 | + |
| 99 | +Expected result: |
| 100 | + |
| 101 | +- `runtime.status` is `online`. |
| 102 | +- `semantic.local.enabled` is `true`. |
| 103 | +- `semantic.local.models` includes `gemma3:12b` and `gemma3:1b`. |
| 104 | + |
| 105 | +## 6. Verify Live Gemma Integration |
| 106 | + |
| 107 | +```powershell |
| 108 | +$env:PROMPT_REFINER_ACCEPTANCE_BASE_URL = 'http://localhost:9000/v1' |
| 109 | +npm.cmd run acceptance:gemma:live |
| 110 | +``` |
| 111 | + |
| 112 | +Expected result: |
| 113 | + |
| 114 | +```text |
| 115 | +Live semantic acceptance passed for gemma3:12b and gemma3:1b at http://localhost:9000/v1. |
| 116 | +Semantic acceptance passed: gemma3:12b -> gemma3:1b and outage provider fallback. |
| 117 | +``` |
| 118 | + |
| 119 | +Warnings such as `HTTP 503` for `gemma3:12b` and `fetch failed` during the outage section are expected. They prove fallback and outage paths are being exercised. The command must still exit successfully. |
| 120 | + |
| 121 | +## 7. Verify Packaged Runtime Directly |
| 122 | + |
| 123 | +```powershell |
| 124 | +npm.cmd run acceptance:package-runtime |
| 125 | +``` |
| 126 | + |
| 127 | +Expected result: |
| 128 | + |
| 129 | +```text |
| 130 | +Package runtime smoke passed: installed gemini-prompt-refiner-8.0.0 and served /api/health on <port>. |
| 131 | +``` |
| 132 | + |
| 133 | +This catches missing production dependencies that are hidden by the local workspace. |
| 134 | + |
| 135 | +## 8. Confirm GitHub CI |
| 136 | + |
| 137 | +After pushing a branch and opening a pull request: |
| 138 | + |
| 139 | +```powershell |
| 140 | +gh pr checks <PR_NUMBER> --repo Coding-Autopilot-System/Promptimprover |
| 141 | +gh run list --repo Coding-Autopilot-System/Promptimprover --branch <BRANCH_NAME> --limit 10 |
| 142 | +``` |
| 143 | + |
| 144 | +Expected result: |
| 145 | + |
| 146 | +- `build-and-test` passes. |
| 147 | +- Both acceptance matrix jobs pass. |
| 148 | +- `stress` passes. |
| 149 | +- `windows` passes. |
| 150 | +- `supply-chain` passes. |
| 151 | +- `release-gate` passes. |
| 152 | + |
| 153 | +Remote CI is the authoritative proof for Linux and Windows clean-checkout behavior. |
0 commit comments