Finding
The smart-contracts skill lacks error-keyed troubleshooting for common CLI and build failures.
Neither the stellar-dev/smart-contracts skill's troubleshooting table
(development.md §Troubleshooting) nor the docs corpus carries
error-message-keyed guidance for common CLI/build failures. Probed this
round: "No such file or directory" on a missing .wasm output path
(build succeeded in the wrong workspace/package, or silently produced no
wasm), "alias already exists", link.exe/MSVC setup on Windows, "Unable to
fund account". stellarDocs.search_docs on the exact strings returns
irrelevant hits (XDR/Lab pages) and the skill table has no
missing-wasm/wrong-path row. Agents answering troubleshooting questions
must fall back on general knowledge, and graders can't distinguish a
good general answer from a corpus-grounded one. Distinct from sk-001
(stale build target — which the agent in this round correctly avoided by
preferring docs).
Evidence
Live probes 2026-07-03: search_docs({query:'"No such file or directory" wasm'}) → XDR/Lab pages only; search_sdk_cli_tools_docs
build/output/workspace queries → cookbook lifecycle pages only; local
read of development.md troubleshooting table → no matching row. The QA
case was graded partial for missing exactly the diagnostic branch
(build-succeeded-but-wrong-path) that no corpus source prompts.
Additional recorded evidence:
- eval/qa/results/2026-07-03T16-06-45-variantA.json (q-ti-cli-rust-windows-troubleshooting)
- live zero-hit probes for exact error strings across search_docs + skill sections (2026-07-03 evening)
- upstream source rechecked 2026-07-09 at stellar/stellar-dev-skill skills commit c2f3c07: smart-contracts development.md contains no "No such file or directory", link.exe, alias-already-exists, or unable-to-fund troubleshooting row
Recommendation
In the upstream skill source, extend the development.md troubleshooting
table with rows keyed by the verbatim error text a builder will paste
("no such file or directory: target/wasm32v1-none/release/*.wasm",
"alias already exists", "linker link.exe not found", "Unable to fund
account"), each with the two-branch diagnosis (did the build produce a
wasm at all? is the path/workspace the one the command reads?). Verbatim
error strings are what agents and humans search by; today those searches
zero-hit.
Source Record
This was found by the downstream Raven eval/improvements loop and recorded as sk-006 (skills, discovered 2026-07-03). Public record: https://github.com/kalepail/stellar-raven/blob/main/improvements/skills/sk-006-error-keyed-troubleshooting-gap.md
Finding
The smart-contracts skill lacks error-keyed troubleshooting for common CLI and build failures.
Neither the
stellar-dev/smart-contractsskill's troubleshooting table(
development.md§Troubleshooting) nor the docs corpus carrieserror-message-keyed guidance for common CLI/build failures. Probed this
round: "No such file or directory" on a missing
.wasmoutput path(build succeeded in the wrong workspace/package, or silently produced no
wasm), "alias already exists", link.exe/MSVC setup on Windows, "Unable to
fund account".
stellarDocs.search_docson the exact strings returnsirrelevant hits (XDR/Lab pages) and the skill table has no
missing-wasm/wrong-path row. Agents answering troubleshooting questions
must fall back on general knowledge, and graders can't distinguish a
good general answer from a corpus-grounded one. Distinct from sk-001
(stale build target — which the agent in this round correctly avoided by
preferring docs).
Evidence
Live probes 2026-07-03:
search_docs({query:'"No such file or directory" wasm'})→ XDR/Lab pages only;search_sdk_cli_tools_docsbuild/output/workspace queries → cookbook lifecycle pages only; local
read of
development.mdtroubleshooting table → no matching row. The QAcase was graded partial for missing exactly the diagnostic branch
(build-succeeded-but-wrong-path) that no corpus source prompts.
Additional recorded evidence:
Recommendation
In the upstream skill source, extend the
development.mdtroubleshootingtable with rows keyed by the verbatim error text a builder will paste
("no such file or directory: target/wasm32v1-none/release/*.wasm",
"alias already exists", "linker
link.exenot found", "Unable to fundaccount"), each with the two-branch diagnosis (did the build produce a
wasm at all? is the path/workspace the one the command reads?). Verbatim
error strings are what agents and humans search by; today those searches
zero-hit.
Source Record
This was found by the downstream Raven eval/improvements loop and recorded as sk-006 (skills, discovered 2026-07-03). Public record: https://github.com/kalepail/stellar-raven/blob/main/improvements/skills/sk-006-error-keyed-troubleshooting-gap.md