Skip to content

Add schema validation when loading .thinktank result files to prevent cryptic crashes #69

@that-github-user

Description

@that-github-user

Problem

apply.ts and list.ts load .thinktank/latest.json with no schema validation:

const result = JSON.parse(fs.readFileSync(..., "utf-8")) as EnsembleResult;

If the file is truncated (e.g. disk full mid-write), written by an older version, or manually edited, JSON.parse throws a generic syntax error and the user sees:

Error: No results found. Run thinktank first.

...with no indication of what actually went wrong.

Solution

Add lightweight schema validation using the existing TypeScript types. A minimal runtime check (not necessarily Zod) would verify:

  • result.agents is a non-empty array
  • Each agent has id, status, diff
  • result.recommendation is a valid agent id
function assertValidResult(data: unknown): asserts data is EnsembleResult {
  if (!data || typeof data !== "object") throw new Error("Invalid result file: not an object");
  const r = data as Record<string, unknown>;
  if (!Array.isArray(r.agents) || r.agents.length === 0) {
    throw new Error("Invalid result file: missing or empty agents array");
  }
  // ... etc
}

Acceptance criteria

  • Loading a truncated or empty JSON file shows an actionable error message (e.g. "Result file appears corrupted. Re-run thinktank to regenerate.")
  • Loading a result from a different schema version shows a version mismatch warning
  • apply, list, compare, and stats all use the same validation path
  • Unit tests cover: truncated JSON, empty file, missing agents field, missing recommendation

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions