Skip to content

test: schema-validate object literals in doc snippets #202

@aspiers

Description

@aspiers

Problem

The doc snippet test suite (tests/validate-doc-snippets.test.ts) currently guards against several classes of documentation drift:

  • Type-string ($type) literals that don't exist in any generated lexicon
  • validate() calls with arguments in the wrong order
  • Imports of symbols that don't exist in the package exports
  • Namespace/mapping accesses against symbols that no longer exist

However, it does not schema-validate the actual record literals embedded in the documentation snippets. This means a snippet like:

const receipt = {
  $type: FUNDING_RECEIPT_NSID,
  subject: { uri: "...", cid: "..." },   // ← field doesn't exist on the lexicon
  paidAt: new Date().toISOString(),      // ← field doesn't exist on the lexicon
  // ... other fields
};

…passes all current tests even though the record is invalid against org.hypercerts.funding.receipt. This exact failure mode was caught by a human reviewer on #196 (the subject/paidAt funding receipt bug fixed in 44c0343), demonstrating the gap concretely.

Proposal

Extend tests/validate-doc-snippets.test.ts with a new check that:

  1. Extracts object literals from TypeScript code blocks in README.md and .agents/skills/building-with-hypercerts-lexicons/SKILL.md.
  2. Identifies each literal's lexicon via its $type marker (either a string literal or a resolved NSID constant).
  3. Runs each literal through the runtime validate() function from generated/lexicons.
  4. Reports failures with source context (file, line, snippet excerpt) to make fixing easy.

Implementation sketch

A possible helper signature:

interface ExampleLiteral {
  obj: unknown;
  nsid: string;          // resolved from $type
  sourceFile: string;
  sourceContext: string; // ~10 lines of surrounding snippet
}

function collectExampleObjectLiterals(
  source: string,
  sourceFile: string,
  nsidConstants: Record<string, string>, // NSID name → value mapping
): ExampleLiteral[];

The new test suite then iterates these and calls validate(obj, nsid, 'main', false) from generated/lexicons.js, expecting result.success === true.

Extraction challenges

Naively regex-extracting object literals is fragile — nested braces, backtick strings, and comments can all confuse a regex. Two plausible approaches:

(a) Regex + bracket counting: Find const <name> = { or <name>: { openers, then walk forward counting {/} while skipping string/comment contents. Simple but brittle around edge cases.

(b) Lightweight TS parsing: Use a small parser (the typescript package is already a transitive dependency; @babel/parser is another option) to AST-walk each block. More robust but adds parse overhead and a direct dependency on a parser.

Given the docs snippets are mostly straightforward const x = { ... } literals, approach (a) is probably sufficient and avoids a new dependency. Approach (b) is only worth it if (a) turns out to misparse real snippets.

NSID resolution

$type in doc snippets is usually one of:

  • A string literal: $type: "org.hypercerts.funding.receipt"
  • An NSID constant: $type: FUNDING_RECEIPT_NSID
  • A dotted namespace access: $type: HYPERCERTS_NSIDS.FUNDING_RECEIPT

The test already parses the import statements (to verify symbols exist), so extending that parse to build a name→value map of imported NSID constants is straightforward.

Why this matters

  • The funding-receipt snippet was wrong for a long time without being noticed.
  • Every added/fixed docs snippet is essentially trusted code for downstream users who copy-paste it.
  • The rest of the test file already establishes the pattern of "lint the docs against the generated code" — this is the logical next step.

Scope

  • tests/validate-doc-snippets.test.ts only.
  • No changes to the lexicons or generated code.
  • No changes to production code.

Out of scope

  • Validating snippets in other markdown files (ERD.md, SCHEMAS.md, changelogs). Can be added later once the core extractor is in place.
  • Validating non-object-literal record construction (e.g. const r = makeReceipt(...)). Only direct object literals are in scope.

References

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions