Skip to content

way-embed: add batch-embed command for multi-text single-model-load #88

Description

@aaronsb

Context

embed-suggest.sh evaluates vocabulary candidates by re-embedding description+vocabulary+candidate for each candidate term, then scoring test prompts against the augmented embedding. This requires loading the MiniLM model once per candidate.

With 61 candidates, total runtime is ~4s (dominated by repeated model loads at ~65ms each). The actual embedding computation per text is <1ms.

Proposed: way-embed batch-embed

A new command that loads the model once and embeds multiple texts in a single session:

way-embed batch-embed --model MODEL < texts.jsonl > embeddings.jsonl

Input (one JSON object per line):

{"id": "candidate_0", "text": "description vocabulary term0"}
{"id": "candidate_1", "text": "description vocabulary term1"}

Output (same format, with embedding vectors):

{"id": "candidate_0", "embedding": [0.123, ...]}
{"id": "candidate_1", "embedding": [0.123, ...]}

This would reduce embed-suggest.sh from N model loads to 1, bringing 61-candidate runtime from ~4s to ~200ms.

Why this matters

The suggest tool is the vocabulary tuning feedback loop for ways. Faster iteration means tighter feedback between "add term" → "measure impact" → "accept/reject". At 4s it's usable; at 200ms it's interactive.

The same batch-embed command would also benefit:

  • Corpus generation (currently sequential in way-embed generate)
  • Any future tool that needs to embed multiple texts for comparison

Implementation notes

  • Model load happens once in engine_init()
  • The embedding function engine_embed() is already stateless per-call
  • Main work: add a loop that reads stdin JSONL, calls engine_embed() per line, writes to stdout
  • No changes to existing generate or match commands

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions