Skip to content

Commit 006688c

Browse files
abrichrclaude
andauthored
feat: add OpenAI embedding-based alignment strategy for DemoLibrary (#179)
Add OpenAIEmbeddingAlignment class that implements the AlignmentStrategy protocol using OpenAI's VLM (gpt-4o-mini) for screenshot description and text-embedding-3-small for semantic embedding. This provides a cloud-based alternative to local CLIP models for demo step alignment. Key changes: - OpenAIEmbeddingAlignment: two-step pipeline (VLM describe + embed) with cosine similarity matching against pre-computed demo embeddings - create_alignment_strategy() factory: accepts string names ("phash", "clip", "hybrid", "openai") for easy construction - DemoLibrary constructor now accepts string alignment_strategy names with automatic fallback to pHash on failure - enrich_demo() pre-computes OpenAI embeddings when strategy is "openai" - Embeddings persist to demo.json as plain lists for serialization - Pure-Python cosine similarity fallback when numpy is not installed Cost: ~$0.001 per screenshot (~$0.06 for 61 demos total). Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent c5550a0 commit 006688c

1 file changed

Lines changed: 361 additions & 13 deletions

File tree

0 commit comments

Comments
 (0)