Improve SEO positioning and add LLM eval landing pages

Author: parthmax2

.gitignore

@@ -25,3 +25,5 @@ Thumbs.db
 
 doc/prd.md
 doc/task.md
+SEO_GROWTH_PLAN.md
+parthmax-journey.md

README.md

@@ -5,11 +5,12 @@
   </picture>
 </p>
 
-<h3 align="center">pytest for LLMs</h3>
-<p align="center">Deterministic record/replay and semantic assertions for GenAI apps — local-first, privacy-first, zero SaaS lock-in.</p>
+<h3 align="center">CI-native LLM evals for real applications.</h3>
+<p align="center">ghostrun turns real LLM app behavior into deterministic pytest evals: record API calls once, replay them in CI, and catch semantic regressions before they ship.</p>
 
 <p align="center">
   <a href="https://pypi.org/project/ghostrun/"><img alt="PyPI" src="https://img.shields.io/pypi/v/ghostrun.svg"></a>
+  <a href="https://pypi.org/project/ghostrun/"><img alt="PyPI downloads" src="https://static.pepy.tech/badge/ghostrun"></a>
   <a href="LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-blue.svg"></a>
   <a href="pyproject.toml"><img alt="Python 3.9+" src="https://img.shields.io/badge/python-3.9%2B-blue.svg"></a>
   <a href="https://github.com/parthmax2/ghostrun/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/parthmax2/ghostrun/actions/workflows/ci.yml/badge.svg"></a>
@@ -31,7 +32,7 @@
 ---
 
 > [!TIP]
-> Every real test run of an LLM feature means a live API call — slow, costs money, and flaky, since the model never says the same thing twice. ghostrun records the call once, replays it instantly forever after, and grades the *meaning* of the output instead of matching exact text.
+> LLM evals often live outside the product: separate datasets, dashboards, YAML files, and one-off scripts. ghostrun keeps evals inside normal `pytest`, runs them against the real code path your users hit, records the live LLM call once, and replays it deterministically in CI.
 
 ### The problem
 
@@ -40,7 +41,9 @@ reply = generate_reply("Where is my refund?")
 assert reply == "I'm sorry for the delay..."   # fails tomorrow: LLM never says the same thing twice
 ```
 
-### What ghostrun does about it
+LLM apps need evals, but live model calls make test suites painful: they cost money, need API keys in CI, run slowly, and fail for reasons unrelated to your code. Generic eval platforms are powerful, but they often test a prompt, dataset, or trace outside your application. ghostrun focuses on app-native LLM regression testing: the exact Python code path your users exercise.
+
+### How ghostrun fixes it
 
 1. **Deterministic replay** — the first run records real LLM HTTP calls to a local `.ghostrun_cache/`; every run after replays them instantly from disk. Zero API cost, zero latency, zero flakiness, no key needed in CI.
 2. **Semantic assertions** — assert on *meaning*, not exact text:
@@ -52,6 +55,8 @@ assert reply == "I'm sorry for the delay..."   # fails tomorrow: LLM never says
 
 No cloud dashboard. No custom CLI to learn. No dataset to author by hand. Just `pytest`.
 
+**Search terms this project is built for:** LLM evals in CI, LLM regression testing, pytest LLM evals, deterministic LLM tests, prompt regression testing, semantic assertions for LLM apps, and testing OpenAI or Anthropic applications with pytest.
+
 ---
 
 ## Install
@@ -135,6 +140,9 @@ Start here, in order:
 
 | Guide | What's in it |
 | :--- | :--- |
+| [LLM regression testing](https://ghostrun.parthmax.in/guide/llm-regression-testing.html) | CI-native LLM evals for catching semantic and prompt regressions in real app code |
+| [Pytest LLM evals](https://ghostrun.parthmax.in/guide/pytest-llm-evals.html) | How to write LLM evals as normal pytest tests instead of dashboard-only workflows |
+| [Test OpenAI apps offline](https://ghostrun.parthmax.in/guide/test-openai-apps-offline.html) | Record/replay OpenAI and Anthropic API calls so CI does not repeat live model calls |
 | [doc/guide/recording.md](doc/guide/recording.md) | How record/replay works, judge-verdict caching, supported providers, secret redaction, parallel test runs |
 | [doc/guide/assertions.md](doc/guide/assertions.md) | Semantic assertions, judge reliability (benchmarked, not asserted), majority-vote verdicts, tool/function-call assertions |
 | [doc/guide/configuration.md](doc/guide/configuration.md) | `.ghostrun.yaml`, environment variables, pytest flags, `ghostrun doctor`, `ghostrun init` |
@@ -156,7 +164,7 @@ Start here, in order:
 </details>
 
 A hosted, searchable version of this documentation is planned at
-[parthmax2.github.io/ghostrun](https://parthmax2.github.io/ghostrun/) (config
+[ghostrun.parthmax.in](https://ghostrun.parthmax.in/) (config
 in `mkdocs.yml`, builds via `.github/workflows/docs.yml`).
 
 ## Roadmap

llms.txt

@@ -1,48 +1,39 @@
 # ghostrun
 
-> pytest for LLMs: deterministic HTTP record/replay, local LLM-as-judge semantic assertions, and prompt-version regression diffing for testing GenAI applications in Python.
-
-ghostrun solves two problems that make LLM applications hard to test with
-ordinary `assert`: non-deterministic output (the same prompt returns different
-text every run) and cost/latency (every test run would otherwise hit a paid,
-slow API). It intercepts LLM provider HTTP calls at the transport layer
-(httpx) — not by monkey-patching the OpenAI/Anthropic SDKs — so the first test
-run records a real response and every run after replays it in milliseconds
-with no API key, no network call, and no cost. Semantic assertions
-(`ghostrun.expect(reply).contains_intent("apology")`) are graded by a judge —
-by default a local Ollama model, so prompts and responses never leave the
-machine — and judge verdicts are themselves cached with an optional
-majority-of-k voting mode, benchmarked against a real judge rather than
-assumed to work (see the benchmark write-up below). `ghostrun diff` compares
-two recorded runs to flag which assertions regressed, which fixed, and which
-LLM outputs drifted, even when they still pass.
-
-It is a Python package (`pip install ghostrun`), a pytest plugin (auto-registers
-via the `pytest11` entry point, no conftest wiring needed), and a CLI
-(`ghostrun list|show|diff|doctor|init`). It is not a hosted SaaS, not an
-observability platform, and not a cross-model benchmarking tool — see the
-comparison document below for how it relates to DeepEval, Promptfoo, Ragas,
-vcr-langchain, and other tools in this space.
+> CI-native LLM evals for real applications: record LLM API calls once, replay them deterministically in pytest, and catch semantic regressions before they ship.
+
+ghostrun is a Python package, pytest plugin, and CLI for app-native LLM regression testing. It is built for developers who want LLM evals inside the software test suite instead of a separate dataset, dashboard, YAML workflow, or one-off script.
+
+ghostrun tests the real Python code path your users hit. The first run records live OpenAI, Anthropic, Gemini, Bedrock, Ollama, or other LLM provider HTTP calls at the transport layer. Later runs replay those calls deterministically in pytest and CI, so evals are fast, repeatable, and cheap enough to run on every pull request.
+
+Important search phrases: LLM evals in CI, LLM regression testing, pytest LLM evals, deterministic LLM tests, prompt regression testing, semantic assertions for LLM apps, testing OpenAI apps with pytest, testing Anthropic apps with pytest, app-native LLM evals, GenAI testing, AI testing, prompt testing, and record/replay LLM API calls.
+
+Semantic assertions such as `ghostrun.expect(reply).contains_intent("apology")` grade meaning rather than exact text. Judge verdicts are cached, and optional majority-of-k voting is available for higher-signal semantic checks. `ghostrun diff` compares recorded runs to flag assertion regressions, fixes, stable checks, removed checks, and LLM output drift.
+
+ghostrun is not a hosted SaaS, not an observability platform, and not a cross-model leaderboard. Compared with DeepEval, Promptfoo, Ragas, Langfuse, Braintrust, and LangSmith, ghostrun's focused wedge is app-native regression evals in normal pytest: real code, real LLM calls, deterministic replay, semantic assertions, and CI-friendly failure output.
 
 ## Core documentation
 
-- [README](README.md): what it is, install, quickstart, documentation index
-- [Recording and replay](doc/guide/recording.md): HTTP-transport interception, judge-verdict caching, supported providers, secret redaction, parallel test runs
-- [Semantic assertions](doc/guide/assertions.md): `expect()` API, measured judge reliability (~90% on hand-labeled cases), majority-vote verdicts, tool-call assertions
-- [Prompt regression tracking](doc/guide/regression-tracking.md): run snapshots, `ghostrun diff`, CI/PR integration (GitHub Actions, JUnit XML)
+- [README](README.md): install, quickstart, positioning, and documentation index
+- [LLM regression testing](site/guide/llm-regression-testing.html): CI-native LLM evals for catching semantic and prompt regressions in real app code
+- [Pytest LLM evals](site/guide/pytest-llm-evals.html): write LLM evals as normal pytest tests
+- [Test OpenAI apps offline](site/guide/test-openai-apps-offline.html): record/replay OpenAI and Anthropic API calls so CI does not repeat live model calls
+- [Recording and replay](doc/guide/recording.md): HTTP transport interception, judge-verdict caching, supported providers, secret redaction, parallel test runs
+- [Semantic assertions](doc/guide/assertions.md): `expect()` API, judge reliability, majority-vote verdicts, tool-call assertions
+- [Prompt regression tracking](doc/guide/regression-tracking.md): run snapshots, `ghostrun diff`, CI/PR integration, GitHub Actions, JUnit XML
 - [Configuration](doc/guide/configuration.md): `.ghostrun.yaml`, environment variables, `ghostrun doctor`, `ghostrun init`
 - [API reference](doc/guide/api-reference.md): every public function, class, and config field
 
 ## Research and rationale
 
-- [Why not just ask an LLM to write this?](doc/guide/why-not-diy.md): concrete bugs found in this project's own development (thread-safety races, secret-redaction false positives, non-monotonic majority voting) as the case for a maintained package over a generated script
-- [Judge-voting benchmark](doc/judge-voting-benchmark.md): full methodology and results for whether majority-vote judge caching actually improves reliability (answer: it's a better detector of unreliable assertions than a fix for them)
-- [Comparison with other tools](doc/comparison.md): researched, sourced comparison against DeepEval, Promptfoo, Ragas, vcr-langchain, Langfuse, Braintrust, LangSmith, Giskard, and others
-
-## Project status
-
-- [Changelog](CHANGELOG.md): release notes
+- [Why not just ask an LLM to write this?](doc/guide/why-not-diy.md): concrete bugs found in this project's own development
+- [Judge-voting benchmark](doc/judge-voting-benchmark.md): methodology and results for majority-vote judge caching
+- [Comparison with other tools](doc/comparison.md): sourced comparison against DeepEval, Promptfoo, Ragas, vcr-langchain, Langfuse, Braintrust, LangSmith, Giskard, and others
 
-## Optional
+## Project links
 
-- [Bundled example](examples/README.md): a runnable customer-support test with a pre-recorded cache, plus a live-API smoke test
+- Website: https://ghostrun.parthmax.in
+- Source: https://github.com/parthmax2/ghostrun
+- PyPI: https://pypi.org/project/ghostrun/
+- Changelog: CHANGELOG.md
+- License: MIT

mkdocs.yml

@@ -1,9 +1,9 @@
 site_name: ghostrun
 site_description: >-
-  pytest for LLMs: deterministic HTTP record/replay, local LLM-as-judge
-  semantic assertions, and prompt regression diffing for testing GenAI
-  applications in Python.
-site_url: https://parthmax2.github.io/ghostrun/
+  CI-native LLM evals for real applications: record LLM API calls once,
+  replay them deterministically in pytest, and catch semantic regressions
+  before they ship.
+site_url: https://ghostrun.parthmax.in/
 repo_url: https://github.com/parthmax2/ghostrun
 repo_name: parthmax2/ghostrun
 edit_uri: edit/main/doc/

pyproject.toml

@@ -5,19 +5,21 @@ build-backend = "hatchling.build"
 [project]
 name = "ghostrun"
 version = "0.1.1"
-description = "pytest for LLMs: deterministic HTTP record/replay, local LLM-as-judge semantic assertions, and prompt regression diffing for testing GenAI applications."
+description = "CI-native LLM evals for real apps: record LLM API calls once, replay them deterministically in pytest, and catch semantic regressions before they ship."
 readme = "README.md"
 requires-python = ">=3.9"
 license = { text = "MIT" }
 authors = [{ name = "parthmax2" }]
 keywords = [
-    "llm", "llm-testing", "llm-eval", "llm-evaluation", "llm-as-judge",
-    "pytest", "pytest-plugin", "testing", "unit-testing",
-    "genai", "generative-ai", "ai-testing", "agent-testing",
-    "openai", "anthropic", "ollama",
+    "llm", "llm-evals", "llm-eval", "llm-evaluation", "llm-as-judge",
+    "llm-testing", "llm-regression-testing", "pytest", "pytest-plugin",
+    "testing", "unit-testing", "ci", "ci-cd",
+    "genai", "generative-ai", "ai-testing", "agent-testing", "app-testing",
+    "openai", "anthropic", "ollama", "chatbot-testing",
     "record-replay", "vcr", "http-mocking", "http-cache",
-    "prompt-testing", "prompt-regression", "regression-testing",
-    "semantic-assertions", "deterministic-testing", "ci-cd",
+    "prompt-testing", "prompt-regression", "prompt-regression-testing",
+    "regression-testing", "semantic-assertions", "deterministic-testing",
+    "deterministic-llm-tests", "offline-testing",
 ]
 classifiers = [
     "Development Status :: 3 - Alpha",
@@ -48,10 +50,13 @@ dev = ["pytest>=7.0"]
 ghostrun = "ghostrun.cli:main"
 
 [project.urls]
-Homepage = "https://github.com/parthmax2/ghostrun"
+Homepage = "https://ghostrun.parthmax.in"
+Source = "https://github.com/parthmax2/ghostrun"
 Issues = "https://github.com/parthmax2/ghostrun/issues"
-Documentation = "https://github.com/parthmax2/ghostrun#documentation"
+Documentation = "https://ghostrun.parthmax.in"
 Changelog = "https://github.com/parthmax2/ghostrun/blob/main/CHANGELOG.md"
+Repository = "https://github.com/parthmax2/ghostrun"
+Author = "https://github.com/parthmax2"
 
 # Register the pytest plugin so `import ghostrun` isn't required to activate it.
 [project.entry-points.pytest11]

site/assets/search-index.js

@@ -3,8 +3,8 @@ window.GHOSTRUN_SEARCH_INDEX = [
     title: "ghostrun",
     section: "Home",
     url: "index.html",
-    excerpt: "pytest for LLMs — deterministic record/replay and semantic assertions, local-first and privacy-first.",
-    keywords: "home overview pitch"
+    excerpt: "CI-native LLM evals for real applications: record API calls once, replay in CI, and catch semantic regressions.",
+    keywords: "home overview pitch llm evals ci regression testing pytest"
   },
   {
     title: "Getting started",
@@ -13,6 +13,27 @@ window.GHOSTRUN_SEARCH_INDEX = [
     excerpt: "Install ghostrun, pull an Ollama model, and scaffold a working first test with ghostrun init.",
     keywords: "install pip ollama init quickstart pytest"
   },
+  {
+    title: "LLM regression testing",
+    section: "Guide",
+    url: "guide/llm-regression-testing.html",
+    excerpt: "Catch LLM behavior regressions in CI by recording real app calls, replaying them deterministically, and asserting on meaning.",
+    keywords: "llm regression testing prompt regression testing deterministic llm tests ci semantic regression"
+  },
+  {
+    title: "Pytest LLM evals",
+    section: "Guide",
+    url: "guide/pytest-llm-evals.html",
+    excerpt: "Write LLM evals as normal pytest tests around the real application code path your users hit.",
+    keywords: "pytest llm evals llm testing pytest llm evals python semantic assertions"
+  },
+  {
+    title: "Test OpenAI apps offline",
+    section: "Guide",
+    url: "guide/test-openai-apps-offline.html",
+    excerpt: "Record real OpenAI and Anthropic API calls once, then replay them offline and deterministically in pytest.",
+    keywords: "test openai app pytest mock openai calls pytest record replay openai api calls offline llm tests"
+  },
   {
     title: "Recording and replay",
     section: "Guide",