|
5 | 5 | Upload documents, ask questions, get answers with citations you can click back to the source — and |
6 | 6 | a measured comparison of which retrieval strategy actually works, and what it costs. |
7 | 7 |
|
8 | | -> **Status: the code is complete and verified end-to-end; the numbers are not.** Every benchmark |
9 | | -> table in this README is empty on purpose — filling them in with invented figures would defeat the |
10 | | -> entire point of the project. [Reproduce them](#reproducing-the-benchmark): about twenty minutes |
11 | | -> and a few dollars of API credit. |
| 8 | +> **Status: deployed and working; the benchmark has not been run.** Every results table below is |
| 9 | +> empty on purpose — filling them in with invented figures would defeat the entire point of the |
| 10 | +> project. [Reproduce them](#reproducing-the-benchmark): about twenty minutes and a few dollars of |
| 11 | +> API credit. |
12 | 12 | > |
13 | | -> Verified against Postgres 16 + pgvector 0.8.5: migrations apply, a PDF ingests, hybrid retrieval |
14 | | -> returns ranked passages, the answer carries citations that resolve and verify at 100% coverage, |
15 | | -> the SSE stream emits every event, and an unanswerable question is refused. `pytest`: 90 passing. |
| 13 | +> Verified in production (Vercel `sin1` + Neon Postgres 18 / pgvector 0.8.1): a PDF ingests, hybrid |
| 14 | +> retrieval returns ranked passages, the answer carries citations that resolve and verify at 100% |
| 15 | +> coverage, cost and latency are reported per request, an unanswerable question is refused, and the |
| 16 | +> SSE stream delivers tokens progressively rather than in one flush. `pytest`: 108 passing. |
| 17 | +> |
| 18 | +> Not yet exercised: the eval harness and benchmark table, the HNSW sweep, the Qdrant backend, |
| 19 | +> Langfuse tracing, and the Kubernetes manifests. All are written; none has been run. |
16 | 20 |
|
17 | | -Frontend: **[documind-web](https://github.com/codebybilal18/documind-web)** · Live demo: _(add your Vercel URL)_ |
| 21 | +**Live API: https://documind-api-theta.vercel.app** ([health](https://documind-api-theta.vercel.app/health) · [docs](https://documind-api-theta.vercel.app/docs)) · Frontend: [documind-web](https://github.com/codebybilal18/documind-web) |
18 | 22 |
|
19 | 23 | --- |
20 | 24 |
|
@@ -366,9 +370,14 @@ boot, because two concurrent instances racing the same migration is worse than a |
366 | 370 | `ACTIVE_CONFIG=serverless` is a **distinct config**, not config D, so the deployment never reports |
367 | 371 | D's numbers for a reranker it is not running. |
368 | 372 |
|
369 | | -Streaming: the Python runtime buffers responses in some configurations. The SSE stream is written |
370 | | -correctly regardless — a buffered deployment delivers every event in one flush and the client |
371 | | -parser handles it unchanged; only the progressive reveal is lost. |
| 373 | +Streaming works. Measured against the live deployment, 33 token events arrived over a 2.23 s |
| 374 | +spread rather than in a single flush, so the progressive reveal is real and not a local-only |
| 375 | +behaviour. (The client parser handles a buffered response identically, should that ever change.) |
| 376 | + |
| 377 | +Latency is the honest weak point: ~10 s to first token on a cold function, of which ~8 s is |
| 378 | +retrieval — cold start, query rewrite, embedding, hybrid search, and the LLM reranker, each a |
| 379 | +round trip to `ap-southeast-1`. Warm requests are far quicker. Cutting it means dropping the |
| 380 | +rewrite and rerank hops, which is exactly the trade the benchmark exists to quantify. |
372 | 381 |
|
373 | 382 | --- |
374 | 383 |
|
|
0 commit comments