Skip to content

Commit 0f55387

Browse files
Yara-abYara-ab
authored
Merge pull request #3124 from oracle-devrel/reranker-rag
reranker-rag-demo
2 parents baf6f91 + 89a836d commit 0f55387

9 files changed

Lines changed: 1983 additions & 0 deletions

File tree

‎ai/generative-ai-service/reranker-rag-demo/README.md‎

Lines changed: 318 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,318 @@
1+
# OCI Reranker RAG Demo - Vision Corp Leave Policy
2+
3+
A lightweight demo that shows the practical difference between using only a vector store and adding OCI Reranker on top of retrieval.
4+
5+
The app uses a small leave-policy knowledge base derived from `Vision Corp Leave policy.pdf`. A user asks a policy question, the backend retrieves candidate passages with OCI embeddings and FAISS, and the UI lets you switch between:
6+
7+
- **Vector store only:** answer from the highest cosine-similarity document.
8+
- **OCI Reranker on:** answer from the same retrieved candidates after OCI Reranker reorders them by query-document relevance.
9+
10+
This makes the reranker effect visible: the retrieved candidate list can contain the right document, but plain vector similarity may not rank it first. The reranker can promote the better passage before the answer is shown.
11+
12+
## Screenshots
13+
14+
### Vector Store Only
15+
16+
![Vector search only](files/screenshots/vector-search-only.png)
17+
18+
### OCI Reranker Enabled
19+
20+
![OCI Reranker enabled](files/screenshots/oci-reranker-enabled.png)
21+
22+
### Side-by-Side Ranking Comparison
23+
24+
![Ranking comparison](files/screenshots/ranking-comparison.png)
25+
26+
## What This Demo Does
27+
28+
- Loads a PDF-derived knowledge base from `files/knowledge_base.json`.
29+
- Embeds every knowledge-base chunk with OCI Generative AI embeddings.
30+
- Stores normalized vectors in an in-memory FAISS `IndexFlatIP` index.
31+
- Embeds the user query with the same OCI embedding model.
32+
- Retrieves the top-k candidates by cosine similarity.
33+
- Optionally sends those same candidates to OCI Generative AI `RerankText`.
34+
- Shows the answer, citations, vector ranking, reranked ranking, and score differences in the browser.
35+
36+
The demo intentionally uses compound policy questions, such as asking about public holidays and annual leave in the same query, so the value of reranking is easier to see.
37+
38+
## Demo Questions
39+
40+
The UI includes three preset questions:
41+
42+
1. **Annual days:** What public holidays are listed, and how many annual leave days do Netherlands and Poland employees get?
43+
2. **Sick certificate:** If leave is without manager approval it may be unpaid, but what if the absence is illness over two days?
44+
3. **Compassionate:** Who counts as a direct relative, and how much compassionate leave is given for death of a spouse or direct relative?
45+
46+
You can also type your own question in the text box.
47+
48+
## Architecture
49+
50+
```mermaid
51+
flowchart LR
52+
UI[Browser UI] --> API[Python HTTP backend]
53+
KB[knowledge_base.json] --> API
54+
API --> EmbedDocs[OCI EmbedText documents]
55+
EmbedDocs --> FAISS[FAISS IndexFlatIP vector store]
56+
UI --> Query[User question]
57+
Query --> API
58+
API --> EmbedQuery[OCI EmbedText query]
59+
EmbedQuery --> FAISS
60+
FAISS --> Candidates[Top-k vector candidates]
61+
Candidates --> VectorAnswer[Vector-only answer]
62+
Candidates --> Rerank[OCI RerankText]
63+
Rerank --> RerankedAnswer[Reranked answer]
64+
VectorAnswer --> UI
65+
RerankedAnswer --> UI
66+
```
67+
68+
## How Retrieval Works
69+
70+
### 1. Knowledge Base
71+
72+
The runtime knowledge base is persisted in:
73+
74+
```text
75+
files/knowledge_base.json
76+
```
77+
78+
It currently contains 12 chunks from the Vision Corp leave policy, including annual leave, sick leave, maternity and paternity leave, compassionate leave, public holidays, manager approval, and direct-relative definitions.
79+
80+
The app does not read the PDF at runtime. The PDF content has already been converted into structured JSON chunks with this shape:
81+
82+
```json
83+
{
84+
"id": "annual-leave",
85+
"title": "Annual Leave",
86+
"source": "Vision Corp Leave policy.pdf - Section 2",
87+
"text": "...policy passage...",
88+
"tags": ["annual leave", "Netherlands", "Poland"],
89+
"answer": "...grounded answer used by the demo..."
90+
}
91+
```
92+
93+
### 2. Vector Store
94+
95+
On the first query, `server.py`:
96+
97+
1. Reads `knowledge_base.json`.
98+
2. Sends each document passage to OCI `EmbedText` with `input_type=SEARCH_DOCUMENT`.
99+
3. Normalizes the embedding matrix with `faiss.normalize_L2`.
100+
4. Builds an in-memory FAISS `IndexFlatIP` index.
101+
102+
Because the vectors are normalized, FAISS inner product is used as cosine similarity.
103+
104+
The vector index is not written to disk. It is rebuilt in memory when the server starts or when the knowledge-base fingerprint changes.
105+
106+
### 3. Query Search
107+
108+
For each question, the backend:
109+
110+
1. Sends the query to OCI `EmbedText` with `input_type=SEARCH_QUERY`.
111+
2. Normalizes the query vector.
112+
3. Searches FAISS for the top-k most similar chunks.
113+
4. Returns the vector results and cosine scores to the UI.
114+
115+
### 4. OCI Reranking
116+
117+
When the switch is on, the backend sends the retrieved candidates to OCI Generative AI `RerankText`:
118+
119+
```text
120+
input: the user question
121+
documents: top-k passages from vector search
122+
top_n: number of candidates to return
123+
model: cohere.rerank-v4.0-fast
124+
region: me-riyadh-1
125+
```
126+
127+
OCI returns a `relevance_score` for each candidate. The app then reorders the same vector-retrieved candidates using that reranker score.
128+
129+
### 5. Answer Display
130+
131+
This demo keeps generation simple and transparent: it displays the curated `answer` field from the top-ranked document.
132+
133+
That means:
134+
135+
- In vector-only mode, the answer comes from the top vector result.
136+
- In reranker mode, the answer comes from the top reranked result.
137+
138+
No LLM chat generation is currently used after retrieval. This keeps the demo focused on proving the retrieval and reranking difference. You can extend it later by sending the top reranked passages into an LLM prompt.
139+
140+
## Scores Explained
141+
142+
The UI shows two different score types:
143+
144+
- **Cosine score:** Produced by FAISS from normalized OCI embeddings. This is used in vector-only retrieval.
145+
- **Relevance score:** Returned by OCI Reranker. This is the reranker model's relevance score for a query-document pair.
146+
147+
These scores are real, but they are not the same scale. Compare cosine scores with cosine scores, and reranker relevance scores with reranker relevance scores. The important signal is how the candidate order changes.
148+
149+
## Tech Stack
150+
151+
- Frontend: HTML, CSS, vanilla JavaScript
152+
- Backend: Python `http.server` with custom API handlers
153+
- Embeddings: OCI Generative AI `cohere.embed-v4.0`
154+
- Reranker: OCI Generative AI `cohere.rerank-v4.0-fast`
155+
- Vector search: FAISS `IndexFlatIP`
156+
- Knowledge base: local JSON file generated from the policy PDF
157+
158+
## Project Structure
159+
160+
```text
161+
Reranker Demo/
162+
|-- README.md
163+
`-- files/
164+
|-- app.js
165+
|-- index.html
166+
|-- knowledge_base.json
167+
|-- screenshots/
168+
| |-- vector-search-only.png
169+
| |-- oci-reranker-enabled.png
170+
| `-- ranking-comparison.png
171+
|-- server.py
172+
`-- styles.css
173+
```
174+
175+
## Setup
176+
177+
### 1. Install Python Dependencies
178+
179+
From the app folder:
180+
181+
```powershell
182+
cd files
183+
python -m venv .venv
184+
.\.venv\Scripts\Activate.ps1
185+
pip install oci numpy faiss-cpu
186+
```
187+
188+
If you already have these packages installed globally, you can run the server without creating a virtual environment.
189+
190+
### 2. Configure OCI Credentials
191+
192+
The backend reads your OCI config from `~/.oci/config` by default and uses the `DEFAULT` profile unless overridden.
193+
194+
Required OCI access:
195+
196+
- A valid OCI config profile with API key authentication.
197+
- A compartment with permission to call OCI Generative AI inference.
198+
- Access to OCI Generative AI in the Riyadh region, `me-riyadh-1`.
199+
200+
Recommended PowerShell environment variables:
201+
202+
```powershell
203+
$env:OCI_CONFIG_PROFILE="DEFAULT"
204+
$env:OCI_REGION="me-riyadh-1"
205+
$env:OCI_COMPARTMENT_ID="<your-compartment-ocid>"
206+
$env:OCI_EMBED_MODEL_ID="cohere.embed-v4.0"
207+
$env:OCI_RERANK_MODEL_ID="cohere.rerank-v4.0-fast"
208+
```
209+
210+
Optional overrides:
211+
212+
```powershell
213+
$env:OCI_CONFIG_FILE="<path-to-your-oci-config>"
214+
$env:OCI_GENAI_ENDPOINT="https://inference.generativeai.me-riyadh-1.oci.oraclecloud.com"
215+
$env:OCI_EMBED_ENDPOINT_ID="<dedicated-embedding-endpoint-ocid>"
216+
$env:OCI_RERANK_ENDPOINT_ID="<dedicated-rerank-endpoint-ocid>"
217+
$env:OCI_EMBED_BATCH_SIZE="96"
218+
$env:HOST="127.0.0.1"
219+
$env:PORT="4173"
220+
```
221+
222+
## Run Locally
223+
224+
```powershell
225+
cd files
226+
python server.py
227+
```
228+
229+
Then open:
230+
231+
```text
232+
http://127.0.0.1:4173/
233+
```
234+
235+
Health check:
236+
237+
```powershell
238+
Invoke-RestMethod -Uri http://127.0.0.1:4173/api/status
239+
```
240+
241+
## API Endpoints
242+
243+
### `GET /api/status`
244+
245+
Returns OCI configuration status, selected region, embedding model, reranker model, and vector-store engine.
246+
247+
### `POST /api/search`
248+
249+
Runs vector retrieval, and optionally reranking.
250+
251+
Example request:
252+
253+
```json
254+
{
255+
"query": "What public holidays are listed, and how many annual leave days do Netherlands and Poland employees get?",
256+
"useReranker": true,
257+
"topK": 12
258+
}
259+
```
260+
261+
Example response fields:
262+
263+
```json
264+
{
265+
"answer": "...",
266+
"answerMode": "reranker",
267+
"vectorResults": [],
268+
"rerankedResults": [],
269+
"vector": {},
270+
"reranker": {},
271+
"timingsMs": {}
272+
}
273+
```
274+
275+
## Updating the Knowledge Base
276+
277+
To change the demo content, edit:
278+
279+
```text
280+
files/knowledge_base.json
281+
```
282+
283+
Each chunk should have a clear `title`, `source`, `text`, `tags`, and `answer`. The `text` field is what gets embedded and reranked. The `answer` field is what the demo displays when that chunk wins.
284+
285+
After editing the JSON, refresh the browser or run another query. The backend fingerprints the knowledge base and rebuilds the in-memory FAISS index when the content changes.
286+
287+
## Troubleshooting
288+
289+
### `ModuleNotFoundError: No module named 'faiss'`
290+
291+
Install FAISS for Python:
292+
293+
```powershell
294+
pip install faiss-cpu
295+
```
296+
297+
### OCI returns `404 Authorization failed or requested resource not found`
298+
299+
The code reached OCI, but the configured profile, compartment, model, or endpoint is not authorized. Check:
300+
301+
- `OCI_COMPARTMENT_ID`
302+
- IAM policies for Generative AI inference
303+
- Region availability, especially `me-riyadh-1`
304+
- Model IDs or dedicated endpoint OCIDs
305+
306+
### Port `4173` is already in use
307+
308+
Use another port:
309+
310+
```powershell
311+
$env:PORT="4174"
312+
python server.py
313+
```
314+
315+
### Reranker and vector answers look the same
316+
317+
That can happen when vector search already ranks the best passage first. Use the compound preset questions or add overlapping KB chunks to make the reranking effect easier to demonstrate.
318+

0 commit comments

Comments
 (0)