|
93 | 93 | }, |
94 | 94 | "outputs": [], |
95 | 95 | "source": [ |
96 | | - "!pip install openai pandas matplotlib requests datasets trl peft nltk rank_bm25 ipywidgets" |
| 96 | + "!uv pip install openai pandas matplotlib requests datasets trl peft nltk rank_bm25 ipywidgets" |
97 | 97 | ] |
98 | 98 | }, |
99 | 99 | { |
|
271 | 271 | "\n", |
272 | 272 | "By combining information retrieval with language generation, RAG systems can reduce hallucinations, incorporate up-to-date knowledge, and provide responses grounded in real documents.\n", |
273 | 273 | "\n", |
| 274 | + "**Key steps in RAG:**\n", |
| 275 | + "- Create/connect knowledge base\n", |
| 276 | + "- Retrieving relevant documents\n", |
| 277 | + "- Using a language model to generate insights\n", |
| 278 | + "- Post-processing the results\n", |
| 279 | + "\n", |
274 | 280 | "<figure style=\"text-align: center;\">\n", |
275 | 281 | " <img src=\"https://raw.githubusercontent.com/Future-House/tutorial-series/main/figures/RAG.png\" \n", |
276 | 282 | " alt=\"rag_framework\" \n", |
277 | 283 | " style=\"max-width: 100%;\">\n", |
278 | 284 | " <figcaption><em>Figure 2.2.1: RAG Framework</em></figcaption>\n", |
279 | 285 | "</figure>\n", |
280 | 286 | "\n", |
281 | | - "\n", |
282 | | - "**Key steps:**\n", |
283 | | - "- Create/connect knowledge base\n", |
284 | | - "- Retrieving relevant documents\n", |
285 | | - "- Using a language model to generate insights\n", |
286 | | - "- Post-processing the results\n", |
287 | | - "\n", |
288 | | - "\n", |
289 | 287 | "### Step 1: Knowledge Base Construction\n", |
290 | 288 | "\n", |
291 | 289 | "First, we need access to a knowledge base. This could be a locally existing folder of PDF files or an online database. \n", |
|
524 | 522 | " for i, abs_text in enumerate(relevant_abstracts, 1):\n", |
525 | 523 | " print(f\"\\n--- Abstract {i} ---\\n{abs_text[:500]}...\") # print first 500 chars\n", |
526 | 524 | "\n", |
527 | | - " print(\"\\nGenerating answer using OpenAI GPT-4...\\n\")\n", |
| 525 | + " print(f\"\\nGenerating answer using OpenAI {llm_model}...\\n\")\n", |
528 | 526 | " answer = generate_answer_with_openai(relevant_abstracts, query, llm_model)\n", |
529 | 527 | " print(\"=== Generated Answer ===\")\n", |
530 | 528 | " print(answer)\n", |
|
0 commit comments