Skip to content

SETUP.md

Latest commit

 

History

History
308 lines (224 loc) · 8.59 KB

SETUP.md

File metadata and controls

308 lines (224 loc) · 8.59 KB
Raw

DIRS — Setup & Usage Guide

This guide covers everything you need to install, configure, and run DIRS from scratch.

Table of Contents

  1. Prerequisites
  2. Installation
  3. Setting Up Ollama & Language Models
  4. Starting the Application
  5. Using the Admin Panel
  6. Using the User Panel
  7. Project Structure
  8. Configuration
  9. Experiment Logs
  10. Troubleshooting

1. Prerequisites

Ensure the following are installed before proceeding.

Requirement Version Check Command
Python 3.10 or higher python --version
pip Latest pip --version
Git Any git --version
Ollama Latest ollama --version

Installing Ollama

Ollama is required to run language models locally.

Windows / macOS: Download and run the installer from https://ollama.com/download

Linux:

curl -fsSL https://ollama.com/install.sh | sh

2. Installation

Step 1 — Clone the Repository

git clone https://github.com/your-username/DIRS.git
cd DIRS

Step 2 — Create a Virtual Environment

Windows:

python -m venv venv
venv\Scripts\activate

macOS / Linux:

python -m venv venv
source venv/bin/activate

You should see (venv) in your terminal prompt once activated.

Step 3 — Install Dependencies

pip install -r requirements.txt

This installs all required packages including Streamlit, FAISS, ChromaDB, Sentence Transformers, Ollama client, and others.

Note: The first run will also download the embedding models from Hugging Face. This requires an internet connection and may take a few minutes depending on your connection speed.

3. Setting Up Ollama & Language Models

DIRS uses Ollama to serve LLMs locally. You must pull the models before using the application.

Start the Ollama Server

ollama serve

Keep this terminal open. Ollama must be running whenever you use DIRS.

Pull Language Models

Open a new terminal and run the following commands to download the supported models:

# LLaMA 3 (recommended, ~4.7 GB)
ollama pull llama3:latest

# Qwen 2.5 7B (~4.4 GB)
ollama pull qwen2.5:7b

# Gemma 7B (~5.0 GB)
ollama pull gemma:7b

You can pull only the models you plan to use. At minimum, pull llama3:latest.

Verify Models are Available

ollama list

You should see the pulled models listed in the output.

4. Starting the Application

Make sure:

  • Your virtual environment is activated
  • The Ollama server is running (ollama serve)

Then start the Streamlit app:

streamlit run app.py

The application will open automatically in your browser at:

http://localhost:8501

To stop the application, press Ctrl + C in the terminal.

5. Using the Admin Panel

The Admin panel is used to upload documents and build the searchable index.

  1. In the sidebar, select Admin.
  2. Click Browse files and upload a PDF document.
  3. Select an Embedding Model:
    • BGE-small — Fast and accurate (recommended)
    • MiniLM — Lightest and fastest
    • E5-small — Strong general-purpose performance
  4. Select a Vector Database:
    • FAISS — In-memory, fast retrieval with hybrid BM25 support (recommended)
    • Chroma — Persistent, suitable for larger collections
  5. Optionally check Force Rebuild if you want to overwrite an existing index for the same document.
  6. Click Build Index.

Once built, the index is saved to the storage/ directory and is immediately available for querying.

6. Using the User Panel

The User panel is used to ask questions against indexed documents.

  1. In the sidebar, select User.
  2. Select a Document from the dropdown (lists all indexed documents).
  3. Select an LLM:
    • llama3:latest — Best overall quality
    • qwen2.5:7b — Strong reasoning, concise answers
    • gemma:7b — Lightweight alternative
  4. Type your question in the text area.
  5. Click Get Answer.

The system will display:

  • The generated answer
  • Performance metrics — embedding time, retrieval time, generation time, tokens/sec
  • Retrieved source chunks — the exact document passages used to generate the answer

7. Project Structure

DIRS/
│
├── app.py                  # Streamlit web interface
├── rag_engine.py           # Core RAG pipeline (build & query)
├── experiment_logger.py    # CSV experiment logging
├── main.py                 # CLI entry point (alternative to Streamlit)
├── config.py               # Centralized configuration (paths, models, parameters)
├── requirements.txt        # Python dependencies
│
├── rag/
│   ├── pdf_loader.py       # PDF text extraction
│   ├── chunker.py          # Text chunking
│   └── bm25_retriever.py   # BM25 keyword retrieval
│
├── models/
│   ├── embedding.py        # Sentence Transformer embedding
│   └── llm.py              # Ollama LLM interface
│
├── vectorstore/
│   └── faiss_store.py      # FAISS vector store wrapper
│
├── storage/                # Auto-created: stores all built indices
│   └── <document_name>/
│       ├── index.faiss
│       ├── chunks.json
│       ├── tokenized_chunks.json
│       └── metadata.json
│
├── results/                # Auto-created: experiment logs
│   └── llm_benchmark.csv
│
├── uploaded_docs/          # Auto-created: stores uploaded PDFs
└── assets/                 # Screenshots for README

8. Configuration

All tunable parameters are centralized in config.py at the project root. You should not need to edit any other file to change pipeline behaviour.

Parameter Default Description
CHUNK_SIZE 500 Number of characters per chunk
CHUNK_OVERLAP 50 Overlap between consecutive chunks
TOP_K 3 Number of chunks retrieved per query
VECTOR_WEIGHT 0.6 Weight given to semantic (vector) score in hybrid fusion
BM25_WEIGHT 0.4 Weight given to keyword (BM25) score in hybrid fusion
DEFAULT_EMBEDDING_MODEL BGE-small Default embedding model for CLI (main.py)
DEFAULT_LLM_MODEL llama3:latest Default LLM for CLI (main.py)
MAX_NEW_TOKENS 150 Maximum tokens the LLM generates per answer

9. Experiment Logs

Every query automatically logs performance data to results/llm_benchmark.csv.

Logged fields include:

Field Description
timestamp Date and time of the query
document Document queried
llm LLM model used
embedding_model Embedding model used
vector_db Vector database used
embedding_time Time to embed the query (seconds)
retrieval_time Time to retrieve chunks (seconds)
generation_time Time for LLM to generate answer (seconds)
total_time End-to-end pipeline time (seconds)
tokens_per_second Approximate generation speed
prompt_length_chars Total prompt size in characters
answer_length_chars Answer length in characters

This log is useful for benchmarking different model and database combinations.

10. Troubleshooting

ollama: command not found Ollama is not installed or not on your PATH. Re-install from https://ollama.com/download and restart your terminal.

Connection refused when querying LLM The Ollama server is not running. Open a terminal and run:

ollama serve

Model not found error The selected LLM has not been pulled. Run the appropriate pull command:

ollama pull llama3:latest

Index already exists error in Admin panel An index for this document already exists. Enable the Force Rebuild checkbox to overwrite it.

Embedding model download is slow Embedding models are downloaded from Hugging Face on first use. This is a one-time download per model. Subsequent runs load from the local cache.

No documents available in User panel No index has been built yet. Switch to the Admin panel and build an index first.

streamlit: command not found Your virtual environment is not activated. Run:

# Windows
venv\Scripts\activate

# macOS / Linux
source venv/bin/activate

Then re-run streamlit run app.py.

For questions or contributions, contact amansri345@gmail.com