|
| 1 | +# RLM-REPL |
| 2 | + |
| 3 | +**Recursive Language Model with REPL Inference Strategy** |
| 4 | + |
| 5 | +A Python library that enables any language model to manage unlimited context using SQL-based retrieval with DuckDB. |
| 6 | + |
| 7 | +[](https://www.python.org/downloads/) |
| 8 | +[](https://opensource.org/licenses/MIT) |
| 9 | + |
| 10 | +## Overview |
| 11 | + |
| 12 | +RLM-REPL implements a human-like reading strategy for processing large documents: |
| 13 | + |
| 14 | +1. **Overview** - Read the beginning to understand document structure |
| 15 | +2. **Search** - Find relevant sections using keyword search |
| 16 | +3. **Deep Read** - Extract detailed information from located sections |
| 17 | +4. **Synthesize** - Combine findings into a comprehensive answer |
| 18 | + |
| 19 | +This approach allows small context window models to effectively work with documents of any size. |
| 20 | + |
| 21 | +## Features |
| 22 | + |
| 23 | +- **Two-sided architecture**: Separate Data and Inference layers |
| 24 | +- **In-memory default**: Fast DuckDB in-memory database (no setup required) |
| 25 | +- **Persistent option**: Optional persistent database for caching |
| 26 | +- **CLI tool**: Instant testing from command line |
| 27 | +- **Python API**: Full programmatic control |
| 28 | +- **Streaming events**: Real-time progress tracking |
| 29 | +- **Configurable verbosity**: Control output detail level |
| 30 | +- **OpenAI-compatible**: Works with any OpenAI-compatible API |
| 31 | + |
| 32 | +## Installation |
| 33 | + |
| 34 | +```bash |
| 35 | +pip install rlm-repl |
| 36 | +``` |
| 37 | + |
| 38 | +Or install from source: |
| 39 | + |
| 40 | +```bash |
| 41 | +git clone https://github.com/labKnowledge/rlm-repl.git |
| 42 | +cd rlm-repl |
| 43 | +pip install -e . |
| 44 | +``` |
| 45 | + |
| 46 | +## Quick Start |
| 47 | + |
| 48 | +### CLI Usage |
| 49 | + |
| 50 | +```bash |
| 51 | +# Interactive mode |
| 52 | +rlm-repl document.txt |
| 53 | + |
| 54 | +# With custom model |
| 55 | +rlm-repl document.txt --base-url http://localhost:11434/v1 --model qwen3-coder |
| 56 | + |
| 57 | +# Single question mode |
| 58 | +rlm-repl document.txt --question "What is the main topic?" |
| 59 | + |
| 60 | +# Quiet mode |
| 61 | +rlm-repl document.txt -q --question "Summarize the document" |
| 62 | +``` |
| 63 | + |
| 64 | +### Python API |
| 65 | + |
| 66 | +```python |
| 67 | +from rlm_repl import RLMREPL, RLMConfig |
| 68 | + |
| 69 | +# Configure for Ollama (local) |
| 70 | +config = RLMConfig( |
| 71 | + base_url="http://localhost:11434/v1", |
| 72 | + api_key="ollama", |
| 73 | + model="qwen3-coder", |
| 74 | +) |
| 75 | + |
| 76 | +# Create REPL and load document |
| 77 | +with RLMREPL(config) as repl: |
| 78 | + repl.load_document("large_book.txt") |
| 79 | + |
| 80 | + result = repl.ask("What are the main themes?") |
| 81 | + print(result.answer) |
| 82 | + print(f"Read {result.total_words} words in {result.elapsed_time:.1f}s") |
| 83 | +``` |
| 84 | + |
| 85 | +## Documentation |
| 86 | + |
| 87 | +Comprehensive documentation is available in the [`docs/`](docs/) directory: |
| 88 | + |
| 89 | +- **[Getting Started](docs/getting-started.md)** - Installation, setup, and first steps |
| 90 | +- **[API Reference](docs/api-reference.md)** - Complete API documentation |
| 91 | +- **[Configuration](docs/configuration.md)** - All configuration options |
| 92 | +- **[Examples](docs/examples.md)** - Detailed usage examples |
| 93 | +- **[Architecture](docs/architecture.md)** - How the system works |
| 94 | +- **[Troubleshooting](docs/troubleshooting.md)** - Common issues and solutions |
| 95 | + |
| 96 | +## Supported Models |
| 97 | + |
| 98 | +Any OpenAI-compatible API: |
| 99 | +- **Ollama** (local): llama3, qwen3, mistral, etc. |
| 100 | +- **OpenAI**: gpt-4, gpt-3.5-turbo |
| 101 | +- **vLLM**: Any hosted model |
| 102 | +- **LMStudio**: Local models |
| 103 | +- **Together AI**, **Groq**, etc. |
| 104 | + |
| 105 | +## Examples |
| 106 | + |
| 107 | +See the [`examples/`](examples/) directory for complete examples: |
| 108 | +- `basic_usage.py` - Simple document Q&A |
| 109 | +- `streaming_events.py` - Real-time progress tracking |
| 110 | +- `persistent_database.py` - Caching documents |
| 111 | +- `api_usage.py` - Building applications with RLM-REPL |
| 112 | + |
| 113 | +## How It Works |
| 114 | + |
| 115 | +1. **Document Loading**: Text is parsed into lines with metadata (headers, code blocks, list items) |
| 116 | +2. **SQL Storage**: Lines are stored in DuckDB with indexes for efficient querying |
| 117 | +3. **Reading Strategy**: LLM decides what to read using SQL queries |
| 118 | +4. **Iterative Reading**: Multiple passes gather relevant information |
| 119 | +5. **Answer Synthesis**: Final answer is generated from gathered context |
| 120 | + |
| 121 | +### Reading Modes |
| 122 | + |
| 123 | +- **overview**: Read document beginning (lines 1-100) |
| 124 | +- **search**: Find keywords with `LIKE '%term%'` |
| 125 | +- **read**: Focused reading (20-50 lines) |
| 126 | +- **deep_read**: Detailed analysis (50-100 lines) |
| 127 | + |
| 128 | +## Development |
| 129 | + |
| 130 | +```bash |
| 131 | +# Install with dev dependencies |
| 132 | +pip install -e ".[dev]" |
| 133 | + |
| 134 | +# Run tests |
| 135 | +pytest |
| 136 | + |
| 137 | +# Format code |
| 138 | +ruff format rlm_repl |
| 139 | + |
| 140 | +# Type checking |
| 141 | +mypy rlm_repl |
| 142 | +``` |
| 143 | + |
| 144 | +## License |
| 145 | + |
| 146 | +MIT License - see LICENSE file for details. |
| 147 | + |
| 148 | +## Contributing |
| 149 | + |
| 150 | +Contributions welcome! Please open an issue or submit a pull request. |
| 151 | + |
| 152 | +## Background & History |
| 153 | + |
| 154 | +RLM-REPL was created by **Remy Gakwaya** after reading the MIT paper on Recursive Language Models. The initial implementation attempted to use a REPL approach where the LLM would generate Python functions to process documents. However, this approach proved challenging, especially with smaller language models that struggled to create complex Python functions reliably. |
| 155 | + |
| 156 | +After **hundreds of iterations and experiments**, Remy developed the **RLM-REPL v8 concept** - a human-like reading strategy specifically designed to work with local, smaller language models on limited computational resources. The philosophy was simple: *if it can work reliably with poor and small models in limited computation, it would perform exceptionally well when powered by leading LLMs.* |
| 157 | + |
| 158 | +The library evolved to use **SQL-based retrieval** instead of LLM-generated Python functions, leveraging DuckDB for efficient document storage and querying. This approach: |
| 159 | +- Works reliably with models of all sizes, from small local models to leading cloud-based LLMs |
| 160 | +- Provides a structured, predictable interface (SQL) that even smaller models can handle |
| 161 | +- Enables efficient querying with database indexes |
| 162 | +- Implements the human-like reading strategy (overview → search → deep read → synthesize) developed in v8 |
| 163 | + |
| 164 | +## Acknowledgments |
| 165 | + |
| 166 | +**Author:** Remy Gakwaya |
| 167 | + |
| 168 | +**Inspiration:** Based on the MIT paper on Recursive Language Models |
| 169 | + |
| 170 | +**Innovation:** The RLM-REPL v8 concept - human-like reading strategies for LLM document processing - was developed by Remy after extensive experimentation (hundreds of iterations) to create a solution that works reliably with local, smaller language models. |
| 171 | + |
| 172 | +**Evolution:** This implementation uses SQL-based retrieval instead of LLM-generated Python functions, making it more reliable and accessible for smaller language models while maintaining the proven v8 reading strategy. |
0 commit comments