LLMs for Science

Educational notebooks exploring production-grade agentic systems and LLMs pipelines from first principles. Learn the core patterns behind tools like Claude Code, Cursor's agent mode, and autonomous research assistants by implementing them yourself.

These notebooks teach you to build the scaffolding that transforms an LLM from a text generator into a truly useful everyday tool.

What You'll Learn

Notebook 1: Basic Agentic Harness

Build a minimal but complete harness from scratch — the foundation of any autonomous agent system.

Core concepts:

• The five components of a harness's core state (goal, trace, memory, budget, status)
• Implementing a control loop that drives an LLM through multi-step tasks
• Defining typed tools the LLM can call safely
• Validating LLM-proposed actions against schemas before execution
• Inspecting execution traces for debugging

What you'll build: A single-agent harness that solves multi-step research tasks by repeatedly composing context, asking the LLM what to do next, executing tool calls, and updating state until the goal is met.

Notebook 2: Small Language Model

Build a complete language model from first principles — counting, not neural networks — to understand what every LLM is really doing under the hood.

Core concepts:

• The language modeling task: estimating P(next word | previous words)
• Building an n-gram (4-gram) model over the WikiText-103 corpus (~103M words)
• Why text is sparse: word-frequency distributions and Zipf's law
• Counting three-word contexts and the single-continuation problem
• Autoregressive generation — feeding the output back in to predict the next token
• Temperature and sampling — greedy decoding vs. probabilistic sampling

What you'll build: A 4-gram model trained on Wikipedia that generates text from a prompt, with a temperature switch that mirrors the same decoding knob exposed by modern LLMs.

Notebook 3: Advanced Agentic Harness

Upgrade every component of the basic harness toward production-grade systems like Claude Code, Devin, or modern research agents.

Core concepts:

• Typed tools with Pydantic — Auto-generated JSON schemas and robust validation
• DAG orchestration — Parallel execution of independent tasks with asyncio instead of sequential processing
• Multi-tier memory — Working, episodic, and semantic memory with retrieval
• Verification hierarchy — Deterministic tests → output diffing → LLM-as-judge → human review
• Multi-agent roles — Planner / Worker / Critic specialization for robustness
• Multi-dimensional budgeting — Graceful degradation under token, time, and cost constraints
• Structured tracing — Full observability and replay capability

What you'll build: A three-city comparison agent that demonstrates natural parallelism (up to 9 concurrent fetches), final aggregation, and verifiable outputs — with trace plots for per-step latency, budget pressure, and tokens by role.

Contents

Notebooks	Blog post	Content
01 - Basic Agentic Harness.ipynb	Building a Basic Agentic Harness	Start here to understand the fundamentals
02 - Small Language Model.ipynb	Build a (small) language model by counting	Build an n-gram language model from scratch and generate text
03 - Advanced Agentic Harness.ipynb	Building an Advanced Agentic Harness	Production-grade patterns: DAG orchestration, memory, verification, and multi-agent roles

Setup

1) Install dependencies (recommended: uv)

1. Install uv (if needed):

curl -LsSf https://astral.sh/uv/install.sh | sh

2. Create an environment and install dependencies:

git clone https://github.com/DataForScience/LLMs.git
cd LLMs
uv venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
uv sync

2) API Keys (Optional)

The notebooks run end-to-end in mock mode without any API keys. They include rule-based mock LLM providers that are smart enough to drive the demos.

To use real LLM APIs, set your Anthropic API key:

export ANTHROPIC_API_KEY=sk-ant-...

Then change BACKEND = "mock" to BACKEND = "anthropic" in the notebook.

3) Launch notebooks

jupyter notebook

Repository Structure

LLMs/
├── 01 - Basic Agentic Harness.ipynb     # Notebook 1: Fundamentals
├── 02 - Small Language Model.ipynb      # Notebook 2: n-gram language model
├── 03 - Advanced Agentic Harness.ipynb  # Notebook 3: Production-grade patterns
├── data/                                 # Logos and assets
│   ├── D4Sci_logo_ball.png
│   ├── D4Sci_logo_full.png
│   ├── bgoncalves.png
│   └── wikitext_quadgram_model.pkl       # Cached 4-gram model (built on first run)
├── d4sci.mplstyle                        # Custom matplotlib style
├── pyproject.toml                        # Dependency manifest (for `uv sync`)
├── uv.lock                               # Lock file for reproducible builds
└── LICENSE                               # MIT License

Learning Path

1. Start with Notebook 1 (01 - Basic Agentic Harness.ipynb) to understand the core concepts
2. Continue with Notebook 2 (02 - Small Language Model.ipynb) to see how language models work from the ground up
3. Finish with Notebook 3 (03 - Advanced Agentic Harness.ipynb) to upgrade the harness with production-grade patterns

Key Features

• Fully reproducible — Deterministic outputs for teaching and debugging
• Production-ready patterns — Learn the same techniques used in Claude Code, Cursor, and Devin
• Hands-on implementation — Build everything from scratch to understand every design decision

---

Questions?

Reach out at info@data4sci.com or open an issue if something isn't working.

Author

Bruno Gonçalves Data For Science, Inc. Web: www.data4sci.com Twitter/X: @bgoncalves LinkedIn: @bmtgoncalves Email: info@data4sci.com Schedule a Call: data4sci.com/call