agent limitations

geemi725 · geemi725 · commit dddc258a3203 · 2026-06-02T15:51:08.000-07:00
diff --git a/chapter_3/agents.md b/chapter_3/agents.md
@@ -84,7 +84,44 @@ At FutureHouse, we're  seeking to accelerate scientific research with an AI agen
 As introduced in chapter 2, [PaperQA2](https://github.com/Future-House/paper-qa) is our scientific information retrieval system. Our data analysis agent is named [Finch](https://github.com/Future-House/finch). Finch is an AI agent framework designed to perform complex scientific data analysis tasks by iteratively working through Jupyter notebooks. This agent takes in datasets and prompts, then systematically explores, analyzes, and interprets the data to provide comprehensive answers and insights. Then we have [Robin](https://www.futurehouse.org/research-announcements/demonstrating-end-to-end-scientific-discovery-with-robin-a-multi-agent-system), our first multi-agent system for scientific discovery. We applied Robin to identify ripasudil, a Rho-kinase (ROCK) inhibitor clinically used to treat glaucoma, as a novel therapeutic candidate for dry age-related macular degeneration (dAMD), a leading cause of irreversible blindness worldwide. The second iteration of Robin, named [Kosmos](https://edisonscientific.com/articles/announcing-kosmos?gad_source=1&gad_campaignid=23563065812&gbraid=0AAAABB7BYdBNoPj2BU82YRgDuN7FLSbFp&gclid=CjwKCAjwwJzPBhBREiwAJfHRnZ_daxcLWR4IyY7swufHvUA5GBDm-dNoDpG-gSJng_9pe96pNd0ciBoCT7wQAvD_BwE), is our first AI scientist. The core innovation in Kosmos is the use of structured world models, which allow efficient incorporation of information extracted over hundreds of agent trajectories. This also allows the agent to maintain coherence towards a specific research objective over tens of millions of tokens. 
 
 
-## 3.1.4 Additional Reading
+## 3.1.4 Limitations of AI Agents
+
+AI agents are comparatively better at performing domain specific tasks than LLMs by themselves. However, this is does **not** mean they are free of limitations. Let's take a look some limitations AI agents face.
+
+* **Context window constraints** LLMs have a finite context window. In multi-step agentic workflows, the accumulated message history — including tool calls, observations, and reasoning — can quickly approach or exceed this limit. When truncation occurs, the agent loses access to earlier reasoning, which can cause it to repeat steps, contradict itself, or fail entirely.
+
+* **No memory across runs**  By default, agents have no memory between runs. Each new rollout starts from scratch. In other words, the `DemoEnvState` is ephemeral. Once a rollout ends, nothing persists. If we wanted the agent to learn from prior protein analyses, we'd need to add external memory (e.g., a vector store or a structured cache).
+
+*  **Hallucination and unreliable tool outputs** LLMs can generate plausible-sounding but factually incorrect content, especially when answering questions outside their training data or when tool outputs are ambiguous. For example `handle_tool_exc=True` in `step()` method catches tool exceptions and converts them to messages rather than crashing. This is safe, but the agent may not recover gracefully. It might keep retrying a broken tool or hallucinate a result.
+
+* **Error propagation across steps** Agents take sequential actions, so an early mistake; a wrong tool choice, a misinterpreted observation, or a malformed argument can cascade through subsequent steps and compound into a larger failure. Errors are not always recoverable, especially if the agent lacks explicit error-handling logic.
+
+* **Incomplete or premature termination** Agents driven by a fixed step budget may terminate before completing a task, or may call a termination tool too early because the LLM misjudges task completion. Conversely, without a step limit, agents can loop indefinitely on unsolvable tasks.
+
+*  **Limited generalization across tasks** Agents are sensitive to how tools are described and how prompts are framed. A system prompt or tool description that works well for one task may perform poorly on a slightly different one. Agents often lack the robustness to handle out-of-distribution inputs gracefully.
+
+### Mitigations & Performance Improvements
+
+| Limitation | Mitigation|
+| --- | --- |
+| Context overflow| Summarize or compress old messages; use sliding window memory; set output length limits on tool responses |
+| No persistent memory |  memoryAdd a vector store or structured cache to persist and retrieve results across runs |
+| Hallucination | Ground LLM outputs with retrieval (RAG); require citations; use tool outputs as the source of truth over model priors |
+| Error propagation | Add explicit error handling and retry logic; validate tool inputs/outputs before passing them downstream |
+| Premature/no termination | Check for termination signals after every step; set sensible step budgets based on task complexity|
+| Poor generalization | Invest in prompt engineering and tool description clarity; test on diverse inputs; use few-shot examples in the system prompt |
+
+### Evaluating agent performance 
+
+This is as important as building the agent itself. Rather than just inspecting final outputs, consider:
+
+- Trajectory analysis: did the agent take the most efficient path, or did it waste steps?
+- Tool use accuracy: did it call the right tools with valid arguments?
+- Failure mode logging: track when and why agents fail to complete tasks, to guide future improvements
+
+Frameworks like LDP and Aviary are designed with evaluation in mind: trajectories are structured objects you can analyze programmatically, making it straightforward to build automated eval pipelines on top of your existing rollout infrastructure.
+
+## 3.1.5 Additional Reading
 - More mathematical explanations can be found in this paper: [Aviary: training language agents on challenging scientific tasks](https://arxiv.org/abs/2412.21154)
 - [AI Agents - IBM tutorials](https://www.ibm.com/think/topics/ai-agents)
 
diff --git a/chapter_3/aviary_agent_implementation.ipynb b/chapter_3/aviary_agent_implementation.ipynb
@@ -528,50 +528,6 @@
     "    if tc.function.name == \"submit_final_answer\":\n",
     "        print(tc.function.arguments[\"answer\"])"
    ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "15",
-   "metadata": {},
-   "source": [
-    "## 3.2.5 Limitations of AI Agents & Improving Performance\n",
-    "\n",
-    "AI agents are comparatively better at performing domain specific tasks than LLMs by themselves. However, this is does **not** mean they are free of limitations. Let's take a look some limitations AI agents face.\n",
-    "\n",
-    "* **Context window constraints** LLMs have a finite context window. In multi-step agentic workflows, the accumulated message history — including tool calls, observations, and reasoning — can quickly approach or exceed this limit. When truncation occurs, the agent loses access to earlier reasoning, which can cause it to repeat steps, contradict itself, or fail entirely.\n",
-    "\n",
-    "* **No memory across runs**  By default, agents have no memory between runs. Each new rollout starts from scratch. In other words, the `DemoEnvState` is ephemeral. Once a rollout ends, nothing persists. If we wanted the agent to learn from prior protein analyses, we'd need to add external memory (e.g., a vector store or a structured cache).\n",
-    "\n",
-    "*  **Hallucination and unreliable tool outputs** LLMs can generate plausible-sounding but factually incorrect content, especially when answering questions outside their training data or when tool outputs are ambiguous. For example `handle_tool_exc=True` in `step()` method catches tool exceptions and converts them to messages rather than crashing. This is safe, but the agent may not recover gracefully. It might keep retrying a broken tool or hallucinate a result.\n",
-    "\n",
-    "* **Error propagation across steps** Agents take sequential actions, so an early mistake; a wrong tool choice, a misinterpreted observation, or a malformed argument can cascade through subsequent steps and compound into a larger failure. Errors are not always recoverable, especially if the agent lacks explicit error-handling logic.\n",
-    "\n",
-    "* **Incomplete or premature termination** Agents driven by a fixed step budget may terminate before completing a task, or may call a termination tool too early because the LLM misjudges task completion. Conversely, without a step limit, agents can loop indefinitely on unsolvable tasks.\n",
-    "\n",
-    "*  **Limited generalization across tasks** Agents are sensitive to how tools are described and how prompts are framed. A system prompt or tool description that works well for one task may perform poorly on a slightly different one. Agents often lack the robustness to handle out-of-distribution inputs gracefully.\n",
-    "\n",
-    "### Mitigations & Performance Improvements\n",
-    "\n",
-    "| Limitation | Mitigation|\n",
-    "| --- | --- |\n",
-    "| Context overflow| Summarize or compress old messages; use sliding window memory; set output length limits on tool responses |\n",
-    "| No persistent memory |  memoryAdd a vector store or structured cache to persist and retrieve results across runs |\n",
-    "| Hallucination | Ground LLM outputs with retrieval (RAG); require citations; use tool outputs as the source of truth over model priors |\n",
-    "| Error propagation | Add explicit error handling and retry logic; validate tool inputs/outputs before passing them downstream |\n",
-    "| Premature/no termination | Check for termination signals after every step; set sensible step budgets based on task complexity|\n",
-    "| Poor generalization | Invest in prompt engineering and tool description clarity; test on diverse inputs; use few-shot examples in the system prompt |\n",
-    "\n",
-    "### Evaluating agent performance \n",
-    "\n",
-    "This is as important as building the agent itself. Rather than just inspecting final outputs, consider:\n",
-    "\n",
-    "- Trajectory analysis: did the agent take the most efficient path, or did it waste steps?\n",
-    "- Tool use accuracy: did it call the right tools with valid arguments?\n",
-    "- Failure mode logging: track when and why agents fail to complete tasks, to guide future improvements\n",
-    "\n",
-    "Frameworks like LDP and Aviary are designed with evaluation in mind: trajectories are structured objects you can analyze programmatically, making it straightforward to build automated eval pipelines on top of your existing rollout infrastructure.\n",
-    "\n"
-   ]
   }
  ],
  "metadata": {
diff --git a/resources/python_basics.ipynb b/resources/python_basics.ipynb
@@ -18,7 +18,7 @@
     "\n",
     "Python is popular because it is easy to read, has a large ecosystem of scientific libraries, and works well with modern AI tools.\n",
     "\n",
-    "## Installing Python\n",
+    "## 4.2.1 Installing Python\n",
     "\n",
     "The easiest way to get started is by installing:\n",
     "\n",
@@ -34,7 +34,7 @@
     "\n",
     "> **Video:** Installation walkthrough [for Windows](https://www.youtube.com/watch?v=YKSpANU8jPE), [for Mac](https://www.youtube.com/watch?v=utVZYVJSTZA).\n",
     "\n",
-    "## Your First Python Program\n",
+    "## 4.2.2 Write Your First Python Program\n",
     "\n",
     "Python scripts are saved with `.py` extension. You can create a file named `hello.py` write `print(\"Hello, world!\")` in it. \n",
     "\n",
@@ -60,7 +60,7 @@
    "id": "2",
    "metadata": {},
    "source": [
-    "# Integrated Development Environments (IDEs)\n",
+    "### Integrated development environments (IDEs)\n",
     "\n",
     "While Python code can be written in a simple text editor, most programmers use an **Integrated Development Environment (IDE)**. An IDE provides tools that make coding easier, including syntax highlighting, code completion, debugging, and project management.\n",
     "\n",
@@ -79,7 +79,7 @@
     "> **Video:** Getting started with VS code - \n",
     "> [Link](https://www.youtube.com/watch?v=D2cwvpJSBX4)\n",
     "\n",
-    "### AI-Assisted Coding with Cursor\n",
+    "### AI-assisted coding with cursor\n",
     "\n",
     "In recent years, AI-powered code editors have become popular tools for learning and programming. One example is **Cursor**, an editor built on top of VS Code that includes AI features for writing, explaining, and debugging code.\n",
     "\n",
@@ -99,7 +99,7 @@
     "\n",
     "Now let's shift gears and take a much more detailed look into writing python codes.\n",
     "\n",
-    "## Variables in python\n",
+    "## 4.2.3 Variables in Python\n",
     "\n",
     "Variables are used to store information that can be referenced and manipulated later in a program. You can think of a variable as a labeled container that holds a value, such as a number, a piece of text, or a collection of data.\n",
     "\n",
@@ -144,7 +144,7 @@
    "id": "6",
    "metadata": {},
    "source": [
-    "## Basic Data Types\n",
+    "## 4.2.4 Basic Data Types\n",
     "\n",
     "Every value stored in Python has a data type. The data type determines what kind of information is stored and what operations can be performed on it.\n",
     "\n",
@@ -171,7 +171,7 @@
     "is_significant = True\n",
     "```\n",
     "\n",
-    "## Basic Arithmetic\n",
+    "## 4.2.5 Basic Arithmetic\n",
     "\n",
     "Python can perform mathematical calculations just like a calculator. Arithmetic operations are useful in many scientific applications, including data analysis, statistical calculations, unit conversions, and numerical simulations.\n",
     "\n",
@@ -208,7 +208,7 @@
    "id": "8",
    "metadata": {},
    "source": [
-    "## Data Structures\n",
+    "## 4.2.6 Data Structures\n",
     "\n",
     "As programs become more complex, we often need to store and organize collections of data rather than individual values. **Data structures** are ways of organizing data so that it can be accessed, modified, and analyzed efficiently.\n",
     "\n",
@@ -244,7 +244,7 @@
    "id": "10",
    "metadata": {},
    "source": [
-    "## Dictionaries\n",
+    "### Dictionaries\n",
     "\n",
     "Dictionaries are data structures that store information as **key-value pairs**. Instead of accessing values by their position (as in a list), values in a dictionary are accessed using a unique key.\n",
     "\n",
@@ -275,7 +275,7 @@
    "id": "12",
    "metadata": {},
    "source": [
-    "## Conditional Statements\n",
+    "### Conditional Statements\n",
     "\n",
     "Conditional statements allow your program to make decisions based on whether a condition is true or false. They are commonly used in scientific analysis to filter data, apply thresholds, or trigger different actions depending on the results.\n",
     "\n",
@@ -302,7 +302,7 @@
    "id": "14",
    "metadata": {},
    "source": [
-    "## Loops\n",
+    "## 4.2.7 Loops\n",
     "\n",
     "Loops allow you to repeat the same operation multiple times without rewriting code. This is especially useful when working with datasets, collections of genes, proteins, samples, or experimental measurements.\n",
     "\n",
@@ -355,7 +355,7 @@
    "id": "18",
    "metadata": {},
    "source": [
-    "## Functions\n",
+    "## 4.2.8 Functions\n",
     "\n",
     "As programs grow larger, it becomes useful to group related code into reusable blocks called **functions**. Functions allow you to perform a specific task whenever needed without rewriting the same code multiple times.\n",
     "\n",
@@ -388,7 +388,7 @@
    "id": "20",
    "metadata": {},
    "source": [
-    "## Best Practices\n",
+    "## 4.2.9 Python Best Practices\n",
     "\n",
     "* Use meaningful variable names.\n",
     "* Write small, reusable functions.\n",
@@ -398,7 +398,7 @@
     "* Use version control (Git) for larger projects.\n",
     "\n",
     "\n",
-    "## Importing Libraries\n",
+    "## 4.2.10 Importing Libraries\n",
     "\n",
     "Python's power comes from its libraries. A library is a collection of pre-written code that provides additional functionality, allowing you to perform complex tasks without writing everything from scratch.\n",
     "\n",
@@ -451,7 +451,7 @@
    "id": "24",
    "metadata": {},
    "source": [
-    "## Working with Data\n",
+    "## 4.2.11 Working with Data\n",
     "\n",
     "One of the main reasons scientists use Python is its ability to work efficiently with data. Experimental results, survey responses, genomic datasets, and simulation outputs are often stored in tabular formats such as CSV (Comma-Separated Values) files.\n",
     "\n",
@@ -486,7 +486,7 @@
    "id": "26",
    "metadata": {},
    "source": [
-    "## Plotting Data\n",
+    "### Plotting Data\n",
     "\n",
     "Visualizing data is an important part of scientific analysis. Graphs and charts can help reveal patterns, trends, outliers, and relationships that may not be obvious when looking at raw numbers alone.\n",
     "\n",
@@ -518,7 +518,7 @@
    "id": "28",
    "metadata": {},
    "source": [
-    "## Learn More\n",
+    "## 4.2.12 Additional Resources\n",
     "\n",
     "* [NumPy basics](https://www.youtube.com/watch?v=QUT1VHiLmmI)\n",
     "* [Pandas basics](https://www.youtube.com/watch?v=EXIgjIBu4EU)\n",
