post

Author: tecunningham

_freeze/posts/2025-09-13-recursive-self-improvement-explosion-optimization/execute-results/html.json

@@ -258,7 +258,7 @@ <h3 class="no-anchor listing-title">
 </a>
 </div>
 </div>
-<div class="quarto-post image-right" data-index="1" data-listing-date-sort="1775476800000" data-listing-file-modified-sort="1775508486903" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="57" data-listing-word-count-sort="11288">
+<div class="quarto-post image-right" data-index="1" data-listing-date-sort="1775476800000" data-listing-file-modified-sort="1775763652527" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="58" data-listing-word-count-sort="11436">
 <div class="thumbnail"><a href="./posts/2026-03-13-apple-picking-ai.html" class="no-external">
 
 <p class="card-img-top"><img src="posts/2026-03-13-apple-picking-ai_files/figure-html/unnamed-chunk-1-1.png"  class="thumbnail-image card-img"/></p>

_freeze/posts/2026-03-13-apple-picking-ai/execute-results/html.json

@@ -31,7 +31,7 @@
 <p>Many people are talking about how AI is autonomously able to contribute to frontier R&amp;D, yet it’s only picking low-hanging fruit: <a href="https://x.com/karpathy/status/2031135152349524125">Andrej Karpathy</a>, <a href="https://www.theatlantic.com/technology/2026/02/ai-math-terrance-tao/686107/">Terence Tao</a>, <a href="https://www.interconnects.ai/p/lossy-self-improvement">Nathan Lambert</a>, <a href="https://www.lesswrong.com/posts/dKpC6wHFqDrGZwnah/ais-can-now-often-do-massive-easy-to-verify-swe-tasks-and-i">Ryan Greenblatt</a>.</p>
 <p>In this note we describe a simple apple-picking model of AI R&amp;D, to help measure the contribution that autonomous agents are making.</p>
 <p>The model takes the metaphor literally: an AI agent helping you to optimize an algorithm is like a robot helping you pick apples. It will take care of all the apples up to a certain height, and it may find apples you haven’t found yet, but there will still be apples out of its reach.</p>
-<p>The model implies that agents can push the frontier forward, but the returns will sharply diminish. It also implies we can meaure an agent’s ability with a human-equivalent time horizon, e.g.&nbsp;as of March 2026 agents seem to be able to find optimizations on frontier algorithms worth about 1 week of professional human labor, yet those effects are not additive.<sup>1</sup> You can’t get two weeks labor by running an agent twice.</p>
+<p>The model implies that agents can push the frontier forward, but the returns will sharply diminish. It also implies we can measure an agent’s ability with a human-equivalent time horizon, e.g.&nbsp;as of March 2026 agents seem to be able to find optimizations on frontier algorithms worth about 1 week of professional human labor, yet those effects are not additive.<sup>1</sup> You can’t get two weeks labor by running an agent twice.</p>
 <p>The basic ideas can all be seen in the drawing below. Here the human and robot are both picking trees. The robot is cheaper to run, but it can only reach the low apples. In the illustration they have both picked 4 apples, yet left the tree in a very different state, such that the robot isn’t ready to replace the human yet:</p>
 <div class="cell" data-layout-align="center">
 <div class="cell-output-display">
@@ -74,7 +74,7 @@
 <p>The model sits between a few different literatures in economics, but as far as I can tell doesn’t already exist:</p>
 <ul>
 <li>Models of tasks or knowledge hierarchies.<sup>2</sup> The apple-picking model differs in that tasks are performed <em>cumulatively</em>, instead of a fresh set of tasks each period.</li>
-<li>Models of discovery.<sup>3</sup> The apple-picking models the discovery landscape in a different way (as far as I’m aware) which gives very simple clear results.</li>
+<li>Models of discovery.<sup>3</sup> The apple-picking model models the discovery landscape in a different way (as far as I’m aware) which gives very simple clear results.</li>
 <li>Models of knowledge accumulation.<sup>4</sup> The apple-picking model differs in that the state of knowledge is not represented by a single scalar, e.g.&nbsp;the stock of ideas, instead the state is the combination of low-hanging and high-hanging ideas.</li>
 </ul>
 </dd>
@@ -98,7 +98,7 @@
 </dd>
 <dt>Traditional models of AI-assisted R&amp;D imply a permanent acceleration.</dt>
 <dd>
-<p>There are two standard models of AI’s effect on R&amp;D used in the RSI literature: (1) it multiplies the effectiveness of human researchers; (2) it replaces human researchers with computer researchers (see more discussion below). Both models imply that when AI capabilities get better it’ll an acceleration in the rate of progress, which persists:</p>
+<p>There are two standard models of AI’s effect on R&amp;D used in the RSI literature: (1) it multiplies the effectiveness of human researchers; (2) it replaces human researchers with computer researchers (see more discussion below). Both models imply that when AI capabilities get better it’ll lead to an acceleration in the rate of progress, which persists:</p>
 <div class="cell" data-layout-align="center">
 <div class="cell-output-display">
 <div class="quarto-figure quarto-figure-center">
@@ -170,7 +170,7 @@
 <dd>
 <ul>
 <li><p><em>Uplift.</em> In practice AI will both (A) accelerate humans; (B) replace humans. The apple-picking model only focuses on the second. In the long-run we expect the second effect to dominate, but it’s not clear where we are now.</p></li>
-<li><p><em>Directed search.</em> We assumed that the probability of finding an apple is independent of other apples already found. Realistically people have an ability to direct their attention, it’s not clear whether the implications would significantly changed.</p></li>
+<li><p><em>Directed search.</em> We assumed that the probability of finding an apple is independent of other apples already found. Realistically people have an ability to direct their attention, it’s not clear whether the implications would significantly change.</p></li>
 <li><p><em>Bottlenecks.</em> The discussion has assumed that progress is purely a function of thinking. In practice there are other bottlenecks, most concretely for AI R&amp;D the reliance on using scarce compute to run experiments. It’s arguable how important compute scarcity is, see for example <span class="citation" data-cites="whitfill2025bottlenecks">Whitfill and Wu (2025)</span>.</p></li>
 </ul>
 </dd>
@@ -236,7 +236,7 @@
 <dl>
 <dt>Apple-picking is a simplification of a landscape-navigation problem.</dt>
 <dd>
-<p>We think of the apple-picking model as a simplification of a more general landscape-navigation model, where you are trying to find a mininmum over a bumpy landscape. You can represent an optimization problem as <img src="https://latex.codecogs.com/png.latex?y=f(%5Cbm%7Bx%7D)">, where you’re trying to choose an <img src="https://latex.codecogs.com/png.latex?%5Cbm%7Bx%7D"> to minimize <img src="https://latex.codecogs.com/png.latex?y">, given some unknown <img src="https://latex.codecogs.com/png.latex?f(%5Ccdot)">. An immediate observation from landscape-navigation problem is that the current elevation is not a sufficient statistic for your state, unless you have a degenerate landscape, like the random landscapes in <span class="citation" data-cites="kortum1997research">Kortum (1997)</span>.</p>
+<p>We think of the apple-picking model as a simplification of a more general landscape-navigation model, where you are trying to find a minimum over a bumpy landscape. You can represent an optimization problem as <img src="https://latex.codecogs.com/png.latex?y=f(%5Cbm%7Bx%7D)">, where you’re trying to choose an <img src="https://latex.codecogs.com/png.latex?%5Cbm%7Bx%7D"> to minimize <img src="https://latex.codecogs.com/png.latex?y">, given some unknown <img src="https://latex.codecogs.com/png.latex?f(%5Ccdot)">. An immediate observation from landscape-navigation problem is that the current elevation is not a sufficient statistic for your state, unless you have a degenerate landscape, like the random landscapes in <span class="citation" data-cites="kortum1997research">Kortum (1997)</span>.</p>
 <p>A reasonable conjecture seems to be that AI agents are good at <em>local</em> optimization, e.g.&nbsp;climbing small hills, worse at <em>global</em> optimization, i.e.&nbsp;finding distant hills.</p>
 <p><a href="https://www.dwarkesh.com/p/terence-tao">Terence Tao has a similar landscape-navigation metaphor</a></p>
 <blockquote class="blockquote">
@@ -394,7 +394,7 @@ Apples are distributed with cumulative distribution function <img src="https://l
 <dt>Human-only labor.</dt>
 <dd>
 <p>With human labor only then apples will grow with diminishing returns: <img src="https://latex.codecogs.com/png.latex?a_t=(1-p%5Et)F(1)."></p>
-<p>This will characterize progress prior to the point at which we can build the first useful robot, <img src="https://latex.codecogs.com/png.latex?a_t%20%5Cgeq%20%5Cbar%7Ba%7D">, after which progress begings to accelerate.</p>
+<p>This will characterize progress prior to the point at which we can build the first useful robot, <img src="https://latex.codecogs.com/png.latex?a_t%20%5Cgeq%20%5Cbar%7Ba%7D">, after which progress begins to accelerate.</p>
 </dd>
 <dt>Recursive self-improvement.</dt>
 <dd>
@@ -410,6 +410,26 @@ With robot labor only (holding <img src="https://latex.codecogs.com/png.latex?h_
 </div>
 </div>
 </div>
+</section>
+<section id="embedding-apple-picking-into-jones-model-rd" class="level2">
+<h2 class="anchored" data-anchor-id="embedding-apple-picking-into-jones-model-rd">Embedding Apple-Picking into Jones-model R&amp;D</h2>
+<p>We can embed the apple-picking model into the workhorse R&amp;D function from <span class="citation" data-cites="jones1995rd">C. I. Jones (1995)</span> as follows, with the following implications:</p>
+<ol type="1">
+<li>Human-only effort has constant elasticity.</li>
+<li>Agent effort has <em>declining</em> elasticity.</li>
+<li>Human &amp; agent effort are complements up to a point.</li>
+</ol>
+<p>We start with this function:</p>
+<p><img src="https://latex.codecogs.com/png.latex?%5Cutt%7B%5Cfrac%7B%5Cdot%7BA%7D%7D%7BA%7D%7D%7Bgrowth%20rate%7D%7Bof%20ideas%7D%0A%20%20%20%20%20%20%20%20%5Cpropto%20%5Cutt%7BR%5E%5Cgamma%7D%7Bresearch%7D%7Binputs%7D%20%5Ctimes%20%5Cutt%7BA%5E%7B-%5Cbeta%7D%7D%7Bfishing%7D%7Bout%7D"></p>
+<p>This can be rewritten in a cumulative form, where <img src="https://latex.codecogs.com/png.latex?%5Cbar%7BR%7D_t"> represents the total research (adjusted for congestion) up to time <img src="https://latex.codecogs.com/png.latex?t">, then (assuming <img src="https://latex.codecogs.com/png.latex?A_0"> is small):</p>
+<p><img src="https://latex.codecogs.com/png.latex?A_t%20%5Capprox%20%5Cbar%7BR%7D_t%5E%7B1/%5Cbeta%7D"></p>
+<p>We can compare this to our apple-picking function, with only human effort <img src="https://latex.codecogs.com/png.latex?x">: <img src="https://latex.codecogs.com/png.latex?%5Cut%7Ba(x)%7D%7Bapples%7D%20=%201-e%5E%7B-rx%7D"></p>
+<p>These can be reconciled if we assume ideas is a nonlinear function of apples-picked (<img src="https://latex.codecogs.com/png.latex?a">): <img src="https://latex.codecogs.com/png.latex?A(a)=%5Cln%5Cleft(%5Cfrac%7B1%7D%7B1-a(x)%7D%5Cright)%5E%7B1/%5Cbeta%7D."></p>
+<p>We can then consider progress in ideas with both humans and robots picking apples, <img src="https://latex.codecogs.com/png.latex?a(x_H,x_L)">: <img src="https://latex.codecogs.com/png.latex?A(a)=%5Cleft%5B%5Cln%5Cleft(%5Cfrac%7B1%7D%7B%5Clambda%20e%5E%7B-r_Hx_H-r_Ax_A%7D+(1-%5Clambda)e%5E%7B-r_Hx_H%7D%7D%5Cright)%5Cright%5D%5E%7B1/%5Cbeta%7D."></p>
+<p>Implications:</p>
+<ol type="1">
+<li>Human-only</li>
+</ol>