You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<p>Thanks to Nate Rush, Manish Shetty, Basil Halperin for helpful comments.</p>
276
+
</div></div><dl>
272
277
<dt>An apple-picking model of AI work.</dt>
273
278
<dd>
274
279
<p>Here’s a simple model useful for thinking about AI’s contribution to solving problems.</p>
@@ -279,11 +284,11 @@ <h1 class="title">An Apple-Plucking Model of Agents</h1>
279
284
<dt>Implications of the apple-picking model.</dt>
280
285
<dd>
281
286
<oltype="1">
282
-
<li>Agents can autonomously advance the state-of-the-art on an optimization problem, yet still not be a perfect substitute for human labor (they can find the unpicked low apples).</li>
283
-
<li>Agent contribution to a problem, as you scale the expenditure, will be higher than human contribution, but then asymptote to a lower maximum value.</li>
284
-
<li>Agents will have relatively bigger value (relative to humans) for problems that are not yet heavily optimized.</li>
287
+
<li>Agents can autonomously advance the state-of-the-art on an optimization problem, yet still not be a perfect substitute for human labor (they can find the low apples that haven’t been picked yet).</li>
288
+
<li>Agent contribution to a problem, as you scale the expenditure, will be higher than human contribution, but then asymptote to a lower maximum value (they’ll pick a lot of apples, but will never be able to pick them all).</li>
289
+
<li>Agents will have relatively bigger value, relative to humans, for problems that are not yet heavily optimized (robots are useful for trees that have never been picked).</li>
285
290
<li>Agents will asymptote to higher points if they are given better human starting points (if a tree is partly picked).</li>
286
-
<li>To gauge the ability of agents we want to test not just for their ability to improve performance, but their <em>reach</em>.</li>
291
+
<li>To gauge the ability of agents we want to test not just for their ability to improve performance, but their <em>reach</em> (we want to benchmark robots not on how many apples they can pick, but the height of the highest apple they can reach).</li>
287
292
</ol>
288
293
</dd>
289
294
<dt>Relation to other literature.</dt>
@@ -393,22 +398,13 @@ <h3 class="anchored" data-anchor-id="state-variables-and-dynamics">1. State vari
<p><strong>A) Activation.</strong> With <spanclass="math inline">\(\lambda_t = 0\)</span>, <spanclass="math inline">\(a_t = h_t \to 1\)</span>. So the agent can ever turn on <strong>iff <spanclass="math inline">\(\bar{a} < 1\)</span></strong>. If <spanclass="math inline">\(\bar{a} \ge 1\)</span>, <spanclass="math inline">\(\lambda_t \equiv 0\)</span> forever. Activation-time approximation: <spanclass="math inline">\(h_t = 1 - p^t \ge \bar{a}\)</span><spanclass="math inline">\(\Leftrightarrow\)</span><spanclass="math inline">\(t \ge \ln(1-\bar{a})/\ln p\)</span>; <spanclass="math inline">\(p\)</span> mainly shifts <em>when</em> activation happens.</p>
395
400
<p><strong>B) Crossing human level.</strong> As <spanclass="math inline">\(t \to \infty\)</span>, <spanclass="math inline">\(h_t \to 1\)</span>. If <spanclass="math inline">\(\lambda_t < 1\)</span>, <spanclass="math inline">\(a_t \to 1\)</span>; if <spanclass="math inline">\(\lambda_t \ge 1\)</span>, <spanclass="math inline">\(a_t = \lambda_t\)</span>. So asymptotically <spanclass="math inline">\(\lambda_{t+1} \to f(1)\)</span> with <spanclass="math inline">\(f(1) = 0\)</span> if <spanclass="math inline">\(1 < \bar{a}\)</span>, and <spanclass="math inline">\(f(1) = \alpha + \beta(1-\bar{a})\)</span> if <spanclass="math inline">\(1 \ge \bar{a}\)</span>. So <strong>takeoff past human level</strong> (eventually <spanclass="math inline">\(\lambda > 1\)</span>) <strong>iff</strong><spanclass="math display">\[\boxed{\alpha + \beta(1-\bar{a}) > 1.}\]</span> Interpretation: “If the orchard were fully human-level (<spanclass="math inline">\(a=1\)</span>), would the next agent be at least human-level?” If not, the system stays below 1. This condition is essentially independent of <spanclass="math inline">\(p\)</span> (timing, not whether).</p>
396
-
<p><strong>C) Above human level: runaway vs saturation.</strong> For <spanclass="math inline">\(\lambda_t \ge 1\)</span>, <spanclass="math inline">\(a_t = \lambda_t\)</span> and <spanclass="math display">\[\lambda_{t+1} = \alpha + \beta(\lambda_t - \bar{a}).\]</span> - <strong>Runaway / hard takeoff</strong> iff <spanclass="math inline">\(\boxed{\beta > 1}\)</span> (roughly geometric growth in <spanclass="math inline">\(\lambda_t\)</span>). - <strong>Soft takeoff / saturation</strong> iff <spanclass="math inline">\(\boxed{\beta < 1}\)</span>: convergence to <spanclass="math display">\[\lambda^* = \frac{\alpha - \beta\bar{a}}{1-\beta}\]</span> (provided the system crosses 1 first). - <strong>Knife-edge</strong><spanclass="math inline">\(\beta = 1\)</span>: linear growth.</p>
401
+
<p><strong>C) Above human level: runaway vs saturation.</strong> For <spanclass="math inline">\(\lambda_t \ge 1\)</span>, <spanclass="math inline">\(a_t = \lambda_t\)</span> and <spanclass="math display">\[\lambda_{t+1} = \alpha + \beta(\lambda_t - \bar{a}).\]</span></p>
402
+
<ul>
403
+
<li><p><strong>Runaway / hard takeoff</strong> iff <spanclass="math inline">\(\boxed{\beta > 1}\)</span> (roughly geometric growth in <spanclass="math inline">\(\lambda_t\)</span>).</p></li>
404
+
<li><p><strong>Soft takeoff / saturation</strong> iff <spanclass="math inline">\(\boxed{\beta < 1}\)</span>: convergence to <spanclass="math display">\[\lambda^* = \frac{\alpha - \beta\bar{a}}{1-\beta}\]</span> (provided the system crosses 1 first).</p></li>
405
+
<li><p><strong>Knife-edge</strong><spanclass="math inline">\(\beta = 1\)</span>: linear growth.</p></li>
<p>The figures below implement this model: four canonical trajectories, phase diagram in <spanclass="math inline">\((\alpha,\beta)\)</span>, cobweb plots of the asymptotic map, and sensitivity to <spanclass="math inline">\(p\)</span> and <spanclass="math inline">\(\bar{a}\)</span>.</p>
0 commit comments