recursive

Author: tecunningham

_freeze/posts/2025-09-13-recursive-self-improvement-explosion/execute-results/html.json

@@ -23,7 +23,27 @@
   margin: 0 0.8em 0.2em -1em; /* quarto-specific, see https://github.com/quarto-dev/quarto-cli/issues/4556 */ 
   vertical-align: middle;
 }
-</style>
+/* CSS for citations */
+div.csl-bib-body { }
+div.csl-entry {
+  clear: both;
+  margin-bottom: 0em;
+}
+.hanging-indent div.csl-entry {
+  margin-left:2em;
+  text-indent:-2em;
+}
+div.csl-left-margin {
+  min-width:2em;
+  float:left;
+}
+div.csl-right-inline {
+  margin-left:2em;
+  padding-left:1em;
+}
+div.csl-indent {
+  margin-left: 2em;
+}</style>
 
 
 <script src="../site_libs/quarto-nav/quarto-nav.js"></script>
@@ -637,15 +657,15 @@ <h1>2026-01-07 | tkwa&nbsp; model</h1>
   intelligence --&gt;|??| economic_value["economic value"]
   intelligence --&gt;|??| RD</code></pre>
 </section>
-<section id="jones-model" class="level1">
+<section id="jones-model" class="level1 page-columns page-full">
 <h1>2026-01-13 | Jones model</h1>
 <p>Basic model with accumulating ideas: <span class="math display">\[\begin{gathered}\dot{A}=R^{\gamma}\\
    \xymatrix{*++[F]{R\&amp;D} \ar[r] &amp; *++[F]{\Delta algorithms}\ar[r] &amp; *++[F]{algorithms}}
 \end{gathered}\]</span></p>
-<p>Model with shoulders/armpits of giants: <span class="math display">\[\begin{gathered}\dot{A}=R^{\gamma}A^{1-\beta}\\
+<p>Model with shoulders/armpits of giants (Jones model):<a href="#fn2" class="footnote-ref" id="fnref2" role="doc-noteref"><sup>2</sup></a> <span class="math display">\[\begin{gathered}\dot{A}=R^{\gamma}A^{1-\beta}\\
    \xymatrix{*++[F]{R\&amp;D} \ar[r] &amp; *++[F]{\Delta knowledge}\ar[r] &amp; *++[F]{knowledge}\ar@/^2em/[l]}
 \end{gathered}\]</span></p>
-<p>Recursive self-improvement, where knowledge actually helps R&amp;D: <span class="math display">\[\begin{gathered}\dot{A}=R^{\gamma}\\
+<div class="no-row-height column-margin column-container"><div id="fn2"><p><sup>2</sup>&nbsp;This was introduced by <span class="citation" data-cites="jones1995rd">Jones (<a href="#ref-jones1995rd" role="doc-biblioref">1995</a>)</span>, where there are diminishing returns to knowledge, whereas Romer (1990) had assumed no diminishing returns to knowledge, <span class="math inline">\(\beta=0\)</span>.</p></div></div><p>Recursive self-improvement, where knowledge actually helps R&amp;D: <span class="math display">\[\begin{gathered}\dot{A}=R^{\gamma}\\
    \xymatrix{*++[F]{R\&amp;D} \ar[r] &amp; *++[F]{\Delta knowledge}\ar[r] &amp; *++[F]{knowledge}\ar@/^2em/[l]\ar@/^3em/[ll]}
 \end{gathered}\]</span></p>
 <dl>
@@ -685,35 +705,50 @@ <h2 class="anchored" data-anchor-id="metaphors">metaphors</h2>
 <dl>
 <dt>Drawing balls from an urn.</dt>
 <dd>
-There’s some distribution of values, then you get a very nice expression. The expected value of <span class="math inline">\(N\)</span> draws just depends on the distribution of values <span class="math inline">\(v\)</span>, &amp; the extreme-value distribution. This is exactly Kortum (1997).
+There’s some distribution of values, then you get a very nice expression. The expected value of <span class="math inline">\(N\)</span> draws just depends on the extreme value distribution of <span class="math inline">\(f(v)\)</span>. This is exactly Kortum (1997).
 </dd>
 <dt>A tool factory makes better tools.</dt>
 <dd>
 I think this is Jones’ metaphor. <em>Distinct</em> from Kortum, because the returns to search now depends on the stock of ideas.
 </dd>
-<dt>Lego.</dt>
+<dt>Machine tools and regular tools</dt>
+<dd>
+<ol type="1">
+<li>The Dutch make machine tools.</li>
+<li>The machine tools make regular tools.</li>
+<li>The regular tools make products.</li>
+</ol>
+<p>At some point the machine tools become good enough to make themselves, but it’s a discrete jump.</p>
+</dd>
+<dt>Lego - combining ideas.</dt>
 <dd>
 Weitzmann’s recombinant search is like this: you combine ideas to make new ideas, now you have a larger stock of ideas to combine.
 </dd>
 <dt>Recipes.</dt>
 <dd>
-You try out recipes, which are combinations of prior recipes. <span class="citation" data-cites="jones2023recipes">@jones2023recipes</span>
+You try out recipes, which are combinations of prior recipes. <span class="citation" data-cites="jones2023recipes">Jones (<a href="#ref-jones2023recipes" role="doc-biblioref">2023</a>)</span> – something like a reconciliation of Weitzman &amp; Kortum.
 </dd>
 <dt>Blacksmith.</dt>
 <dd>
-You have a hammer and you spend time making horseshoes, or working on a new hammer.
-</dd>
-<dd>
-A harder hammer both (1) makes horseshoes faster, or (2) makes your hammer still harder.
+<p>You have a hammer and you spend time making horseshoes, or working on a new hammer.</p>
+<p>A harder hammer both (1) makes horseshoes faster, or (2) makes your hammer still harder.</p>
 </dd>
 </dl>
 
 
+
 </section>
 </section>
 
 
-</main> <!-- /main -->
+<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" role="doc-bibliography" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0" role="list">
+<div id="ref-jones1995rd" class="csl-entry" role="listitem">
+Jones, Charles I. 1995. <span>“R&amp;d-Based Models of Economic Growth.”</span> <em>Journal of Political Economy</em> 103 (4): 759–84. https://doi.org/<a href="https://doi.org/10.1086/262002">https://doi.org/10.1086/262002</a>.
+</div>
+<div id="ref-jones2023recipes" class="csl-entry" role="listitem">
+———. 2023. <span>“Recipes and Economic Growth: A Combinatorial March down an Exponential Tail.”</span> <em>Journal of Political Economy</em> 131 (8): 1994–2031. <a href="https://doi.org/10.1086/723631">https://doi.org/10.1086/723631</a>.
+</div>
+</div></section></div></main> <!-- /main -->
 <script id="quarto-html-after-body" type="application/javascript">
 window.document.addEventListener("DOMContentLoaded", function (event) {
   const toggleBodyColorMode = (bsSheetEl) => {

docs/posts/2025-09-13-recursive-self-improvement-explosion.html

@@ -54,5 +54,40 @@
     "title": "Knowledge-Creating LLMs",
     "section": "Footnotes",
     "text": "Footnotes\n\n\nMany other technologies share knowledge – speaking, writing, printing, the internet – LLMs just continue this progression but further lower the costs of sharing.↩︎"
+  },
+  {
+    "objectID": "posts/2025-09-13-recursive-self-improvement-explosion.html",
+    "href": "posts/2025-09-13-recursive-self-improvement-explosion.html",
+    "title": "Recursive Self-Improvement",
+    "section": "",
+    "text": "TL;DR: an explosion is AI capabilities is possible but it’s intrinsically difficult to forecasat.\n\nDefine an explosion as a significant acceleration of the historical rates of progress in AI, e.g. measured by effective compute required for a given level of accuracy (e.g. perplexity).\n\nProgress in AI is rapid but smooth.\n\nWe can measure progress in various ways, algorithmic progress has accelerated over the last 10-15 years. Epoch estimate that algorithmic efficiency has been growing at 3X/year between 2012-2024, measured by effective compute.\n\nForecasters expect an 8-20% chance of an explosion.\n\nThey define an explosion as a 3X speedup in the historical rate of progress by some measure, from the METR/FRI survey.\n\nAI has already been accelerating AI research.\n\nAI has been self-accelerating for decades: (1) automatic differentiation; (2) bayesian optimization of hyperparameters; (3) neural architecture search; (4) LLM coding autocomplete and chatbots; (5) LLM coding agents.\n\nThere’s good theoretical reason to expect an explosion.\n\n(…)\n\nAI usefulness for optimization depends on features of the setup.\n\nHassabis says AI will make progress wherever there’s (1) combinatorial search space; (2) clear feedback; (3) lots of data or an automatic validator.\nHowever there’s clearly a fourth condition: that the data has some lower-dimensional latent structure. There are many problems that satisfy the first 3 but where we don’t expect substantial progress from autonomous NNs: (A) the telephone book; (B) mapping out stars in the sky; (C) documenting the genome.\n\nAI speedups to discovery depends on the shape of the underlying landscape.\n\n(abc)"
+  },
+  {
+    "objectID": "posts/2025-09-13-recursive-self-improvement-explosion.html#summary-1",
+    "href": "posts/2025-09-13-recursive-self-improvement-explosion.html#summary-1",
+    "title": "Recursive Self-Improvement",
+    "section": "summary",
+    "text": "summary\n\nGoal: forecast when we’re likely to get explosive intelligence growth.\nSummary: once you automate a sufficient number of tasks (do them with capital) then it’ll kick off an intelligence explosion.\nBasic model:\n\nIf \\(\\dot{S}_t=S_t^{1-\\beta}\\), so you get explosion if \\(\\beta<0\\), steady-state growth if \\(\\beta=0\\), and gradual slowing if \\(\\beta>1\\).\nNow if you start automating inputs, the effective exponent gradually gets higher.\n\nMulti-sector version.\n\nYou gradually automate some of the things, so they can be done with capital.\nYou have spillovers between different processes.\n\nEstimates of \\(\\beta\\):\n\nBloom overall \\(\\beta=3\\)\nFor software R&D they estimate \\(\\beta=3\\)\nthey say in software \\(\\beta=0.1\\)\n\nNOTE: \\(\\beta\\)\nQuestions:\n\nAny historical domain where we’ve seen regimes of \\(\\beta<0\\)?"
+  },
+  {
+    "objectID": "posts/2025-09-13-recursive-self-improvement-explosion.html#my-observations",
+    "href": "posts/2025-09-13-recursive-self-improvement-explosion.html#my-observations",
+    "title": "Recursive Self-Improvement",
+    "section": "my observations",
+    "text": "my observations\n\nQ: how to think about the growth effect of uplift vs automation?\nAutomation makes things free, rather than being produced by capital."
+  },
+  {
+    "objectID": "posts/2025-09-13-recursive-self-improvement-explosion.html#tom-houlden-questions",
+    "href": "posts/2025-09-13-recursive-self-improvement-explosion.html#tom-houlden-questions",
+    "title": "Recursive Self-Improvement",
+    "section": "tom houlden questions",
+    "text": "tom houlden questions\nQ: is this graphical system novel?\nQ: is there a nice metaphor/analogy to think about automation/explosion?"
+  },
+  {
+    "objectID": "posts/2025-09-13-recursive-self-improvement-explosion.html#metaphors",
+    "href": "posts/2025-09-13-recursive-self-improvement-explosion.html#metaphors",
+    "title": "Recursive Self-Improvement",
+    "section": "metaphors",
+    "text": "metaphors\n\nDrawing balls from an urn.\n\nThere’s some distribution of values, then you get a very nice expression. The expected value of \\(N\\) draws just depends on the extreme value distribution of \\(f(v)\\). This is exactly Kortum (1997).\n\nA tool factory makes better tools.\n\nI think this is Jones’ metaphor. Distinct from Kortum, because the returns to search now depends on the stock of ideas.\n\nMachine tools and regular tools\n\n\nThe Dutch make machine tools.\nThe machine tools make regular tools.\nThe regular tools make products.\n\nAt some point the machine tools become good enough to make themselves, but it’s a discrete jump.\n\nLego - combining ideas.\n\nWeitzmann’s recombinant search is like this: you combine ideas to make new ideas, now you have a larger stock of ideas to combine.\n\nRecipes.\n\nYou try out recipes, which are combinations of prior recipes. Jones (2023) – something like a reconciliation of Weitzman & Kortum.\n\nBlacksmith.\n\nYou have a hammer and you spend time making horseshoes, or working on a new hammer.\nA harder hammer both (1) makes horseshoes faster, or (2) makes your hammer still harder."
   }
 ]

docs/search.json

@@ -20,4 +20,8 @@
     <loc>tecunningham.github.io/posts/2026-01-29-knowledge-creating-llms.html</loc>
     <lastmod>2026-02-13T19:52:52.473Z</lastmod>
   </url>
+  <url>
+    <loc>tecunningham.github.io/posts/2025-09-13-recursive-self-improvement-explosion.html</loc>
+    <lastmod>2026-02-20T12:54:58.779Z</lastmod>
+  </url>
 </urlset>

docs/sitemap.xml

@@ -409,10 +409,12 @@ Concrete assumptions.
     3. _Shares knowledge._ Can learn tricks. It doesn't affect people at the frontier very much.
 
 Q: what if the capability frontier is the data frontier?
-: - Nice prediction: will not speed up experts, only behind the frontier.
+: 
+    - Nice prediction: will not speed up experts, only behind the frontier.
 
 Natural processes often have explosive *regions*.
-: You 
+: 
+    You 
 
 ##    metaphors
 
@@ -424,16 +426,24 @@ A tool factory makes better tools.
 : 
     I think this is Jones' metaphor. *Distinct* from Kortum, because the returns to search now depends on the stock of ideas.
 
-    Could also talk about machine tools: that you need to make machine tools by hand, which then make lower-quality tools. But eventually.
+Machine tools and regular tools
+: 
+    1. The Dutch make machine tools.
+    2. The machine tools make regular tools.
+    3. The regular tools make products.
+
+    At some point the machine tools become good enough to make themselves, but it's a discrete jump.
 
 Lego - combining ideas.
 : 
     Weitzmann's recombinant search is like this: you combine ideas to make new ideas, now you have a larger stock of ideas to combine.
 
 Recipes.
 : 
-    You try out recipes, which are combinations of prior recipes. @jones2023recipes
+    You try out recipes, which are combinations of prior recipes. @jones2023recipes -- something like a reconciliation of Weitzman & Kortum.
 
 Blacksmith.
-: You have a hammer and you spend time making horseshoes, or working on a new hammer.
-: A harder hammer both (1) makes horseshoes faster, or (2) makes your hammer still harder.
+: 
+    You have a hammer and you spend time making horseshoes, or working on a new hammer.
+
+    A harder hammer both (1) makes horseshoes faster, or (2) makes your hammer still harder.