deploy: 9d36a75

Author: john-b-yang

quickstart/index.html

@@ -666,21 +666,10 @@
 </li>
 
           <li class="md-nav__item">
-  <a href="#example-configs-quick-experiments" class="md-nav__link">
+  <a href="#pvp-configs-full-benchmarks" class="md-nav__link">
     <span class="md-ellipsis">
 
-        Example Configs (Quick Experiments)
-      
-    </span>
-  </a>
-  
-</li>
-        
-          <li class="md-nav__item">
-  <a href="#main-configs-full-benchmarks" class="md-nav__link">
-    <span class="md-ellipsis">
-      
-        Main Configs (Full Benchmarks)
+        PvP Configs (Full Benchmarks)
 
     </span>
   </a>
@@ -1923,21 +1912,10 @@
 </li>
 
           <li class="md-nav__item">
-  <a href="#example-configs-quick-experiments" class="md-nav__link">
+  <a href="#pvp-configs-full-benchmarks" class="md-nav__link">
     <span class="md-ellipsis">
 
-        Example Configs (Quick Experiments)
-      
-    </span>
-  </a>
-  
-</li>
-        
-          <li class="md-nav__item">
-  <a href="#main-configs-full-benchmarks" class="md-nav__link">
-    <span class="md-ellipsis">
-      
-        Main Configs (Full Benchmarks)
+        PvP Configs (Full Benchmarks)
 
     </span>
   </a>
@@ -2194,12 +2172,12 @@ <h2 id="configuration-tiers">Configuration Tiers</h2>
 </tr>
 <tr>
 <td><code>configs/examples/</code></td>
-<td>Short tournaments (5 rounds)</td>
+<td>Single-player &amp; minimal per-arena samples</td>
 <td>Low</td>
-<td>Learning, quick experiments</td>
+<td>Trying one arena or mode</td>
 </tr>
 <tr>
-<td><code>configs/main/</code></td>
+<td><code>configs/pvp/</code></td>
 <td>Full benchmark runs (15 rounds)</td>
 <td>High</td>
 <td>Reproducing paper results</td>
@@ -2216,24 +2194,19 @@ <h3 id="test-configs-free">Test Configs (Free)</h3>
 <p>No LLM API calls - uses dummy agents that make no changes:</p>
 <div class="highlight"><pre><span></span><code>uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/test/battlesnake.yaml
 </code></pre></div>
-<h3 id="example-configs-quick-experiments">Example Configs (Quick Experiments)</h3>
-<p>Short 5-round tournaments for learning and experimentation:</p>
-<div class="highlight"><pre><span></span><code><span class="c1"># Claude Sonnet 4.5 vs o3 in BattleSnake (5 rounds)</span>
-uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/examples/BattleSnake__claude-sonnet-4-5-20250929__o3__r5__s1000.yaml
+<h3 id="pvp-configs-full-benchmarks">PvP Configs (Full Benchmarks)</h3>
+<p>Full 15-round model-vs-model tournaments, as used in the paper:</p>
+<div class="highlight"><pre><span></span><code><span class="c1"># Claude Sonnet 4.5 vs o3 in BattleSnake</span>
+uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/pvp/BattleSnake__claude-sonnet-4-5-20250929__o3__r15__s1000.yaml
 
 <span class="c1"># Same matchup in other arenas</span>
-uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/examples/CoreWar__claude-sonnet-4-5-20250929__o3__r5__s1000.yaml
-uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/examples/Halite__claude-sonnet-4-5-20250929__o3__r5__s250.yaml
-uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/examples/RoboCode__claude-sonnet-4-5-20250929__o3__r5__s250.yaml
-</code></pre></div>
-<h3 id="main-configs-full-benchmarks">Main Configs (Full Benchmarks)</h3>
-<p>Full 15-round tournaments used in the paper:</p>
-<div class="highlight"><pre><span></span><code><span class="c1"># Full BattleSnake tournament</span>
-uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/main/BattleSnake__claude-sonnet-4-5-20250929__o3__r15__s1000.yaml
+uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/pvp/CoreWar__claude-sonnet-4-5-20250929__o3__r15__s1000.yaml
+uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/pvp/Halite__claude-sonnet-4-5-20250929__o3__r15__s250.yaml
+uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/pvp/RoboCode__claude-sonnet-4-5-20250929__o3__r15__s250.yaml
 </code></pre></div>
 <div class="admonition warning">
 <p class="admonition-title">Cost Warning</p>
-<p>Main configs run 15 rounds with real LLM agents. Expect significant API costs depending on your model choices.</p>
+<p>PvP configs run 15 rounds with real LLM agents. Expect significant API costs depending on your model choices.</p>
 </div>
 <h2 id="viewing-results">Viewing Results</h2>
 <p>After a tournament completes, view results with the interactive viewer:</p>
@@ -2252,22 +2225,22 @@ <h2 id="viewing-results">Viewing Results</h2>
 <h2 id="quick-recipes">Quick Recipes</h2>
 <h3 id="run-your-first-real-tournament">Run your first real tournament</h3>
 <div class="highlight"><pre><span></span><code><span class="c1"># 1. Make sure you have API keys in .env</span>
-<span class="c1"># 2. Run a short example tournament</span>
-uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/examples/BattleSnake__claude-sonnet-4-5-20250929__o3__r5__s1000.yaml
+<span class="c1"># 2. Run a tournament</span>
+uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/pvp/BattleSnake__claude-sonnet-4-5-20250929__o3__r15__s1000.yaml
 
 <span class="c1"># 3. View results</span>
 uv<span class="w"> </span>run<span class="w"> </span>python<span class="w"> </span>scripts/run_viewer.py
 </code></pre></div>
 <h3 id="try-a-different-arena">Try a different arena</h3>
 <div class="highlight"><pre><span></span><code><span class="c1"># CoreWar - Assembly-like programming battle</span>
-uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/examples/CoreWar__claude-sonnet-4-5-20250929__o3__r5__s1000.yaml
+uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/pvp/CoreWar__claude-sonnet-4-5-20250929__o3__r15__s1000.yaml
 
 <span class="c1"># RobotRumble - Multi-robot combat</span>
-uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/examples/RobotRumble__claude-sonnet-4-5-20250929__o3__r5__s250.yaml
+uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/pvp/RobotRumble__claude-sonnet-4-5-20250929__o3__r15__s250.yaml
 </code></pre></div>
 <h3 id="create-a-custom-matchup">Create a custom matchup</h3>
-<p>Copy an example config and modify:</p>
-<div class="highlight"><pre><span></span><code>cp<span class="w"> </span>configs/examples/BattleSnake__claude-sonnet-4-5-20250929__o3__r5__s1000.yaml<span class="w"> </span>configs/my_tournament.yaml
+<p>Copy a config and modify:</p>
+<div class="highlight"><pre><span></span><code>cp<span class="w"> </span>configs/pvp/BattleSnake__claude-sonnet-4-5-20250929__o3__r15__s1000.yaml<span class="w"> </span>configs/my_tournament.yaml
 <span class="c1"># Edit configs/my_tournament.yaml to change models, rounds, etc.</span>
 uv<span class="w"> </span>run<span class="w"> </span>codeclash<span class="w"> </span>run<span class="w"> </span>configs/my_tournament.yaml
 </code></pre></div>

reference/arenas/battlesnake/index.html

@@ -2365,7 +2365,12 @@ <h3 id="codeclash.arenas.battlesnake.battlesnake.BattleSnakeArena.validate_code"
 <span class="normal">203</span>
 <span class="normal">204</span>
 <span class="normal">205</span>
-<span class="normal">206</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span><span class="w"> </span><span class="nf">validate_code</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">agent</span><span class="p">:</span> <span class="n">Player</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">tuple</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">str</span> <span class="o">|</span> <span class="kc">None</span><span class="p">]:</span>
+<span class="normal">206</span>
+<span class="normal">207</span>
+<span class="normal">208</span>
+<span class="normal">209</span>
+<span class="normal">210</span>
+<span class="normal">211</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span><span class="w"> </span><span class="nf">validate_code</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">agent</span><span class="p">:</span> <span class="n">Player</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">tuple</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">str</span> <span class="o">|</span> <span class="kc">None</span><span class="p">]:</span>
     <span class="n">listing</span> <span class="o">=</span> <span class="n">agent</span><span class="o">.</span><span class="n">environment</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;ls&quot;</span><span class="p">)[</span><span class="s2">&quot;output&quot;</span><span class="p">]</span>
     <span class="c1"># Non-Python submissions declare how to launch their server via run.sh (any</span>
     <span class="c1"># language). We trust the launch script here; a broken one is caught at runtime</span>
@@ -2387,6 +2392,11 @@ <h3 id="codeclash.arenas.battlesnake.battlesnake.BattleSnakeArena.validate_code"
             <span class="n">error_msg</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;There should be a `</span><span class="si">{</span><span class="n">func</span><span class="si">}</span><span class="s2">` function implemented in `</span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">submission</span><span class="si">}</span><span class="s2">`&quot;</span><span class="p">)</span>
     <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">error_msg</span><span class="p">)</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">:</span>
         <span class="k">return</span> <span class="kc">False</span><span class="p">,</span> <span class="s2">&quot;</span><span class="se">\n</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">error_msg</span> <span class="o">+</span> <span class="p">[</span><span class="s2">&quot;Don&#39;t change the function signatures!&quot;</span><span class="p">])</span>
+    <span class="k">if</span> <span class="s2">&quot;__main__&quot;</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">bot_content</span><span class="p">:</span>
+        <span class="k">return</span> <span class="kc">False</span><span class="p">,</span> <span class="p">(</span>
+            <span class="sa">f</span><span class="s1">&#39;`</span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">submission</span><span class="si">}</span><span class="s1">` must keep its `if __name__ == &quot;__main__&quot;` block that starts &#39;</span>
+            <span class="s2">&quot;the server, or the bot fails to launch.&quot;</span>
+        <span class="p">)</span>
     <span class="k">return</span> <span class="kc">True</span><span class="p">,</span> <span class="kc">None</span>
 </code></pre></div></td></tr></table></div>
             </details>

reference/arenas/cyborg/index.html

@@ -1945,12 +1945,7 @@ <h2 id="codeclash.arenas.cyborg.cyborg.CybORGArena" class="doc doc-heading">
 
                   <details class="mkdocstrings-source">
                     <summary>Source code in <code>codeclash/arenas/arena.py</code></summary>
-                    <div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal"> 78</span>
-<span class="normal"> 79</span>
-<span class="normal"> 80</span>
-<span class="normal"> 81</span>
-<span class="normal"> 82</span>
-<span class="normal"> 83</span>
+                    <div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal"> 83</span>
 <span class="normal"> 84</span>
 <span class="normal"> 85</span>
 <span class="normal"> 86</span>
@@ -1975,7 +1970,12 @@ <h2 id="codeclash.arenas.cyborg.cyborg.CybORGArena" class="doc doc-heading">
 <span class="normal">105</span>
 <span class="normal">106</span>
 <span class="normal">107</span>
-<span class="normal">108</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">config</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="o">*</span><span class="p">,</span> <span class="n">tournament_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">local_output_dir</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">keep_containers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">):</span>
+<span class="normal">108</span>
+<span class="normal">109</span>
+<span class="normal">110</span>
+<span class="normal">111</span>
+<span class="normal">112</span>
+<span class="normal">113</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">config</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="o">*</span><span class="p">,</span> <span class="n">tournament_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">local_output_dir</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">keep_containers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">):</span>
 <span class="w">    </span><span class="sd">&quot;&quot;&quot;The CodeArena class is responsible for running games, i.e., taking a list of code</span>
 <span class="sd">    from different agents/players and running them against each other.</span>
 <span class="sd">    It also provides the environments for the game and agents to run in.</span>

reference/arenas/dummy/index.html

@@ -1845,12 +1845,7 @@ <h2 id="codeclash.arenas.dummy.dummy.DummyArena" class="doc doc-heading">
 
                   <details class="mkdocstrings-source">
                     <summary>Source code in <code>codeclash/arenas/arena.py</code></summary>
-                    <div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal"> 78</span>
-<span class="normal"> 79</span>
-<span class="normal"> 80</span>
-<span class="normal"> 81</span>
-<span class="normal"> 82</span>
-<span class="normal"> 83</span>
+                    <div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal"> 83</span>
 <span class="normal"> 84</span>
 <span class="normal"> 85</span>
 <span class="normal"> 86</span>
@@ -1875,7 +1870,12 @@ <h2 id="codeclash.arenas.dummy.dummy.DummyArena" class="doc doc-heading">
 <span class="normal">105</span>
 <span class="normal">106</span>
 <span class="normal">107</span>
-<span class="normal">108</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">config</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="o">*</span><span class="p">,</span> <span class="n">tournament_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">local_output_dir</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">keep_containers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">):</span>
+<span class="normal">108</span>
+<span class="normal">109</span>
+<span class="normal">110</span>
+<span class="normal">111</span>
+<span class="normal">112</span>
+<span class="normal">113</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">config</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="o">*</span><span class="p">,</span> <span class="n">tournament_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">local_output_dir</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">keep_containers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">):</span>
 <span class="w">    </span><span class="sd">&quot;&quot;&quot;The CodeArena class is responsible for running games, i.e., taking a list of code</span>
 <span class="sd">    from different agents/players and running them against each other.</span>
 <span class="sd">    It also provides the environments for the game and agents to run in.</span>