Merge pull request #125 from exaforge/codex/deep-design

Stabilize core pipeline and simulation throughput

Author: RandomOscillations

.gitignore

@@ -53,6 +53,8 @@ Thumbs.db
 
 austin/
 examples/
+# Keep ad-hoc analysis dumps out of git even if examples are partially unignored later.
+examples/**/analysis/
 
 # Simulation artifacts
 *.sqlite

AGENTS.md

@@ -18,6 +18,7 @@ extropy query summary --json
 ```
 
 **Agent mode behavior:**
+
 - All output is JSON (parseable, no ANSI formatting)
 - Exit codes indicate success/failure (0 = success, non-zero = error)
 - No interactive prompts — commands fail fast if missing required input
@@ -29,20 +30,22 @@ extropy query summary --json
 extropy spec → extropy scenario → extropy persona → extropy sample → extropy network → extropy simulate → extropy results
 ```
 
-| Command | Purpose |
-|---------|---------|
-| `spec` | Create population spec from description |
-| `scenario` | Create scenario with events and outcomes |
-| `persona` | Generate persona rendering config |
-| `sample` | Sample agents from merged spec |
-| `network` | Generate social network |
-| `simulate` | Run simulation |
-| `results` | View results (summary, timeline, segment, agent) |
-| `query` | Export raw data (agents, edges, states, SQL) |
-| `chat` | Chat with simulated agents |
-| `estimate` | Predict simulation cost |
-| `validate` | Validate spec files |
-| `config` | View/set configuration |
+
+| Command    | Purpose                                          |
+| ---------- | ------------------------------------------------ |
+| `spec`     | Create population spec from description          |
+| `scenario` | Create scenario with events and outcomes         |
+| `persona`  | Generate persona rendering config                |
+| `sample`   | Sample agents from merged spec                   |
+| `network`  | Generate social network                          |
+| `simulate` | Run simulation                                   |
+| `results`  | View results (summary, timeline, segment, agent) |
+| `query`    | Export raw data (agents, edges, states, SQL)     |
+| `chat`     | Chat with simulated agents                       |
+| `estimate` | Predict simulation cost                          |
+| `validate` | Validate spec files                              |
+| `config`   | View/set configuration                           |
+
 
 ## Non-Interactive Usage
 
@@ -71,17 +74,19 @@ extropy persona -s ai-adoption -y
 
 ## Exit Codes
 
-| Code | Meaning |
-|------|---------|
-| 0 | Success |
-| 1 | General error / validation failure |
-| 2 | Clarification needed (agent mode) |
-| 3 | File not found |
-| 4 | Sampling error |
-| 5 | Network error |
-| 6 | Simulation error |
-| 7 | Scenario error |
-| 10 | User cancelled |
+
+| Code | Meaning                            |
+| ---- | ---------------------------------- |
+| 0    | Success                            |
+| 1    | General error / validation failure |
+| 2    | Clarification needed (agent mode)  |
+| 3    | File not found                     |
+| 4    | Sampling error                     |
+| 5    | Network error                      |
+| 6    | Simulation error                   |
+| 7    | Scenario error                     |
+| 10   | User cancelled                     |
+
 
 ## Querying Data
 
@@ -150,11 +155,13 @@ export AZURE_ENDPOINT=https://<resource>.services.ai.azure.com/
 
 All commands accept:
 
-| Flag | Purpose |
-|------|---------|
-| `--json` | JSON output (overrides cli.mode) |
-| `--cost` | Show cost summary after command |
-| `--study PATH` | Explicit study folder path |
+
+| Flag           | Purpose                          |
+| -------------- | -------------------------------- |
+| `--json`       | JSON output (overrides cli.mode) |
+| `--cost`       | Show cost summary after command  |
+| `--study PATH` | Explicit study folder path       |
+
 
 ## Study Folder Structure
 
@@ -197,3 +204,4 @@ extropy results --json
 extropy query agents --to agents.jsonl
 extropy query states --to states.jsonl
 ```
+

README.md

@@ -31,6 +31,7 @@ Extropy creates synthetic populations grounded in real-world distributions, conn
 ```bash
 pip install extropy-run
 export OPENAI_API_KEY=sk-...
+# or: ANTHROPIC_API_KEY=... / AZURE_OPENAI_API_KEY=...
 ```
 
 Requires Python 3.11+. [uv](https://github.com/astral-sh/uv) recommended.
@@ -61,7 +62,7 @@ extropy results segment income
 ## How It Works
 
 1. **Population** — LLM discovers attributes, researches real-world distributions, samples agents
-2. **Network** — Connects agents by similarity; edge types affect information flow
+2. **Network** — Builds structural + similarity edges; edge types affect information flow
 3. **Two-pass reasoning** — Agent role-plays reaction, then classifier extracts outcomes
 4. **Propagation** — Opinions spread through network; agents update after hearing from peers
 
@@ -79,9 +80,9 @@ extropy results segment income
 | Similarity edges | Acquaintances and online contacts from attribute similarity |
 | Small-world | Calibrated clustering coefficient and path lengths |
 | **Simulation** | |
-| Two-pass reasoning | Role-play first, classify second — eliminates central tendency bias |
+| Two-pass reasoning | Role-play first, classify second — reduces schema anchoring in outcome extraction |
 | Conversations | Agents talk to each other; both update state independently |
-| Memory | Full reasoning history with emotional trajectory |
+| Memory | Persistent memory traces with fidelity-based prompt trimming |
 | Conviction | Affects sharing probability and flip resistance |
 | THINK vs SAY | Internal monologue separate from public statement |
 | Timeline events | New information injected at specified timesteps |

docs/architecture.md

@@ -10,6 +10,28 @@ extropy spec → extropy scenario → extropy persona → extropy sample → ext
 
 All commands operate within a **study folder** — a directory containing `study.db` and scenario subdirectories. Data is keyed by `scenario_id` rather than `population_id`.
 
+## Reference Study Blueprints
+
+These are the canonical study patterns consolidated from prior showcase and study docs.
+
+| Study Type | Population | Scenario Mode | Horizon | Typical Scale |
+|------------|------------|---------------|---------|---------------|
+| ASI announcement + societal transition | National population | Evolving | Monthly, 6 steps | 5,000 agents |
+| Geopolitical shock (for example Iran strikes) | National population | Evolving | Weekly, 12 steps | 5,000 agents |
+| Asset/market mania shock (for example BTC extreme rally) | National population | Evolving | Weekly, 8-12 steps | 5,000 agents |
+| Election projection (house control / state race) | National or state electorate | Static or lightly evolving | Decision-focused | 1,000-5,000 agents |
+
+## Operational Quality Gates
+
+Before moving forward to the next stage, the architecture assumes these checks:
+
+1. `spec` gate: distribution realism and dependency coherence.
+2. `scenario` gate: timeline/outcome consistency and no contradictory rules.
+3. `persona` gate: natural language quality and no contradictory rendering.
+4. `sample` gate: no impossible household or demographic combinations.
+5. `network` gate: topology pass with realistic structural edge mix.
+6. `simulate` gate: expected timestep dynamics, checkpoint integrity, and tracked outputs.
+
 ---
 
 ## Phase 1: Population Creation (`extropy/population/`)
@@ -52,7 +74,7 @@ When `household_mode: true`:
 - Generate correlated partners (shared attributes: location, income correlation)
 - Generate NPC dependents (children, elderly) based on household type
 - Household types: singles, couples, single parents, couples with kids, multi-generational
-- `agent_focus` controls who reasons: `primary`, `couples`, `families`, `all`
+- `agent_focus_mode` controls who reasons: `primary_only`, `couples`, `all`
 
 ### 7. Network Generation (`network/`)
 
@@ -353,8 +375,8 @@ Resolution order: programmatic > env vars > config file > defaults
 |-------|---------|-------------|
 | `fast` | `= models.fast` | Fast model for Pass 2 |
 | `strong` | `= models.strong` | Strong model for Pass 1 |
-| `max_concurrent` | 50 | Max concurrent LLM calls |
-| `rate_tier` | 1 | Provider rate limit tier |
+| `max_concurrent` | `null` (auto from RPM) | Max concurrent LLM calls |
+| `rate_tier` | `null` | Provider rate limit tier |
 
 ### Providers