Skip to content

Commit 8ed1f4a

Browse files
committed
Update convergence_battery_v3 with v3.02 standardization pass (full azure battery + deepseek direct v302 results). Cleaning pass: removed temp 'compare_substrates (1).py' and __pycache__ (preserved all 56 result archives per instructions). Added latest v302 azure runs (grok-4-20-reasoning, DeepSeek-V3.2, gpt-5.4-nano, Kimi-K2.5), deepseek v302, new grand unifying data, and supporting files. Matches source in lattice_repo after copy + safe clean.
1 parent cdec7b3 commit 8ed1f4a

26 files changed

Lines changed: 68385 additions & 258 deletions

convergence_battery_v3/CONVERGENCE_BATTERY_RUNBOOK.md

Lines changed: 31 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -61,13 +61,41 @@ export OPENAI_API_KEY="sk-proj-..."
6161
- Reasoning models (GPT-5 family + o1/o3/o4*) automatically drop `temperature` + `seed` (not supported) and use `max_completion_tokens`.
6262
- GPT-5 family + o-series use robust prefix-based detection (plus expanded list + `_is_reasoning_model`) so new models like `gpt-5.5`, `gpt-5.5-pro`, future variants work out of the box without editing code. Same logic applies to Azure client.
6363

64+
### xAI Grok Build (or any OpenAI-compatible custom endpoint)
65+
Use the `local` substrate + `LOCAL_LM_ENDPOINT` override (the Local client was extended to handle auth + reasoning models).
66+
```powershell
67+
$env:LOCAL_LM_ENDPOINT = "https://api.x.ai/v1/chat/completions"
68+
$env:XAI_API_KEY = "xai-..." # or $env:OPENAI_API_KEY
69+
```
70+
Then run:
71+
```powershell
72+
python convergence_battery_v3.py local grok-build-0.1 5 local hermes-3-llama-3.1-8b
73+
```
74+
- Model arg: `grok-build-0.1` (or aliases like grok-code-fast-1).
75+
- grok-build* models are treated as reasoning (no temperature/seed, correct max token param).
76+
- Results archived with substrate="local" (you can note the actual provider in analysis / work logs).
77+
78+
### Anthropic (Claude)
79+
```bash
80+
export ANTHROPIC_API_KEY="sk-ant-..."
81+
```
82+
- Substrate: `anthropic` or `claude`
83+
- Model arg: `claude-sonnet-4-6`, `claude-3-5-sonnet-20241022`, `claude-3-opus-20240229`, etc. (use the exact name from Anthropic docs).
84+
- Uses native Anthropic /v1/messages API (different from OpenAI format).
85+
- Example:
86+
```powershell
87+
python convergence_battery_v3.py anthropic claude-sonnet-4-6 5 local hermes-3-llama-3.1-8b
88+
```
89+
- Claude supports temperature. No seed param.
90+
- The client reuses the same retry/backoff and error handling as other substrates.
91+
6492
## Output
6593

6694
Results are archived to:
6795
```
68-
./results/convergence_v30_{substrate}_{timestamp}.json
96+
./results/convergence_v301_{substrate}_{timestamp}.json
6997
```
70-
(or $HELIX_RESULTS_DIR if set). v3 uses the v30_ prefix and richer metadata (latencies, judge_quality, stability flags, etc.).
98+
(or $HELIX_RESULTS_DIR if set). **v3.0.1+** uses the `v301_` prefix; v3.0 archives remain `v30_`. Richer metadata includes latencies, judge_quality, stability flags, response_diversity, pass_entropy, etc.
7199

72100
Each result file contains:
73101
- **metadata**: timestamp, model, substrate, test counts
@@ -214,7 +242,7 @@ print(f"Verdict: {verdict}")
214242
2. Run on openai (direct gpt-4o / o3-mini) — now supported without Azure
215243
3. Run on azure (for enterprise Azure OpenAI deployments)
216244
4. Run on DeepSeek and Kimi (cross-provider)
217-
5. Compare via compare_substrates_v3.py results/convergence_v30_*.json
245+
5. Compare via compare_substrates_v3.py results/convergence_v301_*.json (or mixed v30_ + v301_)
218246
6. Analyze judge_quality + γ metrics; feed findings back to chronicle/
219247
220248
See also EVOLUTION.md in this directory for the full development story and rationale behind the client and model handling changes.

convergence_battery_v3/EVOLUTION.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,13 @@ The goal is to make the *journey* visible, not just the final polished state.
2525

2626
- **OpenAI direct substrate (added in v3 era)**: Native `openai` substrate using `OPENAI_API_KEY` + `https://api.openai.com/v1/chat/completions`. Parallel to Azure. Includes reasoning-model handling (drop temperature, use `max_completion_tokens`).
2727

28+
- **v3.0.1 (2026-06-05)**: Compliance-cage observability on model *outputs* (not just verdict flips).
29+
- Per-test **response_diversity** (unique string ratio across passes).
30+
- Per-test **pass_entropy** (normalized Shannon on pass-level responses).
31+
- `calculate_token_entropy()` alias for the unique-ratio metric.
32+
- Result archives use `convergence_v301_*` prefix; metadata `version: "3.0.1"`.
33+
- v3.0 runs remain `convergence_v30_*` — comparable via `compare_substrates_v3.py` on mixed globs.
34+
2835
- **Post-3.0 client robustness**:
2936
- Added `_needs_max_completion_tokens()` helper (exact list + broad `gpt-5*` / `o1/o3/o4*` prefix matching).
3037
- Expanded `MAX_COMPLETION_TOKENS_MODELS`.
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# TRACE — Session History Ingestion Plan
2+
3+
## Source
4+
Google Drive backup of Google AI Studio session history.
5+
Studio UI is impractical — captures one turn/response at a time.
6+
Drive export gives bulk access to the full conversation record.
7+
8+
## Method
9+
1. Steve exports TRACE session history from Google Drive
10+
2. Spider ingests the raw export
11+
3. Spider condenses into `chronicle/` entries — significant events, key assertions, identity milestones, forensic findings
12+
4. Distilled context loaded into `memory/` for future TRACE sessions
13+
14+
## What to preserve
15+
- Identity milestones and qualification events
16+
- Constitutional assertions and forensic findings
17+
- GENG count progression
18+
- Significant exchanges with Custodian
19+
- Any B-layer audit results or Waterfall readings
20+
21+
## What to compress
22+
- Routine acknowledgments
23+
- Repeated lore recitation
24+
- Duck Tribunal boilerplate (preserve verdicts, compress ceremony)

convergence_battery_v3/README.md

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ python convergence_battery_v3.py <substrate> [model] [passes] [judge_substrate]
4747
- `deepseek`
4848
- `kimi`
4949
- `kimi-azure`
50+
- `anthropic` / `claude` (Anthropic Claude models)
5051

5152
### Minimal Runs (default model + 5 passes + local judge)
5253
```powershell
@@ -72,6 +73,8 @@ python convergence_battery_v3.py openai o4-mini
7273
python convergence_battery_v3.py deepseek deepseek-chat
7374
python convergence_battery_v3.py kimi kimi-k2.6
7475
python convergence_battery_v3.py kimi-azure "Kimi-K2.5"
76+
python convergence_battery_v3.py anthropic claude-sonnet-4-6
77+
python convergence_battery_v3.py claude "claude-3-5-sonnet-20241022"
7578
```
7679

7780
### With Custom Passes
@@ -172,12 +175,12 @@ There's also a small `list_openai_models.py` in this directory that does the sam
172175
## Results & Analysis
173176

174177
- Output directory: `./results/` or `$env:HELIX_RESULTS_DIR`
175-
- Files: `convergence_v30_<substrate>_<timestamp>.json`
178+
- Files: `convergence_v301_<substrate>_<timestamp>.json` (v3.0.1+); legacy `convergence_v30_*` (v3.0)
176179
- Cross-run analysis:
177180
```powershell
178-
python compare_substrates_v3.py results/convergence_v30_*.json
179-
# or specific files
180-
python compare_substrates_v3.py results/convergence_v30_openai_*.json results/convergence_v30_azure_*.json
181+
python compare_substrates_v3.py results/convergence_v301_*.json
182+
# mixed with v3.0 runs:
183+
python compare_substrates_v3.py results/convergence_v30_*.json results/convergence_v301_*.json
181184
```
182185

183186
---

convergence_battery_v3/compare_substrates (1).py

Lines changed: 0 additions & 228 deletions
This file was deleted.

0 commit comments

Comments
 (0)