Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions dataset/igla-coder/v0.1/DECONTAM.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# DECONTAM.md — IGLA CODER v0.1

**Version:** 0.1
**Method:** Lee 2022 k=50 substring bidirectional check.
**Reference:** Lee et al., "Deduplicating Training Data Makes Language Models Better" (arXiv:2107.06499).

## Contract (mandatory when the dataset is used to train IGLA CODER/RACE)

For every training pair `(spec, gen)` and every held-out evaluation program `eval_i`:

1. **Forward direction:** no 50-character substring of `spec` or `gen` appears verbatim in any `eval_i` (case-sensitive, whitespace-normalised).
2. **Reverse direction:** no 50-character substring of any `eval_i` appears verbatim in any `spec` or `gen`.

**Failure of either direction** => the pair is quarantined (moved to `dataset/igla-coder/v0.1/quarantine/`), never enters `pairs/`.

## v0.1 status: SKIPPED — trivially clean

Held-out evaluation set at v0.1 is **empty**. The directory `held-out-eval/` exists but contains no programs. Therefore:

- Bidirectional decontam is **trivially clean** (there is nothing to compare against).
- `manifest.decontam_bidirectional = false` (honest: check was not meaningfully performed).
- Per-pair `metadata.json` marks `decontam_status = "Skipped"`.

**This is a known v0.1 gap and is intentional.** Populating the held-out set before running decontam properly is a v0.2 responsibility.

## v0.2 plan (out of scope this Wave)

1. Author 12-16 held-out programs (matching topic distribution of pairs but name-disjoint).
2. Store under `held-out-eval/<eval_id>/spec.tri` (and `gen.<lang>` where applicable).
3. Add `decontam_check.py` (deterministic, stdlib-only, k=50).
4. Re-run: assert every pair passes both directions. Fail-loud on any violation.
5. Publish `decontam_report_v02.md` alongside MANIFEST v0.2.

## Bibliographic honesty

- Lee 2022 (arXiv:2107.06499) establishes the k=50 threshold for near-duplicate detection in LM training corpora. Applying the same threshold bidirectionally to eval-vs-train is a stricter variant used by BIG-bench maintainers and OpenAI's HumanEval-style protocols.
- We do NOT claim originality on this method. We ship a placeholder because v0.1 is a **seed**, not a training corpus.

## Verdict

`DECONTAM v0.1 = SKIPPED (justified). NOT SAFE TO TRAIN on this dataset as-is; awaiting held-out eval.`
145 changes: 145 additions & 0 deletions dataset/igla-coder/v0.1/MANIFEST.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
{
"schema_version": 1,
"dataset_version": "0.1",
"built_at_iso": "2026-07-08T07:31:18.026643+00:00",
"pair_count": 8,
"decontam_bidirectional": false,
"held_out_eval_defined": false,
"note": "v0.1 seed: 8 spec-only pairs. Codegen pending TRAIN-BOX (rust/zig/ts). Decontam trivially clean because held-out-eval is empty at v0.1; v0.2 must populate held-out-eval FIRST.",
"law_compliance": {
"L1": "traceable via Closes #new-N in each ring PR body",
"L3": "ASCII+English verified per-file below",
"L4": ">=3 invariants + >=8 tests + >=2 bench per spec (visual audit)",
"L6": "no new numeric formats introduced; live catalog count = 83 verified via grep -c '// CATALOG:' specs/numeric/formats_catalog.t27"
},
"pairs": [
{
"pair_id": "0001-experience-format",
"spec_path_in_t27": "specs/organism/experience.tri",
"spec_local_path": "pairs/0001-experience-format/spec.tri",
"spec_sha256": "ea43af76d3c1214721cd40dbc624062cc17707525c55c824612e3941b3c3c7ef",
"spec_bytes": 4096,
"gen_path_in_t27": null,
"gen_sha256": null,
"target_lang": "None",
"ring_id": "ring-105-001",
"license": "Apache-2.0",
"decontam_status": "Skipped",
"codegen_status": "spec_only",
"l3_ascii_pass": true,
"l3_non_ascii_bytes": 0
},
{
"pair_id": "0002-mozg-state-machine",
"spec_path_in_t27": "specs/organism/mozg.tri",
"spec_local_path": "pairs/0002-mozg-state-machine/spec.tri",
"spec_sha256": "ace8235ead1fded0cdd3a2bad33fefcfddfa4547bff66bfd5c6c11d62f392dc6",
"spec_bytes": 3473,
"gen_path_in_t27": null,
"gen_sha256": null,
"target_lang": "Zig",
"ring_id": "ring-105-002",
"license": "Apache-2.0",
"decontam_status": "Skipped",
"codegen_status": "codegen_pending_train_box",
"l3_ascii_pass": true,
"l3_non_ascii_bytes": 0
},
{
"pair_id": "0003-dna-schema",
"spec_path_in_t27": "specs/organism/dna.tri",
"spec_local_path": "pairs/0003-dna-schema/spec.tri",
"spec_sha256": "e17879a9431831d03d101417be6eb36748e07112de95cedadac917742abcad37",
"spec_bytes": 3399,
"gen_path_in_t27": null,
"gen_sha256": null,
"target_lang": "Zig",
"ring_id": "ring-105-002",
"license": "Apache-2.0",
"decontam_status": "Skipped",
"codegen_status": "codegen_pending_train_box",
"l3_ascii_pass": true,
"l3_non_ascii_bytes": 0
},
{
"pair_id": "0004-git-orchestrator",
"spec_path_in_t27": "specs/git/orchestrator.tri",
"spec_local_path": "pairs/0004-git-orchestrator/spec.tri",
"spec_sha256": "0eebd8a9045fd16b1c3bd46fc1518d5c82dcf51cdc579006222c91fcfeb4e244",
"spec_bytes": 4263,
"gen_path_in_t27": null,
"gen_sha256": null,
"target_lang": "Zig",
"ring_id": "ring-105-003",
"license": "Apache-2.0",
"decontam_status": "Skipped",
"codegen_status": "codegen_pending_train_box",
"l3_ascii_pass": true,
"l3_non_ascii_bytes": 0
},
{
"pair_id": "0005-mcp-tool-registry",
"spec_path_in_t27": "specs/mcp/tool_registry.tri",
"spec_local_path": "pairs/0005-mcp-tool-registry/spec.tri",
"spec_sha256": "d00d00154073aa13a376b762702a36c50c4b5c2ea83ac1cd6afdfa5696c2872b",
"spec_bytes": 3959,
"gen_path_in_t27": null,
"gen_sha256": null,
"target_lang": "Rust",
"ring_id": "ring-105-004",
"license": "Apache-2.0",
"decontam_status": "Skipped",
"codegen_status": "codegen_stub_pending_wave2",
"l3_ascii_pass": true,
"l3_non_ascii_bytes": 0
},
{
"pair_id": "0006-scene-schema",
"spec_path_in_t27": "specs/scenes/scene_schema.tri",
"spec_local_path": "pairs/0006-scene-schema/spec.tri",
"spec_sha256": "88029f3274e76ae9112fe05dd672904f761d8e686c273e06719e8f6ffe1712d1",
"spec_bytes": 3843,
"gen_path_in_t27": null,
"gen_sha256": null,
"target_lang": "TypeScript",
"ring_id": "ring-105-005",
"license": "Apache-2.0-anonymised",
"decontam_status": "Skipped",
"codegen_status": "codegen_deferred_wave3",
"l3_ascii_pass": true,
"l3_non_ascii_bytes": 0
},
{
"pair_id": "0007-ring-runtime",
"spec_path_in_t27": "specs/organism/ring_runtime.tri",
"spec_local_path": "pairs/0007-ring-runtime/spec.tri",
"spec_sha256": "ba5421b4fb88095c3edb629415d12fb2ed6bc1a7392934f6426799d430ae2979",
"spec_bytes": 3848,
"gen_path_in_t27": null,
"gen_sha256": null,
"target_lang": "Rust",
"ring_id": "ring-105-006",
"license": "Apache-2.0-anonymised",
"decontam_status": "Skipped",
"codegen_status": "codegen_stub_pending_wave2",
"l3_ascii_pass": true,
"l3_non_ascii_bytes": 0
},
{
"pair_id": "0008-igla-manifest",
"spec_path_in_t27": "specs/dataset/igla_coder_manifest.tri",
"spec_local_path": "pairs/0008-igla-manifest/spec.tri",
"spec_sha256": "a09c423ceda7d99360a66e60c661feb39a35ddc53453b6aeb788bf325894219e",
"spec_bytes": 4366,
"gen_path_in_t27": null,
"gen_sha256": null,
"target_lang": "None",
"ring_id": "ring-105-007",
"license": "Apache-2.0",
"decontam_status": "Skipped",
"codegen_status": "spec_only",
"l3_ascii_pass": true,
"l3_non_ascii_bytes": 0
}
]
}
63 changes: 63 additions & 0 deletions dataset/igla-coder/v0.1/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# IGLA CODER Dataset v0.1 — Seed

**Status:** SEED (not a training corpus).
**Pair count:** 8.
**Total spec bytes:** ~31 KB.
**Estimated token count:** ~7-10K (bpe token equivalent, rough estimate; measured at v0.2 with a tokenizer spec).

## Purpose

First public seed of the IGLA CODER training corpus: parallel `(spec, gen)` pairs where `spec` is a `.tri` T27 specification and `gen` is the corresponding generated code (Zig / Rust / Verilog / TypeScript — the latter deferred to Wave-3).

## What is here (v0.1)

Eight `.tri` specs, one per pilot ring of the 2026-07-08 ecosystem Wave (`ring-105-001..007`):

| Pair ID | Spec | Target lang | Codegen status | Ring |
|---|---|---|---|---|
| 0001-experience-format | organism/experience.tri | None (spec-only) | n/a | 105-001 |
| 0002-mozg-state-machine | organism/mozg.tri | Zig | TRAIN-BOX pending | 105-002 |
| 0003-dna-schema | organism/dna.tri | Zig | TRAIN-BOX pending | 105-002 |
| 0004-git-orchestrator | git/orchestrator.tri | Zig | TRAIN-BOX pending | 105-003 |
| 0005-mcp-tool-registry | mcp/tool_registry.tri | Rust | STUB (Wave-2) | 105-004 |
| 0006-scene-schema | scenes/scene_schema.tri | TypeScript | DEFERRED (Wave-3) | 105-005 |
| 0007-ring-runtime | organism/ring_runtime.tri | Rust | STUB (Wave-2) | 105-006 |
| 0008-igla-manifest | dataset/igla_coder_manifest.tri | None (spec-only) | n/a | 105-007 |

## What is NOT here (honest gaps)

- **No generated code.** All pairs are spec-only or spec-with-codegen-stub. Wave-2 (rust codegen) and Wave-3 (typescript codegen + real train-box regeneration) fill this in.
- **No held-out evaluation set.** `held-out-eval/` is empty. See DECONTAM.md.
- **No tokenizer.** Byte-level fallback assumed; explicit tokenizer spec is Wave-4.
- **No BPB baseline.** Impossible without a real train run.
- **~7-10K tokens is 3+ orders below phi-1-small floor.** IGLA CODER v0.1 CANNOT train a working model. This is a seed, not a corpus. See report/FINAL_REPORT.md §W-18.

## What v0.2..v0.5 must add (roadmap)

- v0.2: populate held-out-eval, rerun decontam bidirectionally, publish decontam_report_v02.md.
- v0.3: apply WP-10 deterministic alpha-rename augmentation to reach ~20K train tokens.
- v0.4: add tokenizer spec `specs/tokenizer/tri_tokenizer.tri`.
- v0.5: reach >=200K train tokens (deficit <=262x per WP-10 analog); actual train-box run authorised.

## Files

- `MANIFEST.json` — canonical index (schema in `specs/dataset/igla_coder_manifest.tri`)
- `DECONTAM.md` — decontamination policy and v0.1 status
- `pairs/<pair_id>/spec.tri` — the .tri spec
- `pairs/<pair_id>/metadata.json` — per-pair metadata (sha256, ring_id, target_lang, license)
- `held-out-eval/` — v0.1: empty by design

## License

Each pair inherits its source repo's license, recorded per-pair in `metadata.json`.

- ring-105-001..004 + 007: Apache-2.0 (from gHashTag/t27)
- ring-105-005, 105-006: Apache-2.0 (anonymised abstractions of PRIVATE 999-multibots-* repos; NO secrets, NO PII, NO wallet, NO tokens in this dataset)

## SHA-256 reproducibility

All `spec_sha256` values in MANIFEST.json are computed with Python stdlib `hashlib.sha256(open(path,"rb").read())` at build time. Re-running `compute_manifest.py` in this workspace must yield identical hex digests.

## Not for training as-is

**Do not attempt to train IGLA CODER on this dataset.** It is a seed for architecture and pipeline validation, not for capability. Real training gates on Wave-5.
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"pair_id": "0001-experience-format",
"spec_path_in_t27": "specs/organism/experience.tri",
"spec_local_path": "pairs/0001-experience-format/spec.tri",
"spec_sha256": "ea43af76d3c1214721cd40dbc624062cc17707525c55c824612e3941b3c3c7ef",
"spec_bytes": 4096,
"gen_path_in_t27": null,
"gen_sha256": null,
"target_lang": "None",
"ring_id": "ring-105-001",
"license": "Apache-2.0",
"decontam_status": "Skipped",
"codegen_status": "spec_only",
"l3_ascii_pass": true,
"l3_non_ascii_bytes": 0
}
Loading
Loading