Skip to content

Commit 529ed55

Browse files
hyperpolymathclaude
andcommitted
feat(extract): MiniZinc full-share + gitignore TPTP output
MiniZinc extractor pivoted from round-robin to full-share across the 5-solver constraint fleet (MiniZinc / Chuffed / OR-Tools / SCIP / GLPK). Every MiniZinc model is solvable by every solver in our fleet, so emitting one record per (model, solver) pair is legitimate training data — quintuples per-solver coverage without new corpus work. Result: 307 records per solver (1 535 total). Also: - .gitignore: training_data/proof_states_tptp.a2ml (now 336 MB from the full-share pivot; exceeds GitHub's 100 MB per-file limit). Reproducible via scripts/extract_tptp.jl + external_corpora/tptp/. - .gitignore: training_data/proof_states_key.jsonl (>100 MB, same rationale). - Refreshed output jsonls for 14 provers that had drifted from their current extractors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 5727013 commit 529ed55

21 files changed

Lines changed: 863929 additions & 204342 deletions

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,10 @@ training_data/vocabulary_*.txt
147147
training_data/proof_states_UNIFIED.jsonl
148148
training_data/proof_states_smtlib.jsonl
149149
training_data/tactics_smtlib.jsonl
150+
# TPTP full-share output exceeds 100MB (reproducible via scripts/extract_tptp.jl)
151+
training_data/proof_states_tptp.a2ml
152+
# KeY output exceeds 100MB (reproducible via scripts/extract_key.jl)
153+
training_data/proof_states_key.jsonl
150154

151155
# Root-level merged artifacts
152156
premises_merged.jsonl

scripts/extract_minizinc.jl

Lines changed: 39 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -378,25 +378,49 @@ function run()::Tuple{Int,Int}
378378
end
379379
println(" Generated $(added) unique synthetic models")
380380

381-
# Assign IDs and normalize schema
382-
# Distribute across solvers for downloaded models
381+
# Full-share (2026-04-18): every MiniZinc model is solvable by
382+
# every constraint solver in the fleet. Emit one record per
383+
# (model, solver) pair — 5× the per-solver coverage without any
384+
# new data. Synthetic entries retain their specific `solver`
385+
# assignment so sub-family statistics stay accurate.
383386
solvers_cycle = ["MiniZinc", "Chuffed", "ORTools", "SCIP", "GLPK"]
384387
current_id = START_ID
385388
output_records = Dict{String,Any}[]
386389

387-
for (i, entry) in enumerate(all_entries)
388-
solver = get(entry, "solver", solvers_cycle[((i - 1) % length(solvers_cycle)) + 1])
389-
record = Dict{String,Any}(
390-
"id" => current_id,
391-
"prover" => solver,
392-
"theorem" => entry["theorem"],
393-
"goal" => entry["goal"],
394-
"context" => get(entry, "constraints", get(entry, "variables", String[])),
395-
"tactic_proof" => JSON3.write(Dict("variables" => get(entry, "variables", String[]), "constraints" => get(entry, "constraints", String[]))),
396-
"source" => get(entry, "source", "minizinc"),
397-
)
398-
push!(output_records, record)
399-
current_id += 1
390+
for entry in all_entries
391+
if haskey(entry, "solver")
392+
# Synthetic entry — keep single-solver assignment.
393+
record = Dict{String,Any}(
394+
"id" => current_id,
395+
"prover" => entry["solver"],
396+
"theorem" => entry["theorem"],
397+
"goal" => entry["goal"],
398+
"context" => get(entry, "constraints", get(entry, "variables", String[])),
399+
"tactic_proof" => JSON3.write(Dict(
400+
"variables" => get(entry, "variables", String[]),
401+
"constraints" => get(entry, "constraints", String[]))),
402+
"source" => get(entry, "source", "minizinc"),
403+
)
404+
push!(output_records, record)
405+
current_id += 1
406+
else
407+
# Real benchmark — emit for every solver.
408+
for solver in solvers_cycle
409+
record = Dict{String,Any}(
410+
"id" => current_id,
411+
"prover" => solver,
412+
"theorem" => entry["theorem"],
413+
"goal" => entry["goal"],
414+
"context" => get(entry, "constraints", get(entry, "variables", String[])),
415+
"tactic_proof" => JSON3.write(Dict(
416+
"variables" => get(entry, "variables", String[]),
417+
"constraints" => get(entry, "constraints", String[]))),
418+
"source" => get(entry, "source", "minizinc"),
419+
)
420+
push!(output_records, record)
421+
current_id += 1
422+
end
423+
end
400424
end
401425

402426
open(OUTPUT_FILE, "w") do fh

training_data/proof_states_acl2.jsonl

Lines changed: 170455 additions & 277 deletions
Large diffs are not rendered by default.

training_data/proof_states_agda.jsonl

Lines changed: 17002 additions & 1 deletion
Large diffs are not rendered by default.

training_data/proof_states_dafny.jsonl

Lines changed: 13399 additions & 48 deletions
Large diffs are not rendered by default.

training_data/proof_states_fstar.jsonl

Lines changed: 3277 additions & 76 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)