Skip to content

Genotypes: per-individual diploid germline (phased recombination + novel alleles) - #5

Merged
MuteJester merged 26 commits into
masterfrom
genotype-modeling
Jun 17, 2026
Merged

Genotypes: per-individual diploid germline (phased recombination + novel alleles)#5
MuteJester merged 26 commits into
masterfrom
genotype-modeling

Conversation

@MuteJester

Copy link
Copy Markdown
Owner

Summary

Adds first-class genotype support to GenAIRR: a per-individual diploid germline complement attached via Experiment.with_genotype(g). With a genotype attached, V(D)J recombination is haplotype-phased (V/D/J of each rearrangement drawn from one chromosome); with none attached the engine is byte-for-byte unchanged (sha256-pinned).

What's included

  • Engine (Rust): GeneId/GeneIndex; a diploid Genotype/Haplotype/GeneCopy model; a single SampleGenotypePass doing phased chromosome→gene→allele sampling with up-front haplotype viability, strict/permissive filter-then-fallback, copy-number dosage, and gene-usage aggregation; new trace addresses/values + replay validation; an intra-pass feasibility fix so productive_only J-sampling respects the V chosen earlier in the same pass.
  • Biological features: allele presence/absence, diploid zygosity, gene deletion (incl. hemizygous), copy number/duplication, and novel/private alleles (synthesized + validated for conserved anchor/stop-free frame, injected into an effective reference — no engine change).
  • Python/DSL: Genotype builder (from_dataconfig/permissive, homozygous/heterozygous/delete_gene/duplicate_gene/chromosome_weights/complete_from_reference/add_novel_allele/with_subject); Experiment.with_genotype with mutual-exclusion/receptor-revision/cartridge-hash guards; per-record subject_id/haplotype provenance; result.genotypes; to_table/to_tsv ground truth.
  • Docs: a dedicated, self-contained guide (site_docs/guides/genotype.md) — concept, engine mechanics, builder API (with a method table + D/J example), provenance, benchmarking workflow, supported loci/chains, limitations — plus a real-tool detection showcase: a planted genotype recovered by TIgGER (precision/recall 1.00, 52/52 genes) and IgDiscover (precision 1.00, recall 0.96), with a figure. All Python examples execute against the live API.

Validation

  • Rust: cargo test --all-features green (1245 lib + integration).
  • Python: full suite green (2356 passed, 0 failed); 42 genotype tests.
  • Backward-compat: no-genotype output sha256-identical to master.
  • mkdocs build --strict clean.
  • Four independent critical-review rounds; all findings fixed.

Deferred to follow-on PRs (documented as limitations)

Cohorts, population priors, external loaders (VDJbase/TIgGER/IgDiscover/partis), cartridge genotype plane, same-haplotype receptor revision.

Test plan

  • CI: full Rust + Python matrix green
  • Backward-compat digest unchanged

… one pass

Consolidates the planned SampleHaplotypePass + SampleGeneAllelePass into a
single pass. The runtime gives each pass a fresh per-pass trace delta, so a
later pass cannot read an earlier pass's choice from ctx.trace; drawing the
chromosome and all V/D/J alleles in one pass keeps the chromosome a local
variable. Emits AlleleSampleSupport compile facts for V/D/J so productive_only
feasibility still builds; records canonical sample_allele.{seg} + sample_gene/
sample_allele_in_slot for provenance and replay.
…ult slots pin

Two contract pins reacted to the genotype additions: the production-pass
parameter_signature completeness pin (SampleGenotypePass now encodes the
genotype identity so distinct genotypes get distinct plan signatures) and the
SimulationResult.__slots__ lockstep pin (now includes _genotypes).
…-pass feasibility

- Replay now consumes sample_gene -> sample_allele_in_slot(if multi) -> sample_allele
  in the same order live emits them (review #1: replay was broken).
- Pass threads strict: permissive viability/sampling is presence-based and never
  errors; strict uses feasibility and returns structured errors (review #2).
- Gene choice weight = usage * total copy dosage, so a duplicated gene recombines
  more often (review #5).
- feasibility.rs: prefer an assignment already present in sim regardless of pass
  index, so J feasibility respects the V chosen earlier in the same consolidated
  pass (review #3). No-op for the flat one-segment-per-pass path.
- Adds a Rust replay round-trip test.
…otype check

- push_genotype_recombine accepts pool-aligned allele weights and aggregates
  them to gene-level usage, so gene choice follows the cartridge allele-usage
  model instead of uniform (review #4). Lowering passes step.weights_{v,d,j}.
- compile-time presence check: reject a genotype with no complete haplotype
  (every chromosome missing a required segment) instead of panicking at run
  (review #2 cross-haplotype case).
- genotype identity tests assert truth_*_call (evidence v_call can be ambiguous).
- delete_gene(haplotype=0|1) on an unspecified gene now raises (was a
  hemizygosity footgun that also suppressed complete_from_reference) (#6).
- to_table reports hemizygous/deleted zygosity and per-haplotype
  allele:copies:weight detail + subject_id; to_tsv enriched (#7).
- with_genotype snapshots the builder so later edits can't desync
  result.genotypes from the compiled engine genotype (#8).
- reject with_genotype + expand_clones/clonal_lineage/clonal_repertoire at
  compile rather than silently dropping provenance (#9).
- chromosome_weights rejects NaN/inf (#10).
- rename complete_from_reference policy to 'homozygous_first_reference'
  (it uses the first cartridge allele, not a frequency-common one) (#11).
…ck, honest usage

Re-review round 2:
- Replay now validates the recorded trace against the genotype: the gene must
  be carried on the drawn chromosome, the canonical allele must be in that gene
  slot, and any within-slot record must agree with it. Tampered traces are
  rejected (was: silently accepted). Adds a Rust test corrupting sample_allele.v.
- Permissive sampling now mirrors SampleAllelePass: prefer feasibility-admissible
  haplotypes/genes/alleles, falling back to the unfiltered carried set only when
  NO admissible candidate exists (was: ignored feasibility whenever strict=False).
- Gene usage: documented that it's driven by typed reference_models.allele_usage
  (dormant for bundled configs without it); added a typed-cartridge regression
  test proving the aggregation is active + effective when allele_usage is set.
New site_docs/guides/genotype.md: what a genotype is (diploid, phased, presence/
absence, copy number, deletion), how recombination samples from it (chromosome
choice, V-D-J linkage, viability/productive, strict vs permissive), the builder
API, ground-truth/provenance (result.genotypes, subject_id/haplotype, to_table/
to_tsv), benchmarking workflow, and PR1 limitations. Worked showcase: a planted
diploid genotype recovered by TIgGER (precision/recall 1.00, 52/52) and
IgDiscover (precision 1.00, recall 0.96, deletions + heterozygosity correct),
with a figure. Adds the guide to mkdocs nav.
…pe deletion; valid duplicate alleles

Caught by executing the guide's Python blocks against the live API: the builder
snippet called delete_gene(haplotype=1) on an unspecified gene (now a guarded
error) and used a non-existent allele in duplicate_gene. All 4 Python examples
now run cleanly.
…tion, programmatic build, inspection (all executed against live API)
…injection)

add_novel_allele(name, base=, mutations=|sequence=) synthesizes a private allele
from a reference base (inheriting gene/anchor/functional/subregions), validates
it, and registers it so it can be placed like any allele. At compile(), a
genotype with novel alleles builds an EFFECTIVE refdata (base catalogue + injected
private alleles) — so they flow through sampling/assembly/AIRR as real pool
entries with no engine changes. to_table flags novel alleles per gene. Docs:
dedicated 'Novel / private alleles' section + discovery-benchmarking recipe;
removed from Limitations. Tests cover synthesis, validation, sampling+AIRR, and
no-novel backward compatibility.
…e identity, gapped projection

Addresses critic findings on the novel-allele slice:
- #1 functional validation: synthesized V/J coding sequence is checked for an
  intact conserved anchor codon (Cys/Trp|Phe) and stop-free coding frame; a
  broken variant is rejected unless allow_nonfunctional=True (then kept + marked
  non-functional). Closes the 'nonfunctional emitted as productive' gap.
- #2 gene identity: the novel allele's gene is taken from its NAME and must equal
  the base allele's gene; dropped the gene= override that left allele.gene stale.
- #3 anchor: inherited from base (correct for same-length/substitution-only
  variants — the conserved residue does not move) and validated to remain intact;
  no reliance on the unavailable _native anchor resolver.
- #4 name uniqueness enforced across all segments + novel set (prevents truth-table
  mislabeling).
- #5 to_tsv now emits the 'novel' column.
- #6 substitutions projected onto the gapped sequence (no stale gapped_seq).
- #7 mutation positions/bases type-checked with clean ValueErrors.
Tests expanded: stop/anchor rejection + allow_nonfunctional, gene-mismatch,
cross-segment collision, tsv export, type-check.
…lidation, docs reproducibility

Correctness:
- effective_dataconfig injects ONLY carried novel alleles (defined-but-unplaced
  novels no longer leak into the aligner reference / v_call). High.
- add_novel_allele tolerates missing/empty/mismatched gapped_seq (falls back to
  ungapped) instead of IndexError. Medium.
- delete_gene / duplicate_gene validate the haplotype argument (both/0/1) instead
  of silently no-op'ing or negative-indexing. Medium.

Docs:
- 'Reproduce it' now builds the EXACT 9-gene planted genotype behind the figure,
  writes reads.fasta (IgDiscover/partis) + germline + truth, and states the
  scoring + reported precision/recall. High.
- Added a method-signature table (segment/haplotype defaults) + a D/J example;
  a 'Supported loci and chains' note (VDJ vs VJ, BCR/TCR); 'at compile time'
  wording fix.
Tests: unplaced-novel non-leak, haplotype validation, missing-gapped-seq fallback.
@MuteJester
MuteJester merged commit a56db5f into master Jun 17, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant