exp: Fine tuning improves accuracy

[timodonnell]

Fine tuning improves accuracy

Question

Can we fine tune from protenix v1 checkpoint?

Hypothesis

Expect to see various loss metrics go down, and get better performance on foldbench from the fine tuned checkpoint

Background

Mostly, this is a sanity check that our training pipeline works

Also I suspect that some of worse performance we expect to see in #4 can be addressed by fine tuning. So hopefully this will give us a better checkpoint to base future experiments off of.

Approach

 - Fine tune on our standard dataset
 - Check that train and eval loss metrics generally go down-ish, or at least don't explode
 - Take the final checkpoint and save it to huggingface
 - Run foldbench on the final checkpoint, compare to existing folding models

-->

Compute estimate

TBD

Success criteria

We get a better checkpoint (in terms of foldbench) than just using protenix v1 weights directly

Baselines

See #4

Notes

Retrospective scope: this issue wraps an already-running fine-tune (v1-finetune-01, H100:8, 10K steps, launched 2026-04-22, W&B run usvcruzi). Notebook pulls its metrics rather than launching new work.

[timodonnell]

🤖 Results — fine-tune ran to step 3,250/10,000 (stopped early once val metrics had visibly converged), final checkpoint benched on FoldBench.

Pipeline validated end-to-end (preprocess → train → val → bench → HF publish). Loss + val metrics trended in the right direction throughout, no divergence, no NaNs; val sweep clean once the torch pin landed (gh#3).

FoldBench delta vs. exp4 Protenix-v1 baseline (sorted by Δ mean LDDT):

category	Δ LDDT	Δ DockQ
interface_protein_rna	+0.074	+0.035
monomer_rna	+0.024	—
monomer_dna	+0.023	—
interface_protein_protein	+0.016	+0.044
interface_antibody_antigen	+0.008	+0.028
interface_protein_peptide	-0.026	+0.044
interface_protein_dna	-0.079	-0.018
interface_protein_ligand	-0.082	—
monomer_protein	-0.102	—

Bottom line: mixed result, not a drop-in replacement for Protenix v1. Wins on nucleic acids + protein-protein interfaces; regressions on monomer_protein (-0.10 — significant) and protein-ligand. Classic uniform-sampling fine-tune failure mode: the model forgot some of what v1 did best while learning a little of what v1 did worst.

Artifacts:

• Notebook: experiments/exp5_v1-finetune-01/ (commit f239030)
• W&B run: usvcruzi
• Checkpoint: exp5_v1-finetune-01/final.pt on the helico-experiments HF Bucket
• Raw step checkpoints step_{250..3250}.pt on the helico-checkpoints Modal Volume if we want to bisect by val LDDT

Followup idea for a fresh experiment: category-weighted sampling (or two-stage schedule) to avoid the monomer_protein regression while keeping the nucleic-acid wins. Filing as a new issue would be cleaner than reopening this one.

Closing this as the question ("can we fine-tune from Protenix v1?") is answered: yes, the pipeline works, and we know what to fix next.