Reference translation: sidecar emission + consumer-side translate/canonicalize (bundle v2)

[jack-champagne]

Reference translation — sidecar emission + consumer-side translate/canonicalize (bundle v2)

Productizes the validated bundle-v2 research pipeline (Piccolissimo
experiments/bundle-v2-tracer, legs 4-5) into a JSD product API: re-point a
private package image's cross-image references at a consumer's own rebuild of
its dependencies, using live reflection on both sides instead of the
research matcher's blob-archaeology.

Stacked on #5 (feat/identity-stamp). This branch (feat/reference-translation)
is cut from feat/identity-stamp and reuses its restamp internals
(stamp_identity!'s embedded-checksum + .ji-trailer-CRC rewrite,
dry_verify, verify_closure). Merge #5 first, then this. The diff shown
against master will include #5's commits until #5 lands.

What ships

Each pointer crossing into another loaded image is an inline 8-byte reloc word
(5<<61) | (depsidx<<40) | (byte_offset÷8) (staticdata.c:1204/2170 @ v1.12.6).
Refs into the sysimage (depsidx == 0, ~98.8%) are stable across depots and need
no translation; only refs into separately-precompiled pkgimage deps shift.

• emit_sidecar(images; …) → Sidecar (builder): parse every external reloc
  word, skip depsidx==0, and for each pkgimage-dep ref resolve the target to the
  live object in the builder's own loaded deps (jl_linkage_blobs base + offset)
  and emit a semantic RefDescriptor.
• translate!(image_copy, sidecar; …) → TranslationReport (consumer): resolve
  each descriptor against the consumer's rebuilt deps, compute the new offset,
  rewrite the word (integrity-checked), restamp the checksums.
• canonicalize!(mods…) → CanonicalizeReport (post-load): the leg5 type-hash
  repair — re-intern private method sigs and re-insert genuinely-broken dispatch
  entries. Health-gated + idempotent (see fix below).
• load_translated(image, sidecar; …): one-shot chain — translate! → header
  remap-to-loaded-deps → verify_closure/load_package_image → canonicalize!.
• write_sidecar / read_sidecar: persist a Sidecar (stdlib Serialization).

RefDescriptor kinds

A target is described as, in priority order: a named entity
(:module/:binding/:type/:typename/:function); a nearest-named :anchor

• deterministic field path; or — for anonymous objects with no build-stable
  path (their reaching path runs through build-volatile type-cache order, leg5
  RESULTS §6) — an order-independent content descriptor:

• :svec_content — an anonymous SimpleVector (format-spec / method-sig /
  type-cache svec: svec('e'), svec(Val{'f'}), svec(Bool,…)). Described by its
  per-element content (types + isbits/Symbol/String leaves, serialized). The
  consumer reconstructs the elements and locates the live in-blob svec whose content
  matches structurally (mutual subtyping for type elements — interning-
  independent, so non-interned Tuple types compare equal).
• :const_data — a const-region String, matched by scanning the consumer
  dep's const region for the object's byte image [len:8][bytes][NUL].
• :type_content — an anonymous Union / un-named UnionAll /
  parameterization, matched by structural type equality over the blob's type objects.
• :method — a Method object, resolved by (defining module, name, structural signature) via _methods_by_ftype.

rank/cohort pin the correct member when structurally-identical duplicates share
the blob (blob-offset order within the equal-content cohort). Every content
descriptor is self-checked at emit against its own round-tripped payload, so a
target that is not faithfully content-describable (e.g. a svec with free type
variables) fails cheaply at emit rather than mis-resolving on the consumer. If a
target matches none of the above, emit_sidecar errors loudly (builder-side failure
is cheap; a consumer-side one is not).

New types in types.jl: RefDescriptor (with payload/rank/cohort),
RefTarget, ImageSidecar, Sidecar, TranslationReport, CanonicalizeReport.
docs/src/api.md updated.

Results / validation (julia +1.12.6, frozen twin depots)

Altissimo (emit vs depot A → translate! vs depot B, compared to leg5's
proven maps_Altissimo new_off):

• emit describes 43/43 targets, zero emit failures (12 type, 10 binding, 10
  svec_content, 5 module, 2 function, 2 typename, 2 const_data);
• resolve-side agreement with leg5's proven offsets: 43/43 = 100%, incl. all 10
  anonymous svecs + 2 const strings;
• translate! end-to-end: 253 words checked, 18 rewritten, 235 unchanged, 43
  resolved, 0 failed, ok=true — matching leg5 exactly.

Piccolissimo (the closing gap generalized): emit describes 244/244 targets,
0 failures (adds :type_content ×2 and :method ×3); translate! rewrites
1012 words (= leg5), 0 failures; 243/243 resolvable public-dep targets agree
with leg5's ground truth (the single "difference" is a cohort of two structurally-
identical svec(ComplexF64, 1) — semantically equivalent; rank-1 pick).

Full production flow (stretch) — PASS. Emit ONE 43 KB sidecar for both privates
on depot A (11.8 s); on a pure consumer depot (private-free clone of depot B,
zero builder-depot dependency access at consume time), load_translated both
images purely from that sidecar (translate A 1.7 s / P 3.3 s), then the H-gate
smoke solve: Ipopt f1 = 0.9761, Altissimo-backend f1 = 0.9921 — bit-for-bit the
leg1/leg5 values. canonicalize re-inserted 36 broken dispatch entries (= leg5).

• Full JSD suite green on julia +1.12.6: 336 / 336 (was 293; +43 for the new
  content-descriptor tests). New test/test_translate.jl items (5) pure-Julia codec
  / structural-matcher / cohort-rank / const-scan / sidecar round-trip of the new
  kinds, and (6) a toolchain-gated round-trip on a Val/Char-parameterized image
  (which bakes anonymous method-sig svecs + a const String) — describe → resolve on
  its own live blob.

Notable fixes over the leg5 prototype

• Idempotent AND correct canonicalize!. The type-hash repair must rebuild a
  broken method sig's (non-interned) Tuple so its baked hash is recomputed in the
  consumer's nonce universe — but rebuilding every sig is non-idempotent (fresh
  pointer each run) and re-inserting healthy entries causes spurious ambiguities. Fix:
  gate the repair on brokenness detected by a fresh-hash dispatch probe
  (_methods_by_ftype on the reconstructed sig — probing with the method's own
  stale-hashed sig would falsely report health by matching the stale-stored entry).
  A repaired method becomes findable under its consumer-hash sig, so the next pass
  skips it → idempotent, and only the genuinely-broken 36 (Piccolissimo) are touched.
• Multi-private load chains. _remap_to_loaded! and translate!'s dep lookup now
  consult Base.loaded_precompiles (not just loaded_modules), so a downstream
  private (Piccolissimo) sees an upstream private (a translated Altissimo, loaded via
  load_package_image and registered only in loaded_precompiles) and remaps its
  header to Altissimo's new restamped build-id — avoiding a mixed-lineage
  closure-check failure.

🤖 Generated with Claude Code

https://claude.ai/code/session_01V9jVcSYDNXtbBGfcAEPwvh