[feat] SID: add ResidualQuantizer / BaseSidModel base classes

[WhiteSwan1]

First of three PRs splitting the Semantic-ID models onto a shared base. This one is purely additive — only the backend-agnostic foundation, with no concrete quantizer or model and no edits to existing files. RQ-KMeans follows in PR2, RQ-VAE in PR3.

What this adds:

• ResidualQuantizer (abstract): owns the state every residual quantizer shares — embed_dim, per-layer codebook sizes (normalize_n_embed), residual-normalization flag, and the layer list — and the shared residual walk. _residual_pass drives the concrete get_codes / decode_codes / output_dim. Subclasses implement just the two per-layer primitives — _quantize_layer (encode) and _lookup_code (decode) — plus forward and get_codebook_embeddings.
• BaseSidModel (abstract): the shared SID model scaffold — embedding-feature extraction, loss/metric initialization, and shared config parsing — that SidRqkmeans and SidRqvae will subclass.
• types.py: QuantizeForwardMode, QuantizeOutput, ResidualQuantizerOutput.

Tests: normalize_n_embed, the abstract-base contract (shared state present, backend primitives raise).

No proto changes and no edits to existing modules — the concrete SidRqkmeans / SidRqvae messages, and the BaseModel.on_train_end lifecycle hook, arrive with their models in PR2 / PR3.

[github-actions]

Review summary

Clean, well-scoped additive PR — the abstract ResidualQuantizer / BaseSidModel split is sensible, docstrings are thorough and accurate, and update_train_metric correctly wires into the real call site (main.py:438). The detach/accumulate semantics of _residual_pass match the docstring. Nothing blocking.

A few items, left inline:

• decode_codes hardcodes dtype=torch.float — pins reconstruction to fp32 and silently upcasts under mixed precision; seed the accumulator from the first lookup so dtype/device follow the codebook.
• _update_unique_sid_ratio divides by B without an empty-batch guard (ZeroDivisionError).
• Test coverage — only the NotImplementedError paths are exercised; the core residual-walk math (get_codes/decode_codes, the detach invariant, the normalize_residuals=True branch) is untested. A tiny fake subclass would cover it.

Minor polish (no need to block):

• update_train_metric(predictions: dict, ...) — every sibling/base uses Dict[str, torch.Tensor].
• # noqa: ANN201 on the abstract forward is a no-op (ANN is not in .ruff.toml lint.select); siblings annotate their forward return.

[github-actions]

Code review summary

Reviewed across code-quality, performance, tests, docs, and security. This is clean, well-documented foundation code for the SID model split — the abstraction boundary (_quantize_layer/_lookup_code primitives vs. the shared _residual_pass walk) is cleanly drawn, the detach/grad semantics are correct and tested, and the decode_codes dtype/device-aware seeding and B == 0 empty-shard guard show good edge-case awareness. No blocking issues.

A few minor, optional items posted inline:

• Tests — test_normalize_residuals_branch only asserts shapes, so it wouldn't catch the normalization branch being removed; and _update_unique_sid_ratio (incl. its empty-shard guard) is pure-tensor logic that's untestable-free and worth a small unit test now.
• Polish — predictions: dict → Dict[str, torch.Tensor] for consistency with BaseModel; an optional assert n_layers >= 1 in ResidualQuantizer.__init__ to self-guard the construction boundary for the upcoming PR2/PR3 subclasses.

Docs were verified accurate (the :class: forward-refs to not-yet-existing subclasses don't break the Sphinx build). LGTM as the additive foundation layer.

_{🤖 Generated with Claude Code}

[github-actions]

Review summary

Clean, genuinely additive PR — backend-agnostic SID base classes only, no edits to existing files. The abstraction is well-factored: ResidualQuantizer isolates the shared residual walk from the two backend primitives (_quantize_layer / _lookup_code), and BaseSidModel centralizes config parsing and shared metrics. Docstrings are thorough (the decode_codes dtype/device seeding and the detach invariant are nicely documented), and the _FakeQuantizer-based tests exercise the concrete base logic without pulling in real backends. The _residual_pass / decode_codes paths are AMP-safe with no spurious host-device syncs.

Two inline comments on items worth confirming:

• sid_model.py — base registers mse/unique_sid_ratio in init_metric but provides no shared update_metric to populate them; docstring says subclasses "extend update_metric" but there's nothing to extend.
• unique_ratio.py — metric is a mean of per-batch ratios (batch-size sensitive), not global codebook coverage; worth confirming the intended semantic.

Minor / non-blocking notes:

• init_metric hardcodes the metric set rather than reading proto metrics config like other model families. Fine if SID metrics are intentionally fixed — worth a one-line note so it isn't later mistaken for a bug.
• The :class: cross-refs to SidRqvae / SidRqkmeans / ResidualVectorQuantizer / ResidualKMeansQuantizer (not yet in the tree) may emit Sphinx "reference target not found" warnings until PR2/PR3 land — expected for the split, just flagging in case the doc build is strict.
• A couple of low-priority test gaps: the n_layers >= 1 assertion and the temperature forwarding path are untested. The absence of a BaseSidModel test is justified (its proto fields don't exist yet); _extract_feature / init_metric could be unit-tested now with a stub if you want coverage before the subclass PR.

Nice work — no blocking issues.