Skip to content

generate_hf_golden_logits: propagate trust_remote_code to tokenizer; add --hf-load-dtype=auto#3775

Merged
copybara-service[bot] merged 1 commit intomainfrom
agagik-logits-gen
Apr 29, 2026
Merged

generate_hf_golden_logits: propagate trust_remote_code to tokenizer; add --hf-load-dtype=auto#3775
copybara-service[bot] merged 1 commit intomainfrom
agagik-logits-gen

Conversation

@gagika
Copy link
Copy Markdown
Collaborator

@gagika gagika commented Apr 29, 2026

Description

  • Pass trust_remote_code to AutoTokenizer.from_pretrained (it was already accepted as a kwarg and forwarded to the
    model, but silently dropped for the tokenizer).
  • Add --hf-load-dtype=autodtype=None so per-tensor dtypes from safetensors are preserved (needed for
    mixed-precision checkpoints where a single forced dtype corrupts non-default tensors).

Tests

Used those for Kimi-k2.5 hf token generation.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@gagika gagika force-pushed the agagik-logits-gen branch from aa62950 to 3a85bcd Compare April 29, 2026 18:27
Copy link
Copy Markdown
Collaborator

@richjames0 richjames0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@copybara-service copybara-service Bot merged commit 82f11f8 into main Apr 29, 2026
46 checks passed
@copybara-service copybara-service Bot deleted the agagik-logits-gen branch April 29, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants