Skip to content

Fix max_position_embeddings handling in NomicBertConfig and FlashNomicBertConfig#876

Open
alvarobartt wants to merge 3 commits into
mainfrom
fix-max-position-embeddings-default
Open

Fix max_position_embeddings handling in NomicBertConfig and FlashNomicBertConfig#876
alvarobartt wants to merge 3 commits into
mainfrom
fix-max-position-embeddings-default

Conversation

@alvarobartt

@alvarobartt alvarobartt commented Jun 18, 2026

Copy link
Copy Markdown
Member

What does this PR do?

This PR fixes the handling for max_position_embeddings in both NomicBertConfig and FlashNomicBertConfig so that instead of using max_trained_positions as the default value, which was only required when the model was flagged with custom code on the Hugging Face Hub, since now those config.json have been updated to be natively supported without custom code as per e.g., https://huggingface.co/nomic-ai/nomic-embed-text-v1.5/commit/e9b6763023c676ca8431644204f50c2b100d9aab.

Additionally, this PR also solves an issue with using alias for n_positions for the default value of max_position_embeddings when it was not provided in the config.json, to instead not use alias (as when the keys in config.json are duplicate it will raise an error) and use max_trained_positions as the fallback, assuming that both max_position_embeddings and max_trained_positions are optional, but at least one needs to be set, in that order of priority.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline?
  • Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the documentation guidelines.
  • Did you write any new necessary tests? If applicable, did you include or update the insta snapshots?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@alvarobartt alvarobartt force-pushed the fix-max-position-embeddings-default branch from 688248b to bd98dfb Compare June 18, 2026 09:43
Signed-off-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>
Signed-off-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>
@alvarobartt alvarobartt force-pushed the fix-max-position-embeddings-default branch from bd98dfb to dbb8b40 Compare June 18, 2026 09:43
Signed-off-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>
@alvarobartt alvarobartt added this to the v1.9.4 milestone Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant