Skip to content

Update NDD (nemo data-designer) version#1755

Merged
ayushdg merged 9 commits intomainfrom
huvu/ndd_version_update
Apr 22, 2026
Merged

Update NDD (nemo data-designer) version#1755
ayushdg merged 9 commits intomainfrom
huvu/ndd_version_update

Conversation

@huvunvidia
Copy link
Copy Markdown
Contributor

@huvunvidia huvunvidia commented Apr 7, 2026

Description

Usage

# Add snippet demonstrating usage

Checklist

  • I am familiar with the Contributing Guide.
  • New or Existing tests cover these changes.
  • The documentation is up to date with these changes.

@huvunvidia huvunvidia requested a review from a team as a code owner April 7, 2026 17:53
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Apr 7, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@huvunvidia
Copy link
Copy Markdown
Contributor Author

huvunvidia commented Apr 7, 2026

/ok to test 8e2d78e

@huvunvidia
Copy link
Copy Markdown
Contributor Author

huvunvidia commented Apr 7, 2026

/ok to test 74f79e5

ayushdg
ayushdg requested changes Apr 17, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@ayushdg ayushdg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to fix failing CI checks and update a few comments. but generally looks good. @huvunvidia Have you verified that the pipelines work as expected with the new version?

Comment thread pyproject.toml Outdated
"transformers>=4.56.0,<5.0", # tranfomers>5 breaks with hf_hub<1.0 added in override
]
override-dependencies = [
"fsspec>=2025.3.0", # data-designer-engine==0.5.5 requires a newer fsspec than older pins allow
Copy link
Copy Markdown
Contributor

@ayushdg ayushdg Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you share which dependencies require this override/cause conflict

Copy link
Copy Markdown
Contributor

@ayushdg ayushdg Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick update the override is needed for nemo-toolkit. We should be good to override, can you update the comment that this is set because nemo-toolkit[asr] pins to an older version (you need to check which one).

Copy link
Copy Markdown
Contributor Author

@huvunvidia huvunvidia Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@ayushdg
Copy link
Copy Markdown
Contributor

ayushdg commented Apr 20, 2026

/ok to test 75fa8e5

@huvunvidia
Copy link
Copy Markdown
Contributor Author

huvunvidia commented Apr 20, 2026

/ok to test c7bbbb5

@huvunvidia
Copy link
Copy Markdown
Contributor Author

huvunvidia commented Apr 21, 2026

/ok to test 3c3f55b

@huvunvidia
Copy link
Copy Markdown
Contributor Author

huvunvidia commented Apr 21, 2026

/ok to test b2b5de5

@huvunvidia
Copy link
Copy Markdown
Contributor Author

huvunvidia commented Apr 21, 2026

/ok to test 9b3b370

@huvunvidia
Copy link
Copy Markdown
Contributor Author

huvunvidia commented Apr 21, 2026

/ok to test 6f8cc0b

ayushdg
ayushdg reviewed Apr 21, 2026
View reviewed changes
Comment thread pyproject.toml
ayushdg
ayushdg reviewed Apr 21, 2026
View reviewed changes
Comment thread pyproject.toml Outdated
"protobuf>=5.29.5,<7.0", # Override nemo-toolkits constraint of ~=5.29.5; <7.0 due to ray serve FieldDescriptor API breakage
"setuptools>=80.10.1", # Override setuptools range in other dependencies to address CVE GHSA-58pv-8j8x-9vj2
"nixl-cu12>=0.10.0; (platform_machine == 'x86_64' and platform_system != 'Darwin')", # Override ray[llm]'s unconditional nixl dep for ARM
"vllm>=0.14.1,<0.16.0; (platform_machine == 'x86_64' and platform_system != 'Darwin')", # ray[llm]==2.54 incompatible with vllm>=0.16.0 (renamed internal modules break LLMServer deployment)
Copy link
Copy Markdown
Contributor

@ayushdg ayushdg Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of adding this to overrides we should just fix the upper bound for vllm where it's defined originally.

Huy Vu2 and others added 7 commits April 21, 2026 15:34
@ayushdg
update ndd version
03bd324 
Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
@huvunvidia @greptile-apps @ayushdg
Update pyproject.toml
6f725f7 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Signed-off-by: Huy Vu <86480512+huvunvidia@users.noreply.github.com>
Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
@claude @ayushdg
Update fsspec override comment to clarify nemo-toolkit[asr] conflict
9615847 
nemo-toolkit[asr]==2.7.2 pins fsspec==2024.12.0, which conflicts with
data-designer-engine==0.5.5 requiring fsspec>=2025.3.0.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
@claude @ayushdg
Regenerate uv.lock with updated fsspec override
d711191 
Updates lock file to reflect fsspec>=2025.3.0 override needed due to
nemo-toolkit[asr]==2.7.2 pinning fsspec==2024.12.0.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
@claude @ayushdg
Cap protobuf <7.0 to fix Ray Serve FieldDescriptor API breakage
d0ba895 
protobuf 7.x dropped the .label attribute from the C extension
FieldDescriptor, breaking ray/serve/_private/config.py:75. Cap to <7.0
in both constraint-dependencies and override-dependencies so uv resolves
to 6.x (was bumping to 7.34.1 after lock regeneration).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
@claude @ayushdg
Cap vllm <0.16.0 to fix Ray Serve LLMServer deployment failure
8629312 
ray[llm]==2.54 is incompatible with vllm>=0.16.0 which renamed internal
modules (translations -> speech_to_text), causing LLMServer to hit
DEPLOY_FAILED. Cap vllm to <0.16.0 until ray[llm] adds support.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
@ayushdg
Address comments
4b80907 
Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
praateekmahajan
praateekmahajan reviewed Apr 21, 2026
View reviewed changes
Comment thread pyproject.toml Outdated
"pandas>=2.1.0",
"pyarrow",
"ray[default,data]>=2.54",
"ray[default,data]==2.54",
Copy link
Copy Markdown
Contributor

@praateekmahajan praateekmahajan Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we hard pinning this? uv.lock shouldn't change? Can we do something like git checkout upstream/main uv.lock and then remove these hard pins, I don't think a uv sync will change the uv.lock..

Same for vllm upper pin below

Copy link
Copy Markdown
Contributor

@ayushdg ayushdg Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did contemplate doing that but the previous commits on this branch somehow did end up modifying the lock file. I think this is a more foolproof way to ensure we don't accidentally change things last minute. 1000+ line lock file changes often slip through the cracks.

@sarahyurick
Copy link
Copy Markdown
Contributor

sarahyurick commented Apr 21, 2026

Is this targeting r1.2.0 as well or just main?

@ayushdg
Copy link
Copy Markdown
Contributor

ayushdg commented Apr 21, 2026

Is this targeting r1.2.0 as well or just main?

1.2.0 as well. Thanks for mentioning

ayushdg added 2 commits April 21, 2026 16:55
@ayushdg
Remove hard pins and rely on base uvlock deps
fa28932 
Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
@ayushdg
Forgot to update ray-serve pin
7871f1d 
Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
@ayushdg
Copy link
Copy Markdown
Contributor

ayushdg commented Apr 22, 2026

/ok to test 7871f1d

@greptile-apps greptile-apps Bot mentioned this pull request Apr 22, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@praateekmahajan praateekmahajan praateekmahajan approved these changes

@thomasdhc thomasdhc thomasdhc approved these changes

@ayushdg ayushdg ayushdg approved these changes

@rlratzel rlratzel Awaiting requested review from rlratzel

@sarahyurick sarahyurick Awaiting requested review from sarahyurick

@suiyoubi suiyoubi Awaiting requested review from suiyoubi

@abhinavg4 abhinavg4 Awaiting requested review from abhinavg4

@oyilmaz-nvidia oyilmaz-nvidia Awaiting requested review from oyilmaz-nvidia

@VibhuJawa VibhuJawa Awaiting requested review from VibhuJawa

+1 more reviewer

@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

r1.2.0 Pick this label for auto cherry-picking into r1.2.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@huvunvidia @ayushdg @sarahyurick @praateekmahajan @thomasdhc