Skip to content

[generation] Fix misleading synced_gpus warning in continuous batching#47158

Open
Partha-Shankar wants to merge 3 commits into
huggingface:mainfrom
Partha-Shankar:generation-fix-synced-gpus-warning
Open

[generation] Fix misleading synced_gpus warning in continuous batching#47158
Partha-Shankar wants to merge 3 commits into
huggingface:mainfrom
Partha-Shankar:generation-fix-synced-gpus-warning

Conversation

@Partha-Shankar

@Partha-Shankar Partha-Shankar commented Jul 8, 2026

Copy link
Copy Markdown

CI

What does this PR do?

This PR fixes a misleading user-facing warning message when calling generate() with synced_gpus=True in continuous batching mode (e.g. cache_implementation="paged").
Closes #47276

Summary

The warning message previously stated:
synced_gpus is not ignored for continuous batching. Got synced_gpus = True
This double negative incorrectly implies that synced_gpus is active or supported. In fact, synced_gpus is discarded during continuous batching execution. The comment directly above the warning also explicitly reads # others are ignored.
This PR corrects the warning to state:
synced_gpus is ignored for continuous batching. Got synced_gpus = True

Root Cause

A simple negation typo in the warning string format.

Solution

Change "is not ignored" to "is ignored" in the warning message in src/transformers/generation/utils.py.

Why This Is a Standalone Fix

This PR addresses a user-facing negation typo in the warning message logged
when synced_gpus is passed during continuous batching. Because continuous
batching ignores synced_gpus, the warning mistakenly read "synced_gpus is
not ignored...", which can cause significant confusion for users debugging
distributed setups who believe the parameter is active. As this is a highly
visible, misleading warning specifically affecting the continuous batching
entry point, and a search of the generation codebase confirmed no other
stale warning negation bugs nearby, it is submitted as a targeted, standalone
fix rather than bundled into a broader cleanup.

Tests Run

  • Verified the fix manually and confirmed no regressions via the full existing test suite: pytest tests/generation/test_continuous_batching.py -v → 49 passed, 71 skipped, 0 failed.
  • Checked code style and formatting via ruff check and ruff format locally.

AI-Assisted Tooling & Guidelines Compliance

  • AI Tooling Used: Google Antigravity was used to review the codebase and
    assist in understanding the continuous batching parameter validation flow.
    The fix was authored directly by the developer.
  • Coordination Link: Misleading synced_gpus warning message in continuous batching #47276
  • Duplicate-PR Check: Searched open and recently-closed pull requests on
    huggingface/transformers for PRs touching the same warning string or file
    (src/transformers/generation/utils.py). No duplicates or similar
    sweeping cleanup PRs were found.
  • Verification & Test Output:
  $ ruff check src/transformers/generation/utils.py
  All checks passed!
  $ ruff format --check src/transformers/generation/utils.py
  1 file already formatted
  $ pytest tests/generation/test_continuous_batching.py -v
  ======================= 49 passed, 71 skipped in 12.14s =======================

Correct the warning negation when synced_gpus is passed in continuous batching generation mode. Since synced_gpus is ignored and not processed, the warning should read 'is ignored' instead of 'is not ignored'.

Add a regression test in test_continuous_batching.py to verify.
@Partha-Shankar Partha-Shankar force-pushed the generation-fix-synced-gpus-warning branch from cf880e8 to 1c18627 Compare July 12, 2026 11:17
@remi-or

remi-or commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Thanks for fixing the nit, please remove the test and I will merge.

@Partha-Shankar

Copy link
Copy Markdown
Author

Done — removed the test, only the warning-message fix remains. Thanks for the review!

@remi-or remi-or left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actions

Copy link
Copy Markdown
Contributor

CI recap

Dashboard: View test results in Grafana
Latest run: 29225200277:2
Result: failure | Jobs: 15 | Tests: 170,777 | Failures: 0 | Duration: 15h 26m

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@remi-or remi-or enabled auto-merge July 13, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Misleading synced_gpus warning message in continuous batching

3 participants