Remove dead bitsandbytes CxB code from 8-bit inference path#34633
Conversation
The `MatmulLtState.CxB` attribute has been always `None` since bitsandbytes v0.45.0 (December 2024), when the col32/ColAmpere tensor layout system was removed. The conditional block that checked `matmul_states[i].CxB is not None` has therefore never executed. bitsandbytes is removing the `CxB` attribute entirely in an upcoming release (see bitsandbytes-foundation/bitsandbytes#1871), which would cause an AttributeError here. This commit removes the dead code proactively. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TimDettmers
requested review from
mgoin,
pavanimajety,
robertgshaw2-redhat,
tlrmchlsmth and
yewentao256
as code owners
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
The pull request effectively removes dead code related to BitsAndBytesLinearMethod._apply_8bit_weight() that accessed MatmulLtState.CxB. This change proactively addresses upcoming bitsandbytes API removals, ensuring future compatibility and preventing potential AttributeError crashes. The removal is a no-op in terms of functionality, as the affected code path was already unreachable. The changes are clean and directly target the identified dead code.
…ject#34633) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ject#34633) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ject#34633) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ject#34633) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ject#34633) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
3 participants
Summary
BitsAndBytesLinearMethod._apply_8bit_weight()that accessesMatmulLtState.CxB— an attribute that has been alwaysNonesince bitsandbytes v0.45.0 (December 2024)CxBwas part of the old col32/ColAmpere tensor layout system that was removed over a year agoContext
bitsandbytes is cleaning up its deprecated API surface. In the current release,
CxBwill still exist as a deprecated stub that emits aFutureWarningand returnsNone(see bitsandbytes#1871), so there is no immediate breakage. However, the attribute will be fully removed in the next bitsandbytes release.This PR removes the dead code proactively so vLLM is ready for both the current and next bitsandbytes versions.
Test plan
🤖 Generated with Claude Code