Skip to content

fix: qkvsync bug fix#161

Merged
chichun-charlie-liu merged 3 commits intofoundation-model-stack:mainfrom
chichun-charlie-liu:qkvsync_bug_fix
Jul 10, 2025
Merged

fix: qkvsync bug fix#161
chichun-charlie-liu merged 3 commits intofoundation-model-stack:mainfrom
chichun-charlie-liu:qkvsync_bug_fix

Conversation

@chichun-charlie-liu
Copy link
Copy Markdown
Collaborator

Description of the change

qkvsync is a function in model_analyzer trying to find Linear layers with common input, e.g. Q, K, V, or gate/up (in Llama arch). The results by this searching, a dict like {"q":"q", "k":"q", "v":"q", ...}, will be saved in qcfg["qkvsync_my_1st_siblings"]. This dict is supposed to contain the full layer name, however, when graph breaks happens, model_analyzer can only identify partial names instead. Later functions attempt to use model.get_submodule() based on this dict will fail.

How to verify the PR

run dq with bamba (ibm-ai-platform/Bamba-9B-V2)

Was the PR tested

  • I have added >=1 unit test(s) for every new method I have added (if that coverage is difficult, please briefly explain the reason)
  • I have ensured all unit tests pass

Checklist for passing CI/CD:

  • All commits are signed showing "Signed-off-by: Name <email@domain.com>" with git commit -signoff or equivalent
  • PR title and commit messages adhere to Conventional Commits
  • Contribution is formatted with tox -e fix
  • Contribution passes linting with tox -e lint
  • Contribution passes spellcheck with tox -e spellcheck
  • Contribution passes all unit tests with tox -e unit

Note: CI/CD performs unit tests on multiple versions of Python from a fresh install. There may be differences with your local environment and the test environment.

Signed-off-by: cliu-us <cliu@us.ibm.com>
…y partial names will be found and cause problems

Signed-off-by: cliu-us <cliu@us.ibm.com>
@chichun-charlie-liu chichun-charlie-liu changed the title Qkvsync bug fix fix: qkvsync bug fix Jul 10, 2025
@github-actions github-actions Bot added the fix label Jul 10, 2025
Signed-off-by: cliu-us <cliu@us.ibm.com>
@chichun-charlie-liu chichun-charlie-liu merged commit ae847c2 into foundation-model-stack:main Jul 10, 2025
11 checks passed
@chichun-charlie-liu chichun-charlie-liu deleted the qkvsync_bug_fix branch July 10, 2025 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants