[NPUW] Model builder: add LoRA adapter support#36320
Merged
AlexanderKalistratov merged 6 commits intoJul 14, 2026
Merged
Conversation
github-actions
Bot
added
category: build
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds LoRA adapter injection support to the NPUW synthetic LLM model builder used in Intel NPU plugin tests, enabling construction of models with LoRA A/B/alpha tensors and supporting both stateless (Parameters) and stateful (ReadValue/Assign) forms.
Changes:
- Extend the test-engine
ModelBuilderLLM configuration andmake_linear()/Attention/FFN builders to optionally inject LoRA subgraphs. - Add stateful→stateless conversion coverage and LoRA-subgraph-fusion compatibility coverage via new unit tests.
- Adjust SDPA scale constant typing to match the query element type to avoid mixed-dtype validation issues.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/plugins/intel_npu/tests/unit/npuw/model_builder_lora_test.cpp | Adds unit tests validating LoRA input naming/shapes, stateful→stateless conversion, fusion, and basic inference impact. |
| src/plugins/intel_npu/tests/unit/CMakeLists.txt | Registers the new LoRA unit test source in the unit test target. |
| src/plugins/intel_npu/tests/functional/behavior/npuw/test_engine/models/model_builder.hpp | Extends make_linear() signature and LLMConfig with LoRA configuration fields. |
| src/plugins/intel_npu/tests/functional/behavior/npuw/test_engine/models/model_builder.cpp | Implements LoRA tensor creation (Parameter or ReadValue/Assign), injects LoRA into linear layers, and wires LoRA into attention/FFN during LLM build. |
| src/plugins/intel_npu/tests/functional/behavior/npuw/test_engine/models/model_builder_types.hpp | Introduces LoRAInjector describing LoRA injection settings and target matching. |
| src/plugins/intel_npu/tests/functional/behavior/npuw/test_engine/models/model_builder_ffn.hpp | Propagates optional LoRA injector through FFN functors (SwiGLU/GELU). |
| src/plugins/intel_npu/tests/functional/behavior/npuw/test_engine/models/model_builder_ffn.cpp | Passes LoRA injector to underlying linear projections in FFN implementations. |
| src/plugins/intel_npu/tests/functional/behavior/npuw/test_engine/models/model_builder_attention.hpp | Adds optional LoRA injector plumbing to attention output projection and Attention struct. |
| src/plugins/intel_npu/tests/functional/behavior/npuw/test_engine/models/model_builder_attention.cpp | Uses element-type-consistent SDPA scale and propagates LoRA injector through Q/K/V and O projections. |
dylanneve1
force-pushed
the
EISW-209517-lora
branch
4 times, most recently
from
fa12d2b to
590a9be
Compare
dylanneve1
force-pushed
the
EISW-209517-lora
branch
6 times, most recently
from
9d0f772 to
81f9582
Compare
dylanneve1
force-pushed
the
EISW-209517-lora
branch
from
81f9582 to
8094a9e
Compare
Add low-rank adaptation (LoRA) injection to the synthetic model builder.
When LLMConfig::lora_rank > 0, every adapted linear projection gets three
extra A/B/alpha tensors wired as:
lora_out = input @ A^T * alpha @ B^T
output = base_linear_out + lora_out
Tensor names follow the NPUW lora_state_* convention:
lora_state_<layer_name>.MatMul.A [rank, in_features]
lora_state_<layer_name>.MatMul.B [out_features, rank]
lora_state_<layer_name>.MatMul.alpha [1, rank]
Default adapted projections: q_proj, k_proj, v_proj, o_proj, gate_proj,
up_proj, down_proj (configurable via LLMConfig::lora_targets). Stateless
form exposes A/B/alpha as Parameters; stateful form (lora_stateful=true)
exposes them as ReadValue/Assign state pairs.
Ported onto the split model_builder.* layout: LoRAInjector lives in
model_builder_types.hpp; make_lora_value + make_linear injection in
model_builder.cpp; SwiGLU/GELU wiring in model_builder_ffn.*; Attention
+ make_attention_output wiring in model_builder_attention.*; LLMConfig
fields and build_llm setup in model_builder.{hpp,cpp}. make_sdpa now
builds its scale constant in q's element type so f16 LoRA models don't
trip SDPA's mixed-input-type check.
All four ModelBuilderLoraTest cases pass.
Add a dedicated LoRAConfig (extends BaseModelConfig) and build_lora_adapter
so a generic LoRA model can be built without standing up a full LLM and
disabling its KV cache / position-id machinery by hand.
build_lora_adapter emits a minimal embed -> [adapted linear] x layers ->
output graph with no attention, KV cache or RoPE. Every layer applies one
LoRA-injected make_linear per target projection, producing the same
lora_state_*.MatMul.{A,B,alpha} Parameters (stateless) or ReadValue/Assign
states (stateful) that NPUW's LoRA transforms consume.
Replace the model_builder_lora_test with two simple sanity checks (stateless
exposes lora_state_ Parameters with the expected A/B/alpha names; stateful
exposes states). Richer LoRA test infrastructure is follow-up work.
LoRA stays explicitly threaded as a const LoRAInjector* through make_linear and the Attention / SwiGLU / GELU functors (no global state). - LoRAInjector::max_rank / precision get safe defaults (0 / f32) and should_adapt() returns false for max_rank == 0, so a default-constructed injector is an inert no-op. - build_llm holds the injector on the stack and recreates only the default dense FFN as a LoRA-aware SwiGLU; custom and MoE FFN graphs are untouched. - Add unit coverage for the build_llm LoRA path (attention + FFN adapted) and the no-LoRA negative case, alongside the build_lora_adapter checks.
…ic LoRA Append "/MatMul" to the projection MatMul friendly name so GenAI's dynamic-LoRA lora_state_* ids carry "MatMul"; NPUW's ReshapeToStatic only pins the LoRA rank static for those, otherwise NPU compile throws "to_shape was called on a dynamic shape".
Extract inject_lora() from make_linear, share the default target list via LoRAInjector::default_targets(), use static ov::Shape for LoRA tensors, and strengthen the stateful test to verify lora_state_*.MatMul.* variable ids.
dylanneve1
force-pushed
the
EISW-209517-lora
branch
from
8094a9e to
1a5c957
Compare
# Conflicts: # src/plugins/intel_npu/tests/functional/behavior/npuw/test_engine/models/model_builder.cpp # src/plugins/intel_npu/tests/functional/behavior/npuw/test_engine/models/model_builder.hpp # src/plugins/intel_npu/tests/functional/behavior/npuw/test_engine/models/model_builder_attention.cpp # src/plugins/intel_npu/tests/functional/behavior/npuw/test_engine/models/model_builder_attention.hpp
AlexanderKalistratov
approved these changes
Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details:
Adds LoRA adapter support to the NPUW synthetic model builder so LoRA-enabled LLM paths can be tested without real models or adapters.
LoRAInjectorcarries the LoRA settings (max rank, targets, precision, stateless/stateful form) throughmake_linear, theAttentionblock and the SwiGLU/GELU FFNs. Adapted projections get the standard low-rank branchout = base + (input @ A^T * alpha) @ B^T.lora_state_*Parameters (NPUW form),ReadValue/Assignstate pairs (GenAI dynamic form before NPUW conversion).[r, in], B[out, r], alpha[1, r]f32, ids ending inMatMul.A/B/alpha).LLMConfiggetslora_rank/lora_targets/lora_stateful;build_lora_adapter(LoRAConfig)builds a minimal standalone adapter model.<layer>/MatMul) so GenAI dynamic-LoRA state ids carryMatMuland NPUW pins the LoRA rank static viaNPUW_LLM_MAX_LORA_RANK.Validated by running synthetic models with a separate PEFT adapter end to end via GenAI dynamic LoRA (NPU and NPUW_DEVICES=CPU), and comparing the resulting subgraphs against TinyLlama with real LoRA adapters: identical LoRA structure.
Tickets: