Skip to content

Fix NPU hardware accelerator detection for custom models loaded from AITK converted genai_config.json#525

Merged
weiyuanyue merged 1 commit into
milly/npufrom
copilot/sub-pr-524
Dec 4, 2025
Merged

Fix NPU hardware accelerator detection for custom models loaded from AITK converted genai_config.json#525
weiyuanyue merged 1 commit into
milly/npufrom
copilot/sub-pr-524

Conversation

Copilot AI commented Dec 4, 2025

Copy link
Copy Markdown
Contributor

AITK-converted NPU models were incorrectly tagged as CPU because the detection logic couldn't parse diverse execution provider configurations from genai_config.json.

Changes

Enhanced GenAI Config Parsing (GenAIConfig.cs)

  • Replaced rigid Dml/Cuda classes with flexible JsonExtensionData parsing to support diverse execution providers (QNN, OpenVINO, VitisAI, DML, CUDA, etc.)
  • Added pipeline-level SessionOptions support—AITK configs specify EPs at both decoder and pipeline levels
  • Implemented HasProvider() and GetProviderOptions() for dynamic provider detection
internal class ProviderOptions
{
    [JsonExtensionData]
    public Dictionary<string, JsonElement>? ExtensionData { get; set; }

    public bool HasProvider(string name) =>
        ExtensionData?.Keys.Any(k => k.Equals(name, StringComparison.OrdinalIgnoreCase)) ?? false;
}

Hardware Detection (DeviceUtils.cs, ExecutionProviderNames.cs)

  • Added HasNPU() using ONNX Runtime's EP device detection
  • Implemented GetAvailableExecutionProviders() with certified EP registration and caching
  • Created ExecutionProviderNames constants class to eliminate hardcoded EP strings

Model Classification (UserAddedModelUtil.cs)

  • Rewrote GetHardwareAcceleratorFromConfig() to:
    • Scan both decoder-level and pipeline-level provider options
    • Detect NPU configs (QNN, OpenVINO with NPU device type, VitisAI)
    • Apply priority: QNN → DML → NPU → GPU → CPU
  • Added ValidateExecutionProviders() to check EP availability before model load
  • Display warning dialog when required EPs are unavailable

Compatibility Checks (ModelCompatibility.cs)

  • Added NPU compatibility validation with proper error messaging
  • Split CPU/QNN/NPU compatibility checks for clarity

Code Quality

  • Standardized EP references using ExecutionProviderNames constants
  • Refactored using LINQ for improved readability

Before/After

Before: NPU model shows CPU tag

After: NPU model correctly tagged

Warning dialog for unavailable EPs:


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI changed the title [WIP] Fix NPU hardware accelerator detection for custom models Fix NPU hardware accelerator detection for custom models loaded from AITK converted genai_config.json Dec 4, 2025
Copilot AI requested a review from weiyuanyue December 4, 2025 02:50
@weiyuanyue
weiyuanyue marked this pull request as ready for review December 4, 2025 02:54
@weiyuanyue
weiyuanyue merged commit f799a2e into milly/npu Dec 4, 2025
2 checks passed
@weiyuanyue
weiyuanyue deleted the copilot/sub-pr-524 branch December 4, 2025 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants