Skip to content

Commit afbb4c3

Browse files
authored
Add nemotron multilingual ASR cuda model to Foundry Local (#5113)
1 parent 640ef9b commit afbb4c3

4 files changed

Lines changed: 60 additions & 0 deletions

File tree

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
extra_config: model.yaml
2+
spec: spec.yaml
3+
type: model
4+
categories: ["Local"]
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
This model is an optimized version of nemotron-3.5-asr-streaming-0.6b to enable local inference on GPUs. This model uses RTN quantization.
2+
3+
# Model Description
4+
5+
- **Developed by:** Microsoft
6+
- **Model type:** ONNX
7+
- **License:** MIT
8+
- **Model Description:** This is an optimized version of the nemotron-3.5-asr-streaming-0.6b model for local inference on GPUs.
9+
- **Disclaimer:** Model is only an optimization of the base model, any risk associated with the model is the responsibility of the user of the model. Please verify and test for your scenarios. There may be a slight difference in output from the base model with the optimizations applied. Note that optimizations applied are distinct from fine tuning and thus do not alter the intended uses or capabilities of the model.
10+
11+
# Base Model Information
12+
13+
See Hugging Face model [nemotron-3.5-asr-streaming-0.6b](https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b) for details.
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
path:
2+
container_name: models
3+
container_path: foundrylocal/models/nemotron-3.5-asr-streaming-0.6b/onnx/cuda/v1
4+
storage_name: foundrylocalassetdata
5+
type: azureblob
6+
publish:
7+
description: description.md
8+
type: custom_model
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
$schema: https://azuremlschemas.azureedge.net/latest/model.schema.json
2+
name: nemotron-3.5-asr-streaming-0.6b-cuda-gpu
3+
version: 1
4+
isArchived: true
5+
path: ./
6+
tags:
7+
foundryLocal: ""
8+
license: "MIT"
9+
licenseDescription: "This model is provided under the License Terms available at <https://github.com/microsoft/Foundry-Local/blob/main/licenses/nemotron-speech-streaming.md>."
10+
author: Microsoft
11+
inputModalities: "audio"
12+
outputModalities: "text"
13+
task: automatic-speech-recognition
14+
maxOutputTokens: 2048
15+
alias: nemotron-3.5-asr-streaming-0.6b
16+
directoryPath: v1
17+
promptTemplate: ""
18+
capabilities: ""
19+
supportsReasoning: ""
20+
reasoningStart: ""
21+
reasoningEnd: ""
22+
contextLength: 0
23+
minFLVersion: "1.2.1"
24+
disable-maap: "true"
25+
type: custom_model
26+
variantInfo:
27+
parents:
28+
- assetId: azureml://registries/azureml/models/nemotron-3.5-asr-streaming-0.6b/versions/1
29+
variantMetadata:
30+
modelType: 'ONNX'
31+
quantization: ['RTN']
32+
device: 'gpu'
33+
executionProvider: 'CUDAExecutionProvider'
34+
fileSizeBytes: 793344637
35+
vRamFootprintBytes: 793344637

0 commit comments

Comments
 (0)