Skip to content

Commit 5eb2a5a

Browse files
vortex-captainYi Ren
andauthored
gemma-3-1b-it (microsoft#150)
Co-authored-by: Yi Ren <reny@microsoft.com>
1 parent 0149ee8 commit 5eb2a5a

3 files changed

Lines changed: 30 additions & 0 deletions

File tree

google-gemma/olive/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,11 @@ The exported ONNX model is saved in `output_model` folder.
2525
To run the ONNX GenAI model, please set up the latest ONNXRuntime GenAI.
2626

2727
The sample chat app to run is found as [model-chat.py](https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/model-chat.py) in the [onnxruntime-genai](https://github.com/microsoft/onnxruntime-genai/) Github repository.
28+
29+
## google/gemma-3-1b-it
30+
31+
```bash
32+
python -m pip install -r requirements.txt
33+
# Use the following command to export the model using Olive with CPUExecutionProvider at FP32 precision:
34+
olive run --config gemma-3-1b-it_model_builder_cpu_fp32.json
35+
```
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"input_model": {
3+
"type": "HfModel",
4+
"model_path": "google/gemma-3-1b-it",
5+
"task": "text-generation"
6+
},
7+
"systems": {
8+
"local_system": {
9+
"type": "LocalSystem",
10+
"accelerators": [ { "device": "cpu", "execution_providers": [ "CPUExecutionProvider" ] } ]
11+
}
12+
},
13+
"engine": { "target": "local_system" },
14+
"passes": {
15+
"builder": { "type": "ModelBuilder", "precision": "fp32"}
16+
17+
}
18+
}

google-gemma/olive/info.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,7 @@ recipes:
77
- file: "README.md"
88
device: gpu
99
ep: CUDAExecutionProvider
10+
- name: gemma-3-1b-it_model_builder_cpu_FP32
11+
file: gemma-3-1b-it_model_builder_cpu_fp32.json
12+
devices: cpu
13+
eps: CPUExecutionProvider

0 commit comments

Comments
 (0)