Skip to content

Commit 3b606ff

Browse files
mlcommons-botmlcommons-bot
committed
Auto-merge updates from auto-update branch
2 parents dfaf895 + fdbaa95 commit 3b606ff

12 files changed

Lines changed: 336 additions & 336 deletions

File tree

‎open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.27.0-default_config/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
| Model | Scenario | Accuracy | Throughput | Latency (in ms) |
22
|----------|------------|------------|--------------|-------------------|
3-
| resnet50 | offline | 76 | 20.843 | - |
3+
| resnet50 | offline | 76 | 16.468 | - |

‎open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.27.0-default_config/resnet50/offline/README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@
22

33
## Host platform
44

5-
* OS version: macOS-26.4-arm64-arm-64bit-Mach-O
5+
* OS version: macOS-26.4-arm64-arm-64bit
66
* CPU version: arm
7-
* Python version: 3.13.14 (v3.13.14:fd17997c386, Jun 10 2026, 08:55:00) [Clang 21.0.0 (clang-2100.1.1.101)]
7+
* Python version: 3.12.10 (v3.12.10:0cc81280367, Apr 8 2025, 08:46:59) [Clang 13.0.0 (clang-1300.0.29.30)]
88
* MLC version: unknown
99

1010
## MLC Run Command
@@ -16,7 +16,7 @@ pip install -U mlcflow
1616

1717
mlc rm cache -f
1818

19-
mlc pull repo mlcommons@mlperf-automations --checkout=6c016284bcad817bed38bebe91d5980f8cfff6cd
19+
mlc pull repo mlcommons@mlperf-automations --checkout=1868c3eb670482dfe71ed00164e67f03aea8f0b0
2020

2121

2222
```
@@ -40,4 +40,4 @@ Model Precision: fp32
4040
`acc`: `76.0`, Required accuracy for closed division `>= 75.6954`
4141

4242
### Performance Results
43-
`Samples per second`: `20.8426`
43+
`Samples per second`: `16.4677`

‎open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.27.0-default_config/resnet50/offline/accuracy/accuracy.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
accuracy=76.000%, good=380, total=500
22

3-
hash=9c585ac96c5a580dab5eee9cc1fda30f1d28dada0696e70e4f5af815be52b9af
3+
hash=986cddf542d04956a6ab5ed33e10b3cc0a745bb6439a5f95ab3e7e15e4127bf3

‎open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.27.0-default_config/resnet50/offline/accuracy/mlperf_log_accuracy.json‎

Lines changed: 144 additions & 144 deletions
Large diffs are not rendered by default.

‎open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.27.0-default_config/resnet50/offline/accuracy/mlperf_log_detail.txt‎

Lines changed: 72 additions & 72 deletions
Large diffs are not rendered by default.

‎open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.27.0-default_config/resnet50/offline/accuracy_console.out‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
python3 python/main.py --profile resnet50-onnxruntime --model "/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_da7c0e12/resnet50_v1.onnx" --dataset-path /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_f3f6b7b6 --output "/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_2cc51768/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.27.0-default_config/resnet50/offline/accuracy" --scenario Offline --count 500 --threads 3 --user_conf /Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/8c89fe57657846ac98c0e7e9527cd5f5.conf --accuracy --use_preprocessed_dataset --cache_dir /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_f3f6b7b6 --dataset-list /Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_4246f230/val.txt
2-
INFO:main:Namespace(dataset='imagenet', dataset_path='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_f3f6b7b6', dataset_list='/Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_4246f230/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_da7c0e12/resnet50_v1.onnx', output='/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_2cc51768/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.27.0-default_config/resnet50/offline/accuracy', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=3, qps=None, cache=0, cache_dir='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_f3f6b7b6', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=True, find_peak_performance=False, debug=False, user_conf='/Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/8c89fe57657846ac98c0e7e9527cd5f5.conf', audit_conf='audit.config', time=None, count=500, performance_sample_count=None, max_latency=None, samples_per_query=8)
1+
python3 python/main.py --profile resnet50-onnxruntime --model "/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_5f6db1d4/resnet50_v1.onnx" --dataset-path /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_96389c4f --output "/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_99a6fc17/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.27.0-default_config/resnet50/offline/accuracy" --scenario Offline --count 500 --threads 3 --user_conf /Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/ef9efa199ed4408bb3e7db7881020420.conf --accuracy --use_preprocessed_dataset --cache_dir /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_96389c4f --dataset-list /Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_c3a3bbd1/val.txt
2+
INFO:main:Namespace(dataset='imagenet', dataset_path='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_96389c4f', dataset_list='/Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_c3a3bbd1/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_5f6db1d4/resnet50_v1.onnx', output='/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_99a6fc17/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.27.0-default_config/resnet50/offline/accuracy', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=3, qps=None, cache=0, cache_dir='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_96389c4f', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=True, find_peak_performance=False, debug=False, user_conf='/Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/ef9efa199ed4408bb3e7db7881020420.conf', audit_conf='audit.config', time=None, count=500, performance_sample_count=None, max_latency=None, samples_per_query=8)
33
INFO:imagenet:Loading 500 preprocessed images using 3 threads
44
INFO:imagenet:loaded 500 images, cache=0, already_preprocessed=True, took=0.0sec
55
INFO:main:starting TestScenario.Offline
6-
TestScenario.Offline qps=0.68, mean=13.9223, time=23.468, acc=76.000%, queries=16, tiles=50.0:13.5478,80.0:20.7731,90.0:21.2778,95.0:22.0291,99.0:22.9972,99.9:23.2151
6+
TestScenario.Offline qps=0.56, mean=16.4905, time=28.568, acc=76.000%, queries=16, tiles=50.0:16.3065,80.0:25.0759,90.0:25.1638,95.0:25.9577,99.0:27.7209,99.9:28.1176

‎open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.27.0-default_config/resnet50/offline/os_info.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,6 @@
1010
"MLC_HOST_OS_PACKAGE_MANAGER_UPDATE_CMD": "brew update",
1111
"MLC_HOST_PLATFORM_FLAVOR": "arm64",
1212
"MLC_HOST_PYTHON_BITS": "64",
13-
"MLC_HOST_SYSTEM_NAME": "sat12-bq150-1445c42a-82f3-4365-a379-c67ddf67da08-669DB3520045.local",
13+
"MLC_HOST_SYSTEM_NAME": "sjc22-be110-9164832f-2a3c-440a-ac6d-ce053529e55a-A602E6117960.local",
1414
"MLC_HOST_FILESYSTEMS": ""
1515
}

0 commit comments

Comments
 (0)