apple-perf #80

Run time

Learn about OS pricing on GitHub Actions

Job	Run time
set-parameters	8s
prepare-test-specs (mv2, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo...	1s
prepare-test-specs (mv2, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	1s
prepare-test-specs (mv3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	1s
prepare-test-specs (mv3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d...	1s
prepare-test-specs (mv2, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d...	1s
prepare-test-specs (mv3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo...	2m 44s
prepare-test-specs (ic4, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo...	1s
prepare-test-specs (w2l, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	1s
prepare-test-specs (mobilebert, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:3085353...	1s
prepare-test-specs (meta-llama/Llama-3.2-1B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_...	1s
prepare-test-specs (ic3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d...	1s
prepare-test-specs (ic4, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	1s
prepare-test-specs (HuggingFaceTB/SmolLM2-135M, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_ipho...	1s
prepare-test-specs (resnet50, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385...	1s
prepare-test-specs (Qwen/Qwen3-0.6B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, arn:...	1s
prepare-test-specs (mobilebert, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535...	1s
prepare-test-specs (ic3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo...	1s
prepare-test-specs (google/gemma-3-1b-it, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15,...	1s
prepare-test-specs (w2l, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d...	2m 46s
prepare-test-specs (meta-llama/Llama-3.2-1B, llama3_coreml_ane, apple_iphone_15+ios_18, arn:aws:d...	1s
prepare-test-specs (ic4, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d...	1s
prepare-test-specs (ic3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	1s
prepare-test-specs (allenai/OLMo-1B-hf, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, a...	2m 46s
prepare-test-specs (edsr, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:...	1s
prepare-test-specs (meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8, llama3_spinquant, apple_...	2m 46s
prepare-test-specs (resnet50, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:dev...	2m 45s
prepare-test-specs (edsr, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114...	2m 45s
prepare-test-specs (meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8, llama3_qlora, apple_iphone_1...	1s
prepare-test-specs (resnet50, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853538...	1s
prepare-test-specs (Qwen/Qwen3-0.6B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, arn:...	1s
prepare-test-specs (meta-llama/Llama-3.2-1B, llama3_fb16, apple_iphone_15, arn:aws:devicefarm:us-...	1s
prepare-test-specs (w2l, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo...	2m 45s
build-benchmark-app / macos-job	8s
export-models (meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8, llama3_spinquant, apple_iphon... / macos-job	2m 45s
export-models (mv2, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:02... / macos-job	1s
export-models (edsr, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devi... / macos-job	1s
export-models (mobilebert, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / macos-job	2m 45s
export-models (meta-llama/Llama-3.2-1B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, a... / macos-job	2m 45s
export-models (Qwen/Qwen3-0.6B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, arn:aws:d... / macos-job	1s
export-models (mv3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	1s
export-models (resnet50, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepo... / macos-job	1s
export-models (mobilebert, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853538511... / macos-job	1s
prepare-test-specs (meta-llama/Llama-3.2-1B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_...	1s
export-models (ic3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / macos-job	1s
export-models (w2l, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	1s
export-models (meta-llama/Llama-3.2-1B, llama3_coreml_ane, apple_iphone_15+ios_18, arn:aws:device... / macos-job	1s
export-models (ic4, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:02... / macos-job	2m 44s
export-models (w2l, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:02... / macos-job	1s
export-models (ic3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:02... / macos-job	1s
export-models (meta-llama/Llama-3.2-1B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, a... / macos-job	2m 45s
export-models (google/gemma-3-1b-it, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, arn:... / macos-job	1s
export-models (meta-llama/Llama-3.2-1B, llama3_fb16, apple_iphone_15, arn:aws:devicefarm:us-west-... / macos-job	2m 45s
export-models (edsr, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	1s
export-models (w2l, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / macos-job	1s
export-models (mv2, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	1s
export-models (allenai/OLMo-1B-hf, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, arn:aw... / macos-job	1s
export-models (resnet50, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:d... / macos-job	1s
export-models (ic4, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	1s
export-models (Qwen/Qwen3-0.6B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, arn:aws:d... / macos-job	1s
export-models (HuggingFaceTB/SmolLM2-135M, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15... / macos-job	1s
export-models (meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8, llama3_qlora, apple_iphone_15, ar... / macos-job	1s
export-models (ic4, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / macos-job	1s
export-models (ic3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job	1s
export-models (mv2, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / macos-job	1s
export-models (resnet50, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / macos-job	1s
export-models (mv3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:device... / macos-job	1s
export-models (mv3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicepool:02... / macos-job	1s
upload-benchmark-app	0s
benchmark-on-device (mv2, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / mobile-job (ios)	1s
benchmark-on-device (ic3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicep... / mobile-job (ios)	1s
benchmark-on-device (allenai/OLMo-1B-hf, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, ... / mobile-job (ios)	1s
benchmark-on-device (Qwen/Qwen3-0.6B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, arn... / mobile-job (ios)	1s
benchmark-on-device (resnet50, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:3085353... / mobile-job (ios)	1s
benchmark-on-device (mobilebert, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535... / mobile-job (ios)	1s
benchmark-on-device (mv2, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicep... / mobile-job (ios)	1s
benchmark-on-device (mv3, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicep... / mobile-job (ios)	1s
benchmark-on-device (mv3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	1d 0h 0m 2s
benchmark-on-device (ic4, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	1s
benchmark-on-device (ic3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / mobile-job (ios)	1s
benchmark-on-device (ic4, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicep... / mobile-job (ios)	1s
benchmark-on-device (meta-llama/Llama-3.2-1B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone... / mobile-job (ios)	1s
benchmark-on-device (ic3, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	1s
benchmark-on-device (edsr, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	1s
benchmark-on-device (HuggingFaceTB/SmolLM2-135M, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iph... / mobile-job (ios)	1d 0h 0m 2s
benchmark-on-device (meta-llama/Llama-3.2-1B, llama3_fb16, apple_iphone_15, arn:aws:devicefarm:us... / mobile-job (ios)	1d 0h 0m 2s
benchmark-on-device (edsr, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853538511... / mobile-job (ios)	1s
benchmark-on-device (mobilebert, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853... / mobile-job (ios)	1s
benchmark-on-device (meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8, llama3_spinquant, apple... / mobile-job (ios)	1s
benchmark-on-device (w2l, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:devicep... / mobile-job (ios)	1s
benchmark-on-device (meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8, llama3_qlora, apple_iphone_... / mobile-job (ios)	1s
benchmark-on-device (resnet50, mps, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:de... / mobile-job (ios)	1s
benchmark-on-device (mv3, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / mobile-job (ios)	1s
benchmark-on-device (Qwen/Qwen3-0.6B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, arn... / mobile-job (ios)	1s
benchmark-on-device (meta-llama/Llama-3.2-1B, llama3_coreml_ane, apple_iphone_15+ios_18, arn:aws:... / mobile-job (ios)	1s
benchmark-on-device (mv2, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	1s
benchmark-on-device (w2l, coreml_fp16, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)	1s
benchmark-on-device (google/gemma-3-1b-it, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15... / mobile-job (ios)	1s
benchmark-on-device (ic4, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / mobile-job (ios)	1s
benchmark-on-device (w2l, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:308535385114:... / mobile-job (ios)	1s
benchmark-on-device (resnet50, xnnpack_q8, apple_iphone_15, arn:aws:devicefarm:us-west-2:30853538... / mobile-job (ios)	1s
benchmark-on-device (meta-llama/Llama-3.2-1B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone... / mobile-job (ios)	1s
upload-benchmark-results	1d 0h 0m 0s
	4d 0h 37m 31s

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

apple-perf #80

Usage

apple-perf #80

Uh oh!

Run time