apple-perf

apple-perf #209

Sign in to view logs

Triggered via schedule November 15, 2025 02:20

main

Status Failure

Total duration 4d 0h 4m 52s

Artifacts –

apple-perf.yml

on: schedule

Matrix: export-models

Matrix: prepare-test-specs

build-benchmark-app / macos-job

upload-benchmark-app

Matrix: benchmark-on-device

upload-benchmark-results

Annotations

101 errors

build-benchmark-app / macos-job

The job was not started because recent account payments have failed or your spending limit needs to be increased. Please check the 'Billing & plans' section in your settings

prepare-test-specs (meta-llama/Llama-3.2-1B, llama3_fb16, apple_iphone_15, public, arn:aws:device...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (mobilebert, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8, llama3_spinquant, apple_...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (ic3, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114:...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (ic4, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (resnet50, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (ic3, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (meta-llama/Llama-3.2-1B, llama3_coreml_ane, apple_iphone_15, ios_18_public, a...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (ic4, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114:...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (google/gemma-3-1b-it, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15,...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (ic4, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (resnet50, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853538...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (edsr, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (mv2, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114:...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (resnet50, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (Qwen/Qwen3-0.6B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, publ...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (ic3, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8, llama3_qlora, apple_iphone_1...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (mobilebert, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (w2l, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114:...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (mv2, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (Qwen/Qwen3-0.6B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, public, a... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (mv3, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (meta-llama/Llama-3.2-1B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (edsr, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (mv3, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (mv2, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853538511... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (w2l, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085353851... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (w2l, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (Qwen/Qwen3-0.6B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, publ...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (ic3, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085353851... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (HuggingFaceTB/SmolLM2-135M, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_ipho...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (mv3, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114:...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (resnet50, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114:... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (meta-llama/Llama-3.2-1B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (ic4, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085353851... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (mv3, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (mv2, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (w2l, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (ic3, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853538511... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

prepare-test-specs (allenai/OLMo-1B-hf, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, p...

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8, llama3_spinquant, apple_iphon... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (Qwen/Qwen3-0.6B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, public, a... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (meta-llama/Llama-3.2-1B, llama3_coreml_ane, apple_iphone_15, ios_18_public, arn:aw... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (w2l, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853538511... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (mobilebert, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (ic3, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8, llama3_qlora, apple_iphone_15, pu... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (google/gemma-3-1b-it, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, publ... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (meta-llama/Llama-3.2-1B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, p... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (mv3, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085353851... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (edsr, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085353851... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (ic4, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (mobilebert, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (resnet50, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (HuggingFaceTB/SmolLM2-135M, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (mv2, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085353851... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (edsr, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (meta-llama/Llama-3.2-1B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, p... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (allenai/OLMo-1B-hf, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, public... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (w2l, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (mv3, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853538511... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (resnet50, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (mv2, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114:devic... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (ic4, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853538511... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

export-models (meta-llama/Llama-3.2-1B, llama3_fb16, apple_iphone_15, public, arn:aws:devicefarm:... / macos-job

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (meta-llama/Llama-3.2-1B, llama3_coreml_ane, apple_iphone_15, ios_18_public, ... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (mobilebert, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (resnet50, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (allenai/OLMo-1B-hf, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, ... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (edsr, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (mobilebert, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (Qwen/Qwen3-0.6B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, pub... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (resnet50, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085353... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (w2l, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (mv3, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (w2l, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (meta-llama/Llama-3.2-1B, llama3_fb16, apple_iphone_15, public, arn:aws:devic... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (meta-llama/Llama-3.2-1B, et_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (mv2, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (resnet50, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (google/gemma-3-1b-it, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (w2l, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (ic4, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (mv3, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8, llama3_spinquant, apple... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (ic4, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (ic4, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (Qwen/Qwen3-0.6B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone_15, pub... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (meta-llama/Llama-3.2-1B, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iphone... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (HuggingFaceTB/SmolLM2-135M, hf_xnnpack_custom_spda_kv_cache_8da4w, apple_iph... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (mv2, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (mv3, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (edsr, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (mv2, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (ic3, xnnpack_q8, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:30853... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (ic3, coreml_fp16, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:3085... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (ic3, mps, apple_iphone_15, public, arn:aws:devicefarm:us-west-2:308535385114... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

benchmark-on-device (meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8, llama3_qlora, apple_iphone_... / mobile-job (ios)

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s

upload-benchmark-results

The job has exceeded the maximum execution time while awaiting a runner for 24h0m0s