Skip to content

Commit 3b27657

Browse files
Remove stale QNN static llama eval job (#20884)
Summary: - Remove the `test-static-llama-qnn-eval-linux` trunk workflow job. - Delete `.ci/scripts/test_qnn_static_llama_eval.sh`, which only called the deleted `executorch.examples.qualcomm.oss_scripts.llama.eval_llama_qnn` entrypoint. - Existing QNN/static LLM coverage remains on the current `llama.py` path via `test_qnn_static_llm.sh` and `TestExampleLLMScript.test_static_llm_model`. Context: - Trunk job `test-static-llama-qnn-eval-linux (baseline, 62) / linux-job` is failing with `No module named executorch.examples.qualcomm.oss_scripts.llama.eval_llama_qnn`. Test Plan: - `grep -RIn "test_qnn_static_llama_eval.sh\|eval_llama_qnn\|static-llama-qnn-eval" .github .ci examples backends` returns no matches. - `git diff --check` Notes: - Local YAML parse check was not run because PyYAML is not installed in this environment. - Commit hook skipped lintrunner because lintrunner is not installed locally.
1 parent be9b80c commit 3b27657

2 files changed

Lines changed: 0 additions & 128 deletions

File tree

.ci/scripts/test_qnn_static_llama_eval.sh

Lines changed: 0 additions & 90 deletions
This file was deleted.

.github/workflows/trunk.yml

Lines changed: 0 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -997,44 +997,6 @@ jobs:
997997
- 'examples/qualcomm/**'
998998
- 'examples/models/llama/**'
999999
1000-
test-static-llama-qnn-eval-linux:
1001-
needs: changes # has dependency on changes jobs defined above
1002-
if: needs.changes.outputs.qnn == 'true'
1003-
name: test-static-llama-qnn-eval-linux
1004-
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
1005-
permissions:
1006-
id-token: write
1007-
contents: read
1008-
strategy:
1009-
fail-fast: false
1010-
matrix:
1011-
config:
1012-
- name: "baseline"
1013-
flags: ""
1014-
threshold: 62.0
1015-
with:
1016-
runner: linux.2xlarge
1017-
docker-image: ci-image:executorch-ubuntu-22.04-qnn-sdk
1018-
submodules: 'recursive'
1019-
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
1020-
timeout: 180
1021-
script: |
1022-
# The generic Linux job chooses to use base env, not the one setup by the image
1023-
CONDA_ENV=$(conda env list --json | jq -r ".envs | .[-1]")
1024-
conda activate "${CONDA_ENV}"
1025-
BUILD_TOOL="cmake"
1026-
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-qnn-deps.sh
1027-
PYTHON_EXECUTABLE=python bash .ci/scripts/build-qnn-sdk.sh
1028-
# Setup executorch
1029-
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-linux.sh --build-tool "${BUILD_TOOL}"
1030-
# Setup install_requirements for llama
1031-
PYTHON_EXECUTABLE=python bash examples/models/llama/install_requirements.sh
1032-
1033-
echo ">>> Running config: ${{ matrix.config.name }}"
1034-
PYTHON_EXECUTABLE=python bash .ci/scripts/test_qnn_static_llama_eval.sh \
1035-
--flags "${{ matrix.config.flags }}" \
1036-
--threshold "${{ matrix.config.threshold }}"
1037-
10381000
unittest-release:
10391001
uses: ./.github/workflows/_unittest.yml
10401002
permissions:

0 commit comments

Comments
 (0)