Skip to content

Commit 62bd46e

Browse files
martinlsmjpiat
authored andcommitted
Arm backend: Add evaluate_model.py (pytorch#18199)
Arm backend: Add evaluate_model.py This patch reimplements the evaluation feature that used to be in aot_arm_compiler.py while introducing a few improvements. The program is evaluate_model.py and it imports functions from aot_arm_compiler.py to compile a model in a similar manner, but runs its own code that is focused on evaluating a model using the evaluators classes in backends/arm/util/arm_model_evaluator.py. The following is supported in evaluate_model.py: - TOSA reference models (INT, FP). - Evaluating a model that is quantized and/or lowered. I.e., it is possible to evaluate a model that is quantized but not lowered, lowered but not quantized, or both at the same time. - The program can cast the model with the --dtype flag to evaluate a model in e.g., bf16 or fp16 format. Also add tests that exercise evaluate_model.py with different command line arguments.
1 parent e08ef53 commit 62bd46e

4 files changed

Lines changed: 469 additions & 1 deletion

File tree

backends/arm/scripts/aot_arm_compiler.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -691,7 +691,7 @@ def _get_args():
691691
if args.evaluate is not None or args.evaluate_config is not None:
692692
logging.error(
693693
"Model evaluation is no longer supported in this script."
694-
" Ignore and continue."
694+
" Use evaluate_model.py instead. Ignore and continue."
695695
)
696696

697697
return args

0 commit comments

Comments
 (0)