Skip to content

Commit 53180df

Browse files
authored
1 parent 4f3f8ed commit 53180df

2 files changed

Lines changed: 91 additions & 3 deletions

File tree

examples/arm/QAT_example/README.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Practical-RIFE Accuracy Flow
2+
3+
`qat_loop.py` measures Practical-RIFE eager, PTQ, and QAT accuracy on real
4+
frame triples. It is intended as the accuracy companion to VGF performance
5+
measurements, so each reported artifact can be tied back to a reproducible
6+
model, ExecuTorch revision, and export command.
7+
8+
## Inputs
9+
10+
Use a standard Practical-RIFE checkout with this layout:
11+
12+
```text
13+
<model-root>/train_log/IFNet_HDv3.py
14+
<model-root>/train_log/flownet.pkl
15+
```
16+
17+
The triples file must identify frame interpolation samples in this order:
18+
19+
```text
20+
name,input0,input1,target
21+
sample_00000,/path/frame_0.png,/path/frame_2.png,/path/frame_1.png
22+
```
23+
24+
The script treats `input0` and `input1` as the two model inputs and `target`
25+
as the expected middle frame.
26+
27+
## Real Accuracy Run
28+
29+
Use real frame triples for accuracy reporting. This example compares eager,
30+
PTQ, and QAT at the default 768x384 shape while preserving uint8 image IO:
31+
32+
```bash
33+
python examples/arm/QAT_example/qat_loop.py \
34+
--triples-list <path-to-real-triples.csv> \
35+
--model-root <path-to-Practical-RIFE> \
36+
--height 768 \
37+
--width 384 \
38+
--max-triples 10 \
39+
--calibration-samples 8 \
40+
--mode all \
41+
--io-quantization uint8 \
42+
--qat-samples 8 \
43+
--qat-steps 50 \
44+
--qat-lr 1e-5 \
45+
--output-dir out/rife_accuracy_10_real_triples_qat50
46+
```
47+
48+
The output directory contains:
49+
50+
- `report.md`: human-readable aggregate metrics.
51+
- `metrics.json`: aggregate and per-sample metrics.
52+
- `metrics.csv`: tabular per-sample metrics.
53+
- `quantization_coverage/`: PTQ and QAT graph coverage details.
54+
55+
## Smoke Runs
56+
57+
Small random or short QAT runs are useful only for checking that the flow runs.
58+
Do not use them as model-quality evidence. For example, two samples and three
59+
QAT steps can produce very low PSNR while still proving that eager, PTQ, and
60+
QAT execution is wired correctly.
61+
62+
## Reference Samples
63+
64+
Keep reference images outside this repository and pass them through
65+
`--triples-list`. For internal comparisons, use a fixed triples list and keep
66+
the image bundle unchanged across graph and quantization experiments.
67+
68+
The triples list is the reproducibility contract. It should use repository- or
69+
bundle-relative paths where possible, so another user can unpack the same image
70+
bundle and run the same command without editing absolute paths.
71+
72+
## Reporting Checklist
73+
74+
When sharing accuracy or performance data, include:
75+
76+
- Practical-RIFE commit hash.
77+
- ExecuTorch commit hash.
78+
- `qat_loop.py` command line and flags.
79+
- Dataset or triples-list identity.
80+
- Input shape and preprocessing mode.
81+
- IO quantization mode.
82+
- PTQ calibration sample count.
83+
- QAT sample count, step count, and learning rate.
84+
- Reference image bundle identity.
85+
- Whether a VGF or PTE was exported from the same run.
86+
87+
This keeps accuracy numbers comparable with board performance results and
88+
avoids mixing smoke-test numbers with real-data validation.

examples/arm/QAT_example/qat_loop.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,14 +7,14 @@
77
"""Measure Practical-RIFE PTQ and QAT accuracy on real frame triples.
88
99
The script is intended as a small handover flow for comparing eager FP32,
10-
PTQ, and QAT PT2E models before generating customer VGF artifacts. The
11-
expected triple order is:
10+
PTQ, and QAT PT2E models before generating VGF artifacts. The expected
11+
triple order is:
1212
1313
* first frame: model input 0
1414
* third frame: model input 1
1515
* middle frame: quality target
1616
17-
By default the fixed input shape is 768x384, matching the OPPO request.
17+
By default the fixed input shape is 768x384.
1818
"""
1919

2020
from __future__ import annotations

0 commit comments

Comments
 (0)