Whisper.cpp Benchmark Report: Complete Performance Analysis on Legacy Hardware (Intel Core i5-460M)

[di-halt]

Whisper.cpp Benchmark Report: Complete Performance Analysis on Legacy Hardware (Intel Core i5-460M)

1. Introduction

This report provides a comprehensive and exhaustive benchmark of whisper.cpp performance on a legacy Intel Core i5-460M (Arrandale) mobile processor from 2010. The test covers all major model sizes (tiny to large-v3) and all relevant quantization types (q2_k, q3_k, q4_0, q4_1, q4_k, q5_0, q5_1, q5_k, q6_k, q8_0, f16). The goal is to empirically determine the optimal configuration for speed, accuracy, and disk footprint on old, resource-constrained hardware, and to identify which formats are beneficial and which are detrimental.

2. Test Environment

• System: ASUS K52F Notebook
• CPU: Intel Core i5-460M (Arrandale, 32nm)
  • Cores/Threads: 2C / 4T
  • Base/Turbo Frequency: 2.53 GHz / 2.8 GHz
  • SIMD Instructions: SSE4.2 (No AVX/AVX2 support)
• RAM: 8 GB DDR3-1066 (Dual Channel)
• OS & Compiler: Windows 10 / MSYS2 MinGW64, compiled with -march=native -O3
• Software: whisper.cpp (latest master as of testing)
• Audio Sample: samples/jfk.wav (11.0 seconds, 16-bit PCM, 16 kHz Mono, clean English speech)

3. Results: All Models & All Quantizations

The table below shows the total processing time, real-time factor (RTF), and subjective transcription quality. Lower Total Time and higher RTF are better. Models that failed or produced unacceptable output are explicitly marked.

Model	Quant	Size (MB)	Total Time (s)	Real-time (RTF)	Quality	Verdict
tiny-ml	q4_0	25.3	6.97	1.58x	Excellent	🏆 KING OF SPEED.
tiny-ml	q8_0	43.0	7.18	1.53x	Excellent	Slightly larger, slower than q4_0.
tiny-ml	q4_1	27.6	7.46	1.47x	Excellent	Slower than q4_0, zero quality gain.
tiny-ml	f16	77.7	25.76	0.43x	Excellent	Avoid: Use only for conversion.
tiny-ml	q5_0	29.3	20.64	0.53x	Good	TERRIBLE: 3x slower than q4_0.
tiny-ml	q5_1	31.6	20.84	0.53x	Good	TERRIBLE: 3x slower than q4_0.
tiny-ml	q2_k	-	-	-	CRASH	Incompatible file format.
tiny-ml	q3_k	-	-	-	CRASH	Incompatible file format.
tiny-ml	q4_k	-	-	-	CRASH	Incompatible file format.
tiny-ml	q5_k	-	-	-	CRASH	Incompatible file format.
tiny-ml	q6_k	-	-	-	CRASH	Incompatible file format.
base-ml	q4_0	46.5	12.86	0.86x	Excellent	🏆 BEST BALANCE.
base-ml	q8_0	81.8	13.49	0.82x	Excellent	q4_0 is strictly better here.
base-ml	q4_1	50.9	14.59	0.75x	Excellent	Avoid: Slower than q4_0.
base-ml	q3_k	37.5	20.26	0.54x	Good	Space-saver only. Slower than q4_0.
base-ml	q5_0	54.7	48.36	0.23x	Good	AWFUL: 3.7x slower than q4_0.
base-ml	q5_1	59.1	46.65	0.24x	Good	AWFUL: 3.6x slower than q4_0.
base-ml	q4_k	46.0	19.67	0.56x	Good	Worse than standard q4_0.
base-ml	q5_k	54.7	21.60	0.51x	Good	Worse than q4_0.
base-ml	q6_k	64.1	19.61	0.56x	Good	Worse than q4_0.
base-ml	q2_k	29.3	90.90	0.12x	GARBAGE	HALLUCINATION. Output was "I, I, I, I..."
base-ml	f16	147.9	59.91	0.18x	Excellent	Avoid.
small-ml	q4_0	145.5	39.65	0.28x	Excellent	🏆 KING OF MID-RANGE.
small-ml	q8_0	264.5	46.74	0.24x	Excellent	q4_0 is faster and much smaller.
small-ml	q4_1	160.3	48.70	0.23x	Excellent	Avoid.
small-ml	q2_k	89.1	84.84	0.13x	Acceptable	Works, but slow. Space-saver only.
small-ml	q5_0	174.6	186.83	0.06x	Good	TERRIBLE: 4.7x slower than q4_0.
small-ml	q5_1	189.5	182.88	0.06x	Good	TERRIBLE: 4.6x slower than q4_0.
small-ml	f16	487.6	248.37	0.04x	Excellent	Avoid.
medium-ml	q4_0	444.5	113.85	0.096x	Excellent	🏆 KING OF HIGH-END.
medium-ml	q8_0	822.8	131.90	0.083x	Excellent	q4_0 is faster and half the size.
medium-ml	q4_1	491.9	145.10	0.076x	Excellent	Avoid.
medium-ml	q2_k	266.3	263.20	0.042x	Acceptable	Works, but very slow.
medium-ml	q5_0	538.6	616.43	0.018x	Good	CATASTROPHIC: 5.4x slower than q4_0.
medium-ml	q5_1	585.9	622.59	0.018x	Good	CATASTROPHIC: 5.5x slower than q4_0.
medium-ml	f16	1533.8	791.73	0.014x	Excellent	Avoid.
large-v3-turbo	q4_0	474.0	142.16	0.077x	Excellent	Best quality for complex audio.
large-v3-turbo	q8_0	874.2	178.75	0.062x	Excellent	q4_0 is faster and half the size.
large-v3-turbo	q4_1	524.0	231.19	0.048x	Excellent	Avoid.
large-v3-turbo	f16	1624.6	1285.57	0.009x	Excellent	Avoid.
large-v3-ru	q4_0	889.3	220.03	0.050x	Excellent	High quality, but very slow on this CPU.
large-v3-ru	q8_0	1656.5	256.90	0.043x	Excellent	Avoid q8_0; use q4_0 if required.
large-v3-ru	q4_1	985.2	436.91	0.025x	Excellent	Avoid.
whisper-small-ru-pruned-ft	all	N/A	FAILED	N/A	CRASH	CRITICAL FAILURE: GGML_ASSERT. Incompatible. DO NOT USE.
whisper-medium-ru (Fake)	all	N/A	N/A	N/A	GARBAGE	Fake Russian model. Produced hallucinated subtitles.
whisper-podlodka-turbo-ru	all	N/A	N/A	N/A	N/A	Fake Russian model. Renamed large-v3-turbo-platinum-ml.

4. Analysis: The Good, The Bad, and The Ugly Formats

Based on the comprehensive data, we can definitively categorize the quantization types:

The Good (Always Use These):

• q4_0: UNDISPUTED CHAMPION. Fastest inference time on legacy hardware across all model sizes. Provides excellent quality identical to higher bitrates. The reduced memory bandwidth requirement is the key to its success on DDR3 systems. This is the only quantization you need.
• q8_0: Reliable and high quality, but strictly worse than q4_0 on this hardware. It is larger and slower with zero quality benefit for transcription. Use only if q4_0 is unavailable.
• q3_k (Situational): Viable for extreme disk space saving on the base model. It offers acceptable quality but is significantly slower than q4_0.

The Bad & The Ugly (Avoid At All Costs):

• q5_0, q5_1: CATASTROPHICALLY SLOW. On this CPU, 5-bit formats are 3x to 5.5x slower than q4_0. The added computational overhead of unpacking completely destroys performance. NEVER USE THESE.
• q4_1: Consistently slower than q4_0 on all models with no quality gain. Obsolete.
• q4_k, q5_k, q6_k: These "K-quants" provide no performance benefit over standard q4_0 on this legacy architecture. They are often slower and larger.
• q2_k: Extreme compression leads to model hallucination and gibberish on base and tiny. It is marginally usable on small and medium but remains very slow. Avoid.
• f16: Only useful as a source file for conversion to q4_0. Never use for inference.

5. Final Recommendations for Legacy Hardware Users

Based on the data, the following models are the only files needed for a complete and optimized speech recognition toolkit on an i5-460M (or similar era) system:

Use Case	Model	Quantization	File Size
Real-time / Fast Drafts	tiny-ml	q4_0	25 MB
High-Quality Balanced	base-ml	q4_0	46 MB
Complex/Noisy Audio	large-v3-turbo	q4_0	474 MB