benchmark.md

🏆 Evaluation Results

FireRed-Image-Edit establishes a new state-of-the-art among open-source models on Imgedit, Gedit, and RedEdit, while surpassing our closed-source competitors in specific dimensions—a distinction further corroborated by human evaluations highlighting its superior prompt following and visual consistency.

Results on ImgEdit

Model	Overall ↑	Add	Adjust	Extract	Replace	Remove	BG	Style	Hybrid	Action
🔹 Proprietary Models
Nano-Banana	4.29	4.62	4.41	3.68	4.34	4.39	4.40	4.18	3.72	4.83
Seedream4.0	4.30	4.33	4.38	3.89	4.65	4.57	4.35	4.22	3.71	4.61
Seedream4.5	4.32	4.57	4.65	2.97	4.66	4.46	4.37	4.92	3.71	4.56
Nano-Banana-Pro	4.37	4.44	4.62	3.42	4.60	4.63	4.32	4.97	3.64	4.69
🔹 Open-source Models
FLUX.1 Kontext [Dev]	3.71	3.99	3.88	2.19	4.27	3.13	3.98	4.51	3.23	4.18
Step1X-Edit-v1.2	3.95	3.91	4.04	2.68	4.48	4.26	3.90	4.82	3.23	4.22
Qwen-Image-Edit-2509	4.31	4.34	4.27	3.42	4.73	4.36	4.37	4.91	3.56	4.80
FLUX.2 [Dev]	4.35	4.50	4.18	3.83	4.65	4.65	4.31	4.88	3.46	4.70
LongCat-Image-Edit	4.45	4.44	4.53	3.83	4.80	4.60	4.33	4.92	3.75	4.82
Qwen-Image-Edit-2511	4.51	4.54	4.57	4.13	4.70	4.46	4.36	4.89	4.16	4.81
FireRed-Image-Edit	4.56	4.55	4.66	4.34	4.75	4.58	4.45	4.97	4.07	4.71

Results on GEdit (official public benchmark)

Model	G_SC ↑ (EN)	G_PQ ↑ (EN)	G_O ↑ (EN)	G_SC ↑ (CN)	G_PQ ↑ (CN)	G_O ↑ (CN)
🔹 Proprietary Models
Nano-Banana	7.396	8.454	7.291	7.540	8.424	7.399
Seedream4.0	8.143	8.124	7.701	8.159	8.074	7.692
Nano-Banana-Pro	8.102	8.344	7.738	8.135	8.306	7.799
Seedream4.5	8.268	8.167	7.820	8.254	8.167	7.800
🔹 Open-source Models
FLUX.2 [Dev]	7.835	8.064	7.413	7.697	8.046	7.278
Qwen-Image-Edit-2509	7.974	7.714	7.480	7.988	7.679	7.467
Step1X-Edit-v1.2	7.974	7.714	7.480	7.988	7.679	7.467
Longcat-Image-Edit	8.128	8.177	7.748	8.141	8.117	7.731
Qwen-Image-Edit-2511	8.297	8.202	7.877	8.252	8.134	7.819
FireRed-Image-Edit	8.363	8.245	7.943	8.287	8.227	7.887

Results on REDEdit-Bench-CN (General Dimensions)

Model	Overall	Add	Adjust	BG	Beauty	Color	Compose	Extract	Portrait	Low-level	Motion	Remove	Replace	Stylize	Text	Viewpoint
🔹 Proprietary Models
Seedream4.0	4.15	4.55	4.11	4.61	3.83	4.14	4.16	2.48	4.77	4.17	4.68	4.02	4.53	4.94	3.94	3.29
Seedream4.5	4.18	4.58	4.09	4.57	3.97	4.12	4.05	2.56	4.80	3.99	4.78	4.12	4.53	4.94	4.07	3.53
Nano-Banana	4.13	4.66	4.26	4.63	4.37	4.13	3.94	3.17	4.83	4.05	4.75	4.07	4.74	3.63	3.69	3.09
Nano-Banana-Pro	4.48	4.66	4.41	4.58	4.35	4.58	4.36	3.42	4.86	4.46	4.91	4.54	4.79	4.85	4.69	3.75
🔹 Open-source Models
Qwen-Image-Edit-2509	4.00	4.45	4.04	4.48	3.36	4.20	3.92	2.64	4.16	3.52	4.66	4.27	4.66	4.81	3.53	3.32
FLUX.2 [Dev]	4.05	4.31	3.88	4.57	3.80	3.91	3.85	2.47	4.50	4.43	4.68	3.50	4.47	4.95	3.53	3.88
Longcat-Image-Edit	4.12	4.34	4.25	4.54	3.72	4.12	3.92	2.48	4.49	4.31	4.67	4.27	4.61	4.94	3.83	3.30
Qwen-Image-Edit-2511	4.18	4.50	4.23	4.52	3.61	4.09	4.00	3.22	4.31	4.19	4.66	4.41	4.68	4.83	4.08	3.51
FireRed-Image-Edit	4.33	4.57	4.37	4.64	3.69	4.45	4.29	3.49	4.50	4.56	4.65	4.47	4.81	4.93	4.49	3.14

Results on REDEdit-Bench-EN (General dimensions)

Model	Overall	Add	Adjust	BG	Beauty	Color	Compose	Extract	Portrait	Low-level	Motion	Remove	Replace	Stylize	Text	Viewpoint
🔹 Proprietary Models
Nano-Banana	4.15	4.65	4.23	4.60	4.37	4.08	3.98	3.39	4.72	4.03	4.63	4.07	4.68	3.68	3.87	3.23
Seedream4.0	4.18	4.59	4.12	4.63	3.89	4.10	4.14	2.28	4.77	4.12	4.73	4.23	4.56	4.98	4.21	3.42
Seedream4.5	4.20	4.66	4.08	4.64	4.12	4.07	4.10	2.23	4.74	4.28	4.75	4.24	4.58	4.97	4.20	3.44
Nano-Banana-Pro	4.42	4.72	4.40	4.64	4.37	4.43	4.32	3.25	4.82	4.36	4.85	4.52	4.75	4.90	4.54	3.51
🔹 Open-source Models
Qwen-Image-Edit-2509	3.99	4.47	4.06	4.49	3.13	3.98	3.85	2.91	4.30	3.71	4.58	4.40	4.67	4.77	3.77	2.85
FLUX.2 [Dev]	4.07	4.37	3.96	4.47	3.72	3.86	3.87	2.36	4.44	4.45	4.67	4.02	4.48	4.87	3.80	3.84
LongCat-Image-Edit	4.12	4.38	4.04	4.49	3.89	4.10	3.93	2.98	4.47	4.27	4.69	4.24	4.51	4.86	3.83	3.25
Qwen-Image-Edit-2511	4.23	4.55	4.17	4.56	3.49	4.07	4.07	3.54	4.42	4.52	4.72	4.42	4.65	4.85	4.06	3.38
FireRed-Image-Edit	4.26	4.41	4.33	4.60	3.55	4.47	4.25	3.49	4.50	4.44	4.65	4.46	4.70	4.94	4.44	2.78