ml-benchmarks.md

name	Benchmarks
description	Benchmarks for machine learning models and applications in materials science.
tags	Data Code/ML App

Benchmarks

Item (URL)	Description	Tags
Matbench Discovery	Interactive leaderboard for machine learning models on atomistic simulations.	Data/Comp, Code/ML, App
QM/ML Datasets	Curated archive of benchmark datasets (molecules, solids, and liquids) for ML models.	Data/Comp
Minnesota Database 2.0	Curated collection of benchmark datasets for evaluating energetic and structural properties in chemistry and physics.	Data/Comp
JARVIS-Leaderboard	Open benchmarking resource comparing the performance of AI models, DFT codes, and classical force fields against high-quality reference data.	App, Data
LeMaterial	Open-source collaborative dataset and benchmark platform for materials science.	Data
MatTools	Benchmarking Large Language Models for materials science tools.	Data, Code/ML
BOOM	Benchmarking out-of-distribution molecular property predictions of machine learning models.	Data, Code/ML
ZEBRA-dataset	Materials Project dataset for evaluating materials property prediction models.	Data, Code/ML
RADII	Benchmark dataset and evaluation framework for characterizing the extrapolation frontier of graph generative models.	Data, Code/ML
quantum-machine.org	Databases and benchmarks for machine learning in quantum chemistry.	Data/Comp, App
ChemBench	Systematic evaluation of chemistry knowledge in large language models.	Data
MaCBench	Multimodal reasoning benchmark for materials science and chemistry.	Data, Code/ML
PDEBench	Extensive benchmark suite for scientific machine learning on partial differential equations.	Data, Code/ML
VaspAgent_with_Benchmark	Agentic framework for autonomous materials computation and evaluation.	Code/WF, Code/ML
PhononBench	Phonon-based benchmark for dynamical stability evaluation of AI-generated crystals.	Data/Comp, Code/ML
AtomWorldBench	Benchmark evaluating spatial reasoning and structural modification of CIF files by large language models.	Data, Code/ML
MADE	Benchmarking environment for agentic systems in closed-loop materials discovery.	Code/WF, Code/ML
PRBench-Eval-Handson	Evaluation harness benchmarking artificial intelligence agents on reproducing scientific research.	Code/WF, Code/ML
SciConvBench	Conversational benchmark evaluating large language models on multi-turn task clarification for computational science.	Data, Code/ML
MatFormBench	Benchmarking ecosystem evaluating generative strategies for target-driven formulation.	Data, Code/ML
WF-Bench	Benchmark dataset evaluating the expressivity and scaling laws of neural-network wavefunctions.	Data/Comp, Code/ML
ML-PEG	Interactive benchmarking and extrapolation guide for machine learning interatomic potentials.	App, Code/ML
SciVerseGym	Gymnasium-compatible environment for reinforcement learning and Bayesian optimization in crystal discovery.	Code/WF, Code/ML
physci-deepresearch	Benchmark evaluating LLM agent systems on multi-step scientific reasoning tasks in physics and chemistry.	Code/WF, Code/ML
UniFFBench	Evaluation of universal machine learning force fields against experimental materials data.	Data, Code/ML
incarbench	Benchmark evaluating LLMs on generating and repairing VASP INCAR configuration files.	Data, Code/ML
MatSciFig	Multimodal dataset of panel-level images and scientific text extracted from materials science literature.	Data
MaterialScope	Object detection dataset for layout analysis and panel localization in materials science figures.	Data
TheNanotechnologyMolecularOptimizationBenchmark	Benchmark suite evaluating generative molecular design models on nanotechnology applications.	Data, Code/ML
Charlotte Electride Database	Database of computationally screened inorganic electride materials.	Data/Comp, App
MatPhaseBench	Semantics-guided benchmark evaluating vision-language models on materials phase diagrams understanding.	Data, Code/ML
MatPES	Potential energy surface dataset computed using PBE and r2SCAN functionals for machine learning interatomic potentials.	App, Data/Comp, Code/ML