Inquiry About the Overall Cost of Running the Full Evaluation with GPT-4o

[ligeng0197]

Hi, thanks for the excellent work and open-sourcing the test set!

I noticed that the evaluation process involves using GPT-4o to parse or generate answers. Could you please provide an estimate of the total cost (e.g., number of API calls or approximate dollar amount) for running the full evaluation on the entire test set?

This would help me better understand the computational and financial resources required.

Thanks a lot!

[mxin262]

Hi, thank you for your interest. Running the full evaluation on the entire test set will cost approximately $0.42.