feat: add ReVSI evaluation

[eamonn-zh]

Summary

• Add evaluation code for ReVSI benchmark

In scope

• Add a new task revsi

Type of Change

• Bug fix (non-breaking change)
• New feature
• New benchmark/task
• New model integration
• Breaking change
• Documentation update
• Refactoring (no functional changes)

[kcz358]

Hi, thank you for the contributions. For the different frames, I think this should be model parameters adjustment instead of creating extra tasks? So maybe we should only need the all frame yaml if that is the case. And the the aggregation, it would be better if you can add the metrics in the metrics list so it will be logged in the result.json instead just use logger to log the results. Thank you.

[eamonn-zh]

Hi @kcz358, thanks for the suggestions! We have added the metrics to the metrics list and removed the self-logging of results.

Regarding the different frame settings, ReVSI defines four frame-budget subsets. Each subset provides different ground-truth answers to ensure that questions remain answerable under the corresponding frame sampling budget and are not invalidated by missing visual evidence. This frame-adaptive evaluation protocol is an important feature and core contribution of ReVSI, so all four sub-tasks (all-frame, 64-frame, 32-frame and 16-frame) are necessary. Thank you!

[kcz358]

Thanks for the clarification. LGTM