Feature request: benchmark mode

[JorisVanEijden]

Summary

Add a headless "benchmark" mode to the emulator, allowing it to execute a given DOS program as fast as possible without UI rendering, while collecting performance metrics. This will enable easier performance testing, regression detection, and optimization validation.

Use Case & Motivation

• Measure execution speed under different configurations (e.g., changes in CPU emulation, memory management, or instruction handling).
• Detect performance regressions when modifying the emulator.
• Compare execution time across different versions of the emulator.
• Automate benchmarking for continuous performance tracking.

Proposed Implementation

1. Command-Line Mode:

   • Introduce a CLI flag (e.g., --benchmark <path_to_executable>) to launch the emulator in benchmark mode.
   • Optionally allow specifying parameters such as:
     • --max-instructions <N> to limit execution.
     • --timeout <seconds> to prevent infinite loops.

2. Performance Metrics Collection:

   • Measure total execution time.
   • Count executed instructions.
   • Track cycle counts and key statistics (e.g., opcode distribution, memory access patterns).
   • Optionally output a JSON or CSV report (--output <file>).

3. Exit Conditions:

   • Program terminates normally.
   • Reaches a configured instruction/time limit.
   • Encounters an unsupported instruction or exception.

Optional Enhancements (Future Work)

• Support deterministic replays for comparing different versions.
• Profiling integration (e.g., detailed breakdown of CPU/memory usage).
• Compare multiple emulator builds automatically and report differences.

[maximilien-noal]

An interesting package for this idea: https://github.com/JimmyCushnie/Benchmark-Buddy

[maximilien-noal]

A benchmark tool for emulators : https://www.vogons.org/viewtopic.php?p=1356839#p1356839