Enable continuous BenchmarkDotNet publishing in CI

[askpt]

Overview

Enable continuous and automated benchmarking for the repository using BenchmarkDotNet.

Proposal

• Implement continuous benchmarking using the martincostello/benchmarkdotnet-results-publisher GitHub Action as a reference.
• There is already a benchmarks project: test/OpenFeature.Benchmarks, which should be used for automated runs.
• Benchmark results may be published as a GitHub Page for visualization and tracking performance over time.
• The benchmarking workflow should be triggered daily and on every merge to the main branch.

Goals

• Set up automated benchmark execution in CI/CD pipeline
• Publish benchmark results using the BenchmarkDotNet results publisher action
• Visualize results via GitHub Pages
• Ensure the workflow runs daily and on merges to main

Reference

• GitHub Action: https://github.com/martincostello/benchmarkdotnet-results-publisher
• Benchmarks project: test/OpenFeature.Benchmarks

[toddbaert]

We tried some benchmarking before and the problem is that GH actions seem too susceptible to noisy neighbor and resource provisioning constraints.

I think we'd need self-hosted runners to get really good performance testing done.

[askpt]

We tried some benchmarking before and the problem is that GH actions seem too susceptible to noisy neighbor and resource provisioning constraints.

I think we'd need self-hosted runners to get really good performance testing done.

@toddbaert: I want to start by having something set up and ready to run on GitHub runners. This will give us a clear baseline to work from. If possible, we can look at migrating to a self-hosted setup later on, but at least we’ll have an initial working solution in place from the start.