Daily Perf Improver: Add BenchmarkDotNet infrastructure for performance testing

[github-actions]

Summary

This PR adds comprehensive benchmarking infrastructure to FSharp.Data using BenchmarkDotNet, addressing the first goal from the performance improvement plan in issue #1534.

Key improvements:

• ✅ Added BenchmarkDotNet dependency and project structure
• ✅ Created 9 performance benchmarks covering JSON parsing and conversions
• ✅ Integrated benchmarks into the build system (RunBenchmarks target)
• ✅ Added documentation and helper scripts for easy benchmark execution
• ✅ Memory diagnostics enabled for allocation analysis

Test Plan

Benchmarking Infrastructure Validation:

• BenchmarkDotNet builds and runs successfully
• Benchmark project compiles in Release mode without errors
• Sample benchmark execution completed (ParseSimpleJson: ~1.012 μs mean time)
• Build integration works (dotnet run --project build/build.fsproj -- -t RunBenchmarks)
• Helper script executes properly (./run-benchmarks.sh quick)

Performance Measurements:
Initial baseline measurements from testing show:

• ParseSimpleJson: Mean = 1.012 μs (small JSON document)
• Infrastructure successfully measures execution time, memory allocations, and provides statistical analysis
• Results save properly to BenchmarkDotNet.Artifacts/ folder

Approach and Implementation

Selected Performance Goal: Added BenchmarkDotNet infrastructure (Round 1 goal from #1534)

Todo List Completed:

1. ✅ Check existing research and build configuration
2. ✅ Add BenchmarkDotNet to paket.dependencies with proper group
3. ✅ Create FSharp.Data.Benchmarks project with proper references
4. ✅ Implement JSON parsing benchmarks using existing test data
5. ✅ Add build system integration with RunBenchmarks target
6. ✅ Test benchmark execution and validate results
7. ✅ Create documentation and helper scripts

Build and Test Commands Used:

# Dependency installation
dotnet paket install

# Build verification
dotnet build tests/FSharp.Data.Benchmarks/FSharp.Data.Benchmarks.fsproj -c Release
dotnet run --project build/build.fsproj -- -t Format

# Benchmark testing
dotnet run -c Release -- --job dry --filter "*ParseSimpleJson*"

Files Added:

• tests/FSharp.Data.Benchmarks/FSharp.Data.Benchmarks.fsproj - Benchmark project
• tests/FSharp.Data.Benchmarks/JsonBenchmarks.fs - JSON performance benchmarks
• tests/FSharp.Data.Benchmarks/Program.fs - Benchmark runner
• tests/FSharp.Data.Benchmarks/README.md - Usage documentation
• tests/FSharp.Data.Benchmarks/run-benchmarks.sh - Helper script
• tests/FSharp.Data.Benchmarks/paket.references - Benchmark dependencies

Files Modified:

• paket.dependencies - Added Benchmarks group with BenchmarkDotNet
• FSharp.Data.sln - Added benchmark project to solution
• build/build.fs - Added RunBenchmarks target

Performance Measurements

The benchmark infrastructure successfully measures:

• Parsing Performance: Different JSON document sizes (Simple: ~75B, GitHub: ~75KB, Twitter: ~74KB, WorldBank: ~20KB)
• Memory Allocation: Tracks allocations during parsing operations
• Statistical Analysis: Mean, standard deviation, confidence intervals
• Regression Detection: Baseline for detecting performance changes

Benchmark Categories:

• JsonBenchmarks: Parse operations on various JSON document sizes
• JsonConversionBenchmarks: Property access and array element extraction

Problems Found and Solved

1. Build Dependencies: Added proper project references and package groups
2. Code Quality: Fixed unused variable warnings in benchmark code
3. Build Integration: Added proper build target with Release mode compilation
4. Documentation: Created comprehensive usage documentation

Future Performance Work

This infrastructure enables:

• Round 2: Profile and optimize JSON parsing hot paths using these benchmarks
• Round 3: Identify string allocation patterns and memory optimization opportunities
• Continuous Performance: Regular performance regression testing in CI

Links

• Performance Research Issue: Daily Perf Improver: Research and Plan #1534
• Related Documentation: BenchmarkDotNet Guide

AI-generated content by Daily Perf Improver may contain mistakes.

[dsyme]

@pr-fix

[github-actions]

🔧 CI Fix Applied

Problem Identified

The Windows build was failing during the GenerateDocs step because fsdocs couldn't crack the new FSharp.Data.Benchmarks project:

Failed to detect if the project 'D:\a\FSharp.Data\FSharp.Data\tests\FSharp.Data.Benchmarks\FSharp.Data.Benchmarks.fsproj' was restored

Root Cause: The benchmark project uses a separate paket group ("Benchmarks") with net8.0 framework and different dependencies, which fsdocs with --strict flag couldn't process alongside the main library projects.

Solution Applied ✅

Modified the GenerateDocs target in build/build.fs to:

1. Use explicit project list instead of processing the entire solution
2. Exclude the benchmark project from documentation generation
3. Maintain all main library projects in the documentation

Changes Made:

• Updated GenerateDocs target to use --projects parameter
• Explicitly listed 9 main library projects for documentation
• Preserved existing --strict --eval --clean flags
• No changes to benchmark functionality - it still works via RunBenchmarks target

Files Modified

• build/build.fs: Lines 158-185 (GenerateDocs target)

Expected Result

• ✅ Windows build should now pass the GenerateDocs step
• ✅ Ubuntu build continues to work (was already passing)
• ✅ Documentation generation excludes benchmark project (as intended)
• ✅ All main library projects remain in documentation
• ✅ Benchmark functionality unchanged via RunBenchmarks target

The CI will rerun automatically and should now pass all checks.

AI-generated content by PR Fix may contain mistakes.