Skip to content

test: Benchmark #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nikcio merged 1 commit into from
Mar 14, 2026
Merged

test: Benchmark #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Directory.Packages.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
<CentralPackageTransitivePinningEnabled>false</CentralPackageTransitivePinningEnabled>
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="BenchmarkDotNet" Version="0.15.8" />
<PackageVersion Include="Nikcio.OpenApiCodeGen" Version="1.0.0" />
<PackageVersion Include="coverlet.collector" Version="8.0.0" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.3.0" />
<PackageVersion Include="Microsoft.OpenApi" Version="3.4.0" />
Expand Down
3 changes: 3 additions & 0 deletions OpenApiCodeGenerator.slnx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@
<Project Path="tests/OpenApiCodeGenerator.Tests/OpenApiCodeGenerator.Tests.csproj" />
<Project Path="tests/OpenApiCodeGenerator.IntegrationTests/OpenApiCodeGenerator.IntegrationTests.csproj" />
</Folder>
<Folder Name="/benchmarks/">
<Project Path="benchmarks/OpenApiCodeGenerator.Benchmarks/OpenApiCodeGenerator.Benchmarks.csproj" />
</Folder>
<Folder Name="/examples/">
<Project Path="examples/OpenApiCodeGenerator.Examples/OpenApiCodeGenerator.Examples.csproj" />
</Folder>
Expand Down
31 changes: 31 additions & 0 deletions ...chmarks/BenchmarkDotNet.Artifacts/results/ComparisonBenchmarks-report-github.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
```

BenchmarkDotNet v0.15.8, Windows 11 (10.0.26200.7840/25H2/2025Update/HudsonValley2)
AMD Ryzen 7 7800X3D 4.20GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK 10.0.200-preview.0.26103.119
[Host] : .NET 10.0.3 (10.0.3, 10.0.326.7603), X64 RyuJIT x86-64-v4
DefaultJob : .NET 10.0.3 (10.0.3, 10.0.326.7603), X64 RyuJIT x86-64-v4


```
| Method | Categories | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Gen2 | Allocated | Alloc Ratio |
|--------- |------------------------------- |----------------:|--------------:|---------------:|----------------:|------:|--------:|------------:|----------:|--------:|---------------:|------------:|
| Current | GitHubApiNext,GenerateOnly | 19,590.73 μs | 913.760 μs | 2,694.242 μs | 18,504.19 μs | ? | ? | 687.5000 | 343.7500 | 15.6250 | 37497.21 KB | ? |
| | | | | | | | | | | | | |
| Released | GitHubApiNext,ParseAndGenerate | 4,275,059.67 μs | 82,573.361 μs | 113,027.358 μs | 4,264,655.55 μs | 1.00 | 0.04 | 204000.0000 | 9000.0000 | - | 10055299.83 KB | 1.00 |
| Current | GitHubApiNext,ParseAndGenerate | 758,891.62 μs | 14,788.118 μs | 23,023.320 μs | 754,015.15 μs | 0.18 | 0.01 | 13000.0000 | 8000.0000 | - | 678324.84 KB | 0.07 |
| | | | | | | | | | | | | |
| Current | GitHubApi,GenerateOnly | 15,883.52 μs | 308.995 μs | 317.315 μs | 15,751.93 μs | ? | ? | 718.7500 | 343.7500 | - | 39430.52 KB | ? |
| | | | | | | | | | | | | |
| Released | GitHubApi,ParseAndGenerate | 610,237.06 μs | 14,185.529 μs | 40,928.499 μs | 601,908.25 μs | 1.00 | 0.09 | 8000.0000 | 7000.0000 | - | 435401.99 KB | 1.00 |
Comment on lines +1 to +20
| Current | GitHubApi,ParseAndGenerate | 548,678.89 μs | 10,787.996 μs | 19,175.653 μs | 548,303.45 μs | 0.90 | 0.07 | 8000.0000 | 7000.0000 | - | 412075.61 KB | 0.95 |
| | | | | | | | | | | | | |
| Current | Petstore,GenerateOnly | 18.11 μs | 0.732 μs | 2.053 μs | 17.33 μs | ? | ? | 1.1292 | 0.0305 | - | 56.34 KB | ? |
| | | | | | | | | | | | | |
| Released | Petstore,ParseAndGenerate | 384.84 μs | 7.691 μs | 9.155 μs | 382.91 μs | 1.00 | 0.03 | 17.5781 | 7.8125 | - | 883.69 KB | 1.00 |
| Current | Petstore,ParseAndGenerate | 474.75 μs | 26.998 μs | 79.604 μs | 454.69 μs | 1.23 | 0.21 | 16.6016 | 5.8594 | - | 849.99 KB | 0.96 |
| | | | | | | | | | | | | |
| Current | StripeApi,GenerateOnly | 18,655.74 μs | 923.037 μs | 2,721.595 μs | 18,123.93 μs | ? | ? | 625.0000 | 250.0000 | 31.2500 | 34779.54 KB | ? |
| | | | | | | | | | | | | |
| Released | StripeApi,ParseAndGenerate | 419,754.60 μs | 15,256.846 μs | 44,745.693 μs | 417,412.80 μs | 1.01 | 0.15 | 5000.0000 | 4000.0000 | - | 275602.76 KB | 1.00 |
| Current | StripeApi,ParseAndGenerate | 393,432.72 μs | 16,806.124 μs | 49,553.240 μs | 377,968.55 μs | 0.95 | 0.15 | 4000.0000 | 3000.0000 | - | 255438.53 KB | 0.93 |
13 changes: 13 additions & 0 deletions ...odeGenerator.Benchmarks/BenchmarkDotNet.Artifacts/results/ComparisonBenchmarks-report.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Method;Job;AnalyzeLaunchVariance;EvaluateOverhead;MaxAbsoluteError;MaxRelativeError;MinInvokeCount;MinIterationTime;OutlierMode;Affinity;EnvironmentVariables;Jit;LargeAddressAware;Platform;PowerPlanMode;Runtime;AllowVeryLargeObjects;Concurrent;CpuGroups;Force;HeapAffinitizeMask;HeapCount;NoAffinitize;RetainVm;Server;Arguments;BuildConfiguration;Clock;EngineFactory;NuGetReferences;Toolchain;IsMutator;InvocationCount;IterationCount;IterationTime;LaunchCount;MaxIterationCount;MaxWarmupIterationCount;MemoryRandomization;MinIterationCount;MinWarmupIterationCount;RunStrategy;UnrollFactor;WarmupCount;Categories;Mean;Error;StdDev;Median;Ratio;RatioSD;Gen0;Gen1;Gen2;Allocated;Alloc Ratio
Current;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"GitHubApiNext,GenerateOnly";"19,590.73 μs";913.760 μs;"2,694.242 μs";"18,504.19 μs";?;?;687.5000;343.7500;15.6250;37497.21 KB;?
Released;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"GitHubApiNext,ParseAndGenerate";"4,275,059.67 μs";"82,573.361 μs";"113,027.358 μs";"4,264,655.55 μs";1.00;0.04;204000.0000;9000.0000;0.0000;10055299.83 KB;1.00
Current;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"GitHubApiNext,ParseAndGenerate";"758,891.62 μs";"14,788.118 μs";"23,023.320 μs";"754,015.15 μs";0.18;0.01;13000.0000;8000.0000;0.0000;678324.84 KB;0.07
Current;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"GitHubApi,GenerateOnly";"15,883.52 μs";308.995 μs;317.315 μs;"15,751.93 μs";?;?;718.7500;343.7500;0.0000;39430.52 KB;?
Released;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"GitHubApi,ParseAndGenerate";"610,237.06 μs";"14,185.529 μs";"40,928.499 μs";"601,908.25 μs";1.00;0.09;8000.0000;7000.0000;0.0000;435401.99 KB;1.00
Current;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"GitHubApi,ParseAndGenerate";"548,678.89 μs";"10,787.996 μs";"19,175.653 μs";"548,303.45 μs";0.90;0.07;8000.0000;7000.0000;0.0000;412075.61 KB;0.95
Current;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"Petstore,GenerateOnly";18.11 μs;0.732 μs;2.053 μs;17.33 μs;?;?;1.1292;0.0305;0.0000;56.34 KB;?
Released;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"Petstore,ParseAndGenerate";384.84 μs;7.691 μs;9.155 μs;382.91 μs;1.00;0.03;17.5781;7.8125;0.0000;883.69 KB;1.00
Current;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"Petstore,ParseAndGenerate";474.75 μs;26.998 μs;79.604 μs;454.69 μs;1.23;0.21;16.6016;5.8594;0.0000;849.99 KB;0.96
Current;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"StripeApi,GenerateOnly";"18,655.74 μs";923.037 μs;"2,721.595 μs";"18,123.93 μs";?;?;625.0000;250.0000;31.2500;34779.54 KB;?
Released;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"StripeApi,ParseAndGenerate";"419,754.60 μs";"15,256.846 μs";"44,745.693 μs";"417,412.80 μs";1.01;0.15;5000.0000;4000.0000;0.0000;275602.76 KB;1.00
Current;DefaultJob;False;Default;Default;Default;Default;Default;Default;1111111111111111;Empty;RyuJit;Default;X64;8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c;.NET 10.0;False;True;False;True;Default;Default;False;False;False;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;Default;16;Default;"StripeApi,ParseAndGenerate";"393,432.72 μs";"16,806.124 μs";"49,553.240 μs";"377,968.55 μs";0.95;0.15;4000.0000;3000.0000;0.0000;255438.53 KB;0.93
Comment on lines +1 to +13
41 changes: 41 additions & 0 deletions ...deGenerator.Benchmarks/BenchmarkDotNet.Artifacts/results/ComparisonBenchmarks-report.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
<!DOCTYPE html>
<html lang='en'>
<head>
<meta charset='utf-8' />
<title>ComparisonBenchmarks-20260314-212521</title>
Comment on lines +1 to +5

<style type="text/css">
table { border-collapse: collapse; display: block; width: 100%; overflow: auto; }
td, th { padding: 6px 13px; border: 1px solid #ddd; text-align: right; }
tr { background-color: #fff; border-top: 1px solid #ccc; }
tr:nth-child(even) { background: #f8f8f8; }
</style>
</head>
<body>
<pre><code>
BenchmarkDotNet v0.15.8, Windows 11 (10.0.26200.7840/25H2/2025Update/HudsonValley2)
AMD Ryzen 7 7800X3D 4.20GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK 10.0.200-preview.0.26103.119
[Host] : .NET 10.0.3 (10.0.3, 10.0.326.7603), X64 RyuJIT x86-64-v4
DefaultJob : .NET 10.0.3 (10.0.3, 10.0.326.7603), X64 RyuJIT x86-64-v4
</code></pre>
<pre><code></code></pre>

<table>
<thead><tr><th>Method</th><th>Categories </th><th>Mean </th><th>Error </th><th>StdDev </th><th>Median </th><th>Ratio</th><th>RatioSD</th><th>Gen0 </th><th>Gen1</th><th>Gen2</th><th>Allocated</th><th>Alloc Ratio</th>
</tr>
</thead><tbody><tr><td>Current</td><td>GitHubApiNext,GenerateOnly</td><td>19,590.73 &mu;s</td><td>913.760 &mu;s</td><td>2,694.242 &mu;s</td><td>18,504.19 &mu;s</td><td>?</td><td>?</td><td>687.5000</td><td>343.7500</td><td>15.6250</td><td>37497.21 KB</td><td>?</td>
</tr><tr><td>Released</td><td>GitHubApiNext,ParseAndGenerate</td><td>4,275,059.67 &mu;s</td><td>82,573.361 &mu;s</td><td>113,027.358 &mu;s</td><td>4,264,655.55 &mu;s</td><td>1.00</td><td>0.04</td><td>204000.0000</td><td>9000.0000</td><td>-</td><td>10055299.83 KB</td><td>1.00</td>
</tr><tr><td>Current</td><td>GitHubApiNext,ParseAndGenerate</td><td>758,891.62 &mu;s</td><td>14,788.118 &mu;s</td><td>23,023.320 &mu;s</td><td>754,015.15 &mu;s</td><td>0.18</td><td>0.01</td><td>13000.0000</td><td>8000.0000</td><td>-</td><td>678324.84 KB</td><td>0.07</td>
</tr><tr><td>Current</td><td>GitHubApi,GenerateOnly</td><td>15,883.52 &mu;s</td><td>308.995 &mu;s</td><td>317.315 &mu;s</td><td>15,751.93 &mu;s</td><td>?</td><td>?</td><td>718.7500</td><td>343.7500</td><td>-</td><td>39430.52 KB</td><td>?</td>
</tr><tr><td>Released</td><td>GitHubApi,ParseAndGenerate</td><td>610,237.06 &mu;s</td><td>14,185.529 &mu;s</td><td>40,928.499 &mu;s</td><td>601,908.25 &mu;s</td><td>1.00</td><td>0.09</td><td>8000.0000</td><td>7000.0000</td><td>-</td><td>435401.99 KB</td><td>1.00</td>
</tr><tr><td>Current</td><td>GitHubApi,ParseAndGenerate</td><td>548,678.89 &mu;s</td><td>10,787.996 &mu;s</td><td>19,175.653 &mu;s</td><td>548,303.45 &mu;s</td><td>0.90</td><td>0.07</td><td>8000.0000</td><td>7000.0000</td><td>-</td><td>412075.61 KB</td><td>0.95</td>
</tr><tr><td>Current</td><td>Petstore,GenerateOnly</td><td>18.11 &mu;s</td><td>0.732 &mu;s</td><td>2.053 &mu;s</td><td>17.33 &mu;s</td><td>?</td><td>?</td><td>1.1292</td><td>0.0305</td><td>-</td><td>56.34 KB</td><td>?</td>
</tr><tr><td>Released</td><td>Petstore,ParseAndGenerate</td><td>384.84 &mu;s</td><td>7.691 &mu;s</td><td>9.155 &mu;s</td><td>382.91 &mu;s</td><td>1.00</td><td>0.03</td><td>17.5781</td><td>7.8125</td><td>-</td><td>883.69 KB</td><td>1.00</td>
</tr><tr><td>Current</td><td>Petstore,ParseAndGenerate</td><td>474.75 &mu;s</td><td>26.998 &mu;s</td><td>79.604 &mu;s</td><td>454.69 &mu;s</td><td>1.23</td><td>0.21</td><td>16.6016</td><td>5.8594</td><td>-</td><td>849.99 KB</td><td>0.96</td>
</tr><tr><td>Current</td><td>StripeApi,GenerateOnly</td><td>18,655.74 &mu;s</td><td>923.037 &mu;s</td><td>2,721.595 &mu;s</td><td>18,123.93 &mu;s</td><td>?</td><td>?</td><td>625.0000</td><td>250.0000</td><td>31.2500</td><td>34779.54 KB</td><td>?</td>
</tr><tr><td>Released</td><td>StripeApi,ParseAndGenerate</td><td>419,754.60 &mu;s</td><td>15,256.846 &mu;s</td><td>44,745.693 &mu;s</td><td>417,412.80 &mu;s</td><td>1.01</td><td>0.15</td><td>5000.0000</td><td>4000.0000</td><td>-</td><td>275602.76 KB</td><td>1.00</td>
</tr><tr><td>Current</td><td>StripeApi,ParseAndGenerate</td><td>393,432.72 &mu;s</td><td>16,806.124 &mu;s</td><td>49,553.240 &mu;s</td><td>377,968.55 &mu;s</td><td>0.95</td><td>0.15</td><td>4000.0000</td><td>3000.0000</td><td>-</td><td>255438.53 KB</td><td>0.93</td>
</tr></tbody></table>
</body>
</html>
Loading
Loading