Skip to content

Commit 6dc701e

Browse files
author
Michael Conrad
authored
More cleanup for 2.0 (#384)
1 parent 2b8c1f6 commit 6dc701e

31 files changed

Lines changed: 150 additions & 1047 deletions

‎Benchmarks.md‎

Lines changed: 42 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -10,15 +10,13 @@ All CacheManager instances used in the benchmarks have only one cache handle con
1010
We are using the same configuration for all benchmarks and running two jobs each, one for x86 and one for x64. Regarding the different platforms, the conclusion is obviously that x64 is always faster than the x86 platform, but x64 consumes slightly more memory of course.
1111

1212
```ini
13-
BenchmarkDotNet=v0.9.1.0
14-
OS=Microsoft Windows NT 6.2.9200.0
15-
Processor=Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz, ProcessorCount=8
16-
Frequency=3328117 ticks, Resolution=300.4702 ns
17-
HostCLR=MS.NET 4.0.30319.42000, Arch=32-bit RELEASE
18-
19-
Type=PutWithRegionSingleBenchmark Mode=Throughput Platform=X64
20-
Jit=RyuJit LaunchCount=1 WarmupCount=2
21-
TargetCount=100
13+
BenchmarkDotNet v0.14.0, Windows 11 (10.0.26100.3194)
14+
12th Gen Intel Core i7-12700KF, 1 CPU, 20 logical and 12 physical cores
15+
.NET SDK 9.0.200
16+
[Host] : .NET 8.0.13 (8.0.1325.6609), X64 RyuJIT AVX2
17+
Job-BIBDFC : .NET 8.0.13 (8.0.1325.6609), X64 RyuJIT AVX2
18+
19+
IterationCount=10 LaunchCount=1 WarmupCount=2
2220
```
2321

2422
### Add
@@ -27,73 +25,47 @@ TargetCount=100
2725
Redis will be a lot slower in this scenario because CacheManager waits for the response to be able to return the `bool` value if the key has been added or not.
2826
In general, it is good to see how fast the Dictionary handle is compared to the `System.Runtime` one. One thing you cannot see here is that also the memory footprint of the Dictionary handle is much lower.
2927

30-
Method | Platform | Median | StdDev | Scaled |
31-
-----------: |:-----------: |-----------: |----------: |-------: |
32-
Dictionary | X64 | 1.7254 us | 0.0511 us | 1.00 |
33-
Dictionary | X86 | 1.9563 us | 0.0399 us | 1.00 |
34-
Runtime | X64 | 4.9839 us | 0.0778 us | 2.89 |
35-
Runtime | X86 | 7.0324 us | 0.2012 us | 3.59 |
36-
Redis | X64 | 56.6671 us | 1.7202 us | 32.84 |
37-
Redis | X86 | 58.0775 us | 0.9517 us | 29.69 |
28+
29+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
30+
|----------- |-------------:|------------:|------------:|-------:|--------:|-------:|-------:|----------:|------------:|
31+
| Dictionary | 121.0 ns | 2.00 ns | 1.04 ns | 1.00 | 0.01 | 0.0153 | - | 200 B | 1.00 |
32+
| Runtime | 637.2 ns | 28.46 ns | 16.94 ns | 5.27 | 0.14 | 0.2384 | 0.0010 | 3120 B | 15.60 |
33+
| MsMemory | 198.8 ns | 4.59 ns | 3.04 ns | 1.64 | 0.03 | 0.0260 | - | 340 B | 1.70 |
34+
| Redis | 105,395.6 ns | 3,758.09 ns | 2,485.74 ns | 871.20 | 20.86 | - | - | 1256 B | 6.28 |
3835

3936
*Adding one item per run with using region*
4037

41-
Method | Platform | Median | StdDev | Scaled |
42-
-----------: |:-----------: |-----------: |----------: |-------: |
43-
Dictionary | X64 | 1.8094 us | 0.0386 us | 1.00 |
44-
Dictionary | X86 | 2.5029 us | 0.1179 us | 1.00 |
45-
Runtime | X64 | 6.6934 us | 0.1275 us | 3.70 |
46-
Runtime | X86 | 9.2334 us | 0.1637 us | 3.69 |
47-
Redis | X64 | 58.5355 us | 1.7054 us | 32.35 |
48-
Redis | X86 | 61.0272 us | 1.4178 us | 24.38 |
38+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
39+
|----------- |-------------:|------------:|------------:|-------:|--------:|-------:|----------:|------------:|
40+
| Dictionary | 144.2 ns | 4.65 ns | 3.07 ns | 1.00 | 0.03 | 0.0231 | 304 B | 1.00 |
41+
| Runtime | 342.6 ns | 18.01 ns | 11.91 ns | 2.38 | 0.09 | 0.0610 | 800 B | 2.63 |
42+
| MsMemory | 164.1 ns | 5.74 ns | 3.80 ns | 1.14 | 0.03 | 0.0231 | 304 B | 1.00 |
43+
| Redis | 139,200.0 ns | 4,863.92 ns | 2,894.44 ns | 966.01 | 27.25 | - | 1528 B | 5.03 |
44+
4945

5046
### Put
5147
*Put 1 item per run*
5248
Redis is as fast as the other handles in this scenario because CacheManager uses fire and forget for those operations. For Put it doesn't matter to know if the item has been added or updated...
5349

54-
Method | Platform | Median | StdDev | Scaled |
55-
-----------: |:-----------: |-----------: |----------: |-------: |
56-
Dictionary | X64 | 1.6802 us | 0.0235 us | 1.00 |
57-
Dictionary | X86 | 1.9445 us | 0.0341 us | 1.00 |
58-
Runtime | X64 | 4.4431 us | 0.0651 us | 2.64 |
59-
Runtime | X86 | 6.5231 us | 0.1063 us | 3.35 |
60-
Redis | X64 | 2.6869 us | 0.0934 us | 1.60 |
61-
Redis | X86 | 3.5490 us | 0.0848 us | 1.83 |
62-
63-
*Put 1 item per run with region*
64-
65-
Method | Platform | Median | StdDev | Scaled |
66-
-----------: |:-----------: |-----------: |----------: |-------: |
67-
Dictionary | X64 | 1.7401 us | 0.0365 us | 1.00 |
68-
Dictionary | X86 | 2.4589 us | 0.1022 us | 1.00 |
69-
Runtime | X64 | 6.1772 us | 0.3683 us | 3.55 |
70-
Runtime | X86 | 9.9298 us | 0.5574 us | 4.04 |
71-
Redis | X64 | 3.0807 us | 0.0906 us | 1.77 |
72-
Redis | X86 | 3.6385 us | 0.2123 us | 1.48 |
50+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
51+
|----------- |------------:|-----------:|-----------:|------:|--------:|-------:|-------:|----------:|------------:|
52+
| Dictionary | 95.01 ns | 1.888 ns | 1.249 ns | 1.00 | 0.02 | 0.0122 | - | 160 B | 1.00 |
53+
| Runtime | 887.35 ns | 19.929 ns | 13.181 ns | 9.34 | 0.18 | 0.4263 | 0.0095 | 5576 B | 34.85 |
54+
| MsMemory | 172.40 ns | 5.030 ns | 3.327 ns | 1.81 | 0.04 | 0.0336 | - | 440 B | 2.75 |
55+
| Redis | 4,136.37 ns | 301.420 ns | 199.371 ns | 43.54 | 2.07 | 0.0839 | 0.0610 | 1095 B | 6.84 |
7356

57+
7458
### Get
7559
*Get 1 item per run*
7660
With `Get` operations we can clearly see how much faster an in-memory cache is, compared to the distributed variant. That's why it makes so much sense to use CacheManager with a first and secondary cache layer.
7761

78-
Method | Platform | Median | StdDev | Scaled |
79-
-----------: |:-----------: |-----------: |----------: |-------: |
80-
Dictionary | X64 | 169.8708 ns | 4.6186 ns | 1.00 |
81-
Dictionary | X86 | 540.0879 ns | 8.9363 ns | 1.00 |
82-
Runtime | X64 | 310.4025 ns | 8.2928 ns | 1.83 |
83-
Runtime | X86 | 1,030.5911 ns | 8.0532 ns | 1.91 |
84-
Redis | X64 | 56,917.3537 ns | 2,035.4765 ns | 335.06 |
85-
Redis | X86 | 59,519.5459 ns | 1,527.9613 ns | 110.20 |
86-
87-
*Get 1 item per run with region*
8862

89-
Method | Platform | Median | StdDev | Scaled |
90-
-----------: |:-----------: |-----------: |----------: |-------: |
91-
Dictionary | X64 | 234.9676 ns | 7.9172 ns | 1.00 |
92-
Dictionary | X86 | 600.0429 ns | 13.7090 ns | 1.00 |
93-
Runtime | X64 | 497.4116 ns | 15.6998 ns | 2.12 |
94-
Runtime | X86 | 1,191.6452 ns | 19.8484 ns | 1.99 |
95-
Redis | X64 | 57,257.9868 ns | 1,705.0148 ns | 243.68 |
96-
Redis | X86 | 61,789.3944 ns | 1,775.6064 ns | 102.97 |
63+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
64+
|----------- |-------------:|-------------:|-------------:|---------:|--------:|-------:|----------:|------------:|
65+
| Dictionary | 34.97 ns | 0.532 ns | 0.317 ns | 1.00 | 0.01 | - | - | NA |
66+
| Runtime | 179.45 ns | 2.104 ns | 1.392 ns | 5.13 | 0.06 | 0.0153 | 200 B | NA |
67+
| MsMemory | 75.12 ns | 0.338 ns | 0.201 ns | 2.15 | 0.02 | - | - | NA |
68+
| Redis | 75,534.71 ns | 5,303.351 ns | 3,507.838 ns | 2,160.12 | 97.47 | 0.1221 | 2008 B | NA |
9769

9870

9971
### Serializer comparison
@@ -149,12 +121,14 @@ Pretty simple but large enough to analyze the performance.
149121
**Results:**
150122

151123

152-
Method | Platform | Median | StdDev | Scaled | Scaled-SD |
153-
-----: |:-------: |-----------: |----------: |-------: |-------: |
154-
BinarySerializer | X64 | 62.4158 ms | 1.8668 ms | 2.20 | 0.07 |
155-
JsonSerializer | X64 | 28.5521 ms | 0.4633 ms | 1.00 | 0.00 |
156-
JsonGzSerializer | X64 | 102.5552 ms | 4.8584 ms | 3.60 | 0.18 |
157-
ProtoBufSerializer | X64 | 11.1276 ms | 0.1252 ms | 0.39 | 0.01 |
124+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
125+
|------------------------- |----------:|---------:|---------:|------:|--------:|--------:|-------:|----------:|------------:|
126+
| JsonSerializer | 83.21 us | 1.163 us | 0.769 us | 1.00 | 0.01 | 14.8926 | 2.4414 | 191.16 KB | 1.00 |
127+
| JsonGzSerializer | 346.37 us | 4.785 us | 3.165 us | 4.16 | 0.05 | 21.9727 | 2.9297 | 280.75 KB | 1.47 |
128+
| ProtoBufSerializer | 39.18 us | 0.701 us | 0.463 us | 0.47 | 0.01 | 8.6060 | 1.0986 | 110.7 KB | 0.58 |
129+
| BondBinarySerializer | 19.03 us | 0.398 us | 0.263 us | 0.23 | 0.00 | 4.7302 | 0.6714 | 60.53 KB | 0.32 |
130+
| BondFastBinarySerializer | 19.42 us | 0.431 us | 0.285 us | 0.23 | 0.00 | 4.7607 | 0.7324 | 60.84 KB | 0.32 |
131+
| BondSimpleJsonSerializer | 63.56 us | 1.132 us | 0.748 us | 0.76 | 0.01 | 11.9629 | 1.9531 | 153.35 KB | 0.80 |
158132

159133

160134
As expected the protobuf serialization outperforms everything else by a huge margin!

‎benchmarks/CacheManager.Benchmarks/CacheManager.Benchmarks.csproj‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
</ItemGroup>
2020
<ItemGroup>
2121
<PackageReference Include="BenchmarkDotNet" Version="0.14.0" />
22+
<PackageReference Include="Microsoft.Garnet" Version="1.0.57" />
2223
</ItemGroup>
2324
<PropertyGroup Condition=" '$(Configuration)' == 'Debug' ">
2425
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>

‎benchmarks/CacheManager.Benchmarks/Program.cs‎

Lines changed: 41 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,33 +1,53 @@
11
using System;
22
using System.Diagnostics.CodeAnalysis;
3+
using System.Net;
34
using System.Reflection;
45
using BenchmarkDotNet.Configs;
56
using BenchmarkDotNet.Jobs;
67
using BenchmarkDotNet.Running;
8+
using Garnet;
9+
using Garnet.server;
10+
using Microsoft.Extensions.Logging;
711

8-
namespace CacheManager.Benchmarks
12+
namespace CacheManager.Benchmarks;
13+
14+
[ExcludeFromCodeCoverage]
15+
public class Program
916
{
10-
[ExcludeFromCodeCoverage]
11-
public class Program
17+
public static GarnetServer StartServer(ILoggerFactory loggerFactory = null)
18+
{
19+
var server = new GarnetServer(new GarnetServerOptions()
20+
{
21+
EnableLua = true,
22+
LuaOptions = new LuaOptions(LuaMemoryManagementMode.Native, string.Empty, TimeSpan.FromSeconds(5)),
23+
EndPoint = new IPEndPoint(IPAddress.Loopback, 6379)
24+
},
25+
loggerFactory: loggerFactory);
26+
27+
server.Start();
28+
29+
return server;
30+
}
31+
32+
public static void Main(string[] args)
1233
{
13-
public static void Main(string[] args)
34+
StartServer();
35+
36+
do
1437
{
15-
do
16-
{
17-
var config = ManualConfig.CreateMinimumViable()
18-
.AddJob(Job.Default
19-
.WithIterationCount(10)
20-
.WithWarmupCount(2)
21-
.WithLaunchCount(1))
22-
.AddDiagnoser(BenchmarkDotNet.Diagnosers.MemoryDiagnoser.Default);
23-
24-
BenchmarkSwitcher
25-
.FromAssembly(typeof(Program).GetTypeInfo().Assembly)
26-
.Run(args, config);
27-
28-
Console.WriteLine("done!");
29-
Console.WriteLine("Press escape to exit or any key to continue...");
30-
} while (Console.ReadKey().Key != ConsoleKey.Escape);
31-
}
38+
var config = ManualConfig.CreateMinimumViable()
39+
.AddJob(Job.Default
40+
.WithIterationCount(10)
41+
.WithWarmupCount(2)
42+
.WithLaunchCount(1))
43+
.AddDiagnoser(BenchmarkDotNet.Diagnosers.MemoryDiagnoser.Default);
44+
45+
BenchmarkSwitcher
46+
.FromAssembly(typeof(Program).GetTypeInfo().Assembly)
47+
.Run(args, config);
48+
49+
Console.WriteLine("done!");
50+
Console.WriteLine("Press escape to exit or any key to continue...");
51+
} while (Console.ReadKey().Key != ConsoleKey.Escape);
3252
}
3353
}

‎samples/OutputCacheExample/App_Start/BundleConfig.cs‎

Lines changed: 0 additions & 14 deletions
This file was deleted.

‎samples/OutputCacheExample/App_Start/FilterConfig.cs‎

Lines changed: 0 additions & 13 deletions
This file was deleted.

‎samples/OutputCacheExample/App_Start/RouteConfig.cs‎

Lines changed: 0 additions & 23 deletions
This file was deleted.

‎samples/OutputCacheExample/Content/Site.css‎

Lines changed: 0 additions & 24 deletions
This file was deleted.

‎samples/OutputCacheExample/Controllers/HomeController.cs‎

Lines changed: 0 additions & 70 deletions
This file was deleted.

‎samples/OutputCacheExample/Global.asax‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

0 commit comments

Comments
 (0)