Skip to content

Commit 287a4aa

Browse files
saucecontrolsaucecontrol
committed
add allocation-free hashing
1 parent 7e876fb commit 287a4aa

11 files changed

Lines changed: 345 additions & 84 deletions

File tree

‎readme.md‎

Lines changed: 32 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
Blake2Fast
22
==========
33

4-
These [RFC 7693](https://tools.ietf.org/html/rfc7693)-compliant Blake2 implementations have been tuned for high speed and low memory usage. The .NET Core 2.1 build supports the new X86 SIMD Intrinsics for even greater speed and `Span<T>` for even lower memory usage.
4+
These [RFC 7693](https://tools.ietf.org/html/rfc7693)-compliant BLAKE2 implementations have been tuned for high speed and low memory usage. The .NET Core 2.1 build supports the new X86 SIMD Intrinsics for even greater speed and `Span<T>` for even lower memory usage.
55

66
Sample benchmark results comparing with built-in .NET algorithms, 10MiB input, .NET Core 2.1 x64 and x86 runtimes:
77

@@ -19,7 +19,7 @@ MD5 | X86 | 20.06 ms | 0.0996 ms | 0.0931 ms | 0 B |
1919
SHA256 | X86 | 52.47 ms | 0.3252 ms | 0.3042 ms | 0 B |
2020
SHA512 | X86 | 44.07 ms | 0.1643 ms | 0.1372 ms | 0 B |
2121

22-
You can find more detailed comparison between Blake2Fast and other .NET Blake2 implementations starting [here](https://photosauce.net/blog/post/fast-hashing-with-blake2-part-1-nuget-is-a-minefield)
22+
You can find more detailed comparison between Blake2Fast and other .NET BLAKE2 implementations starting [here](https://photosauce.net/blog/post/fast-hashing-with-blake2-part-1-nuget-is-a-minefield)
2323

2424
Installation
2525
------------
@@ -36,26 +36,29 @@ Usage
3636
### All-at-Once Hashing
3737

3838
The simplest and lightest-weight way to calculate a hash is the all-at-once `ComputeHash` method.
39+
3940
```C#
40-
Blake2b.ComputeHash(data);
41+
var hash = Blake2b.ComputeHash(data);
4142
```
4243

43-
Blake2 supports variable digest lengths from 1 to 32 bytes for `Blake2s` or 1 to 64 bytes for `Blake2b`.
44+
BLAKE2 supports variable digest lengths from 1 to 32 bytes for BLAKE2s or 1 to 64 bytes for BLAKE2b.
45+
4446
```C#
45-
Blake2b.ComputeHash(48, data);
47+
var hash = Blake2b.ComputeHash(42, data);
4648
```
4749

48-
Blake2 also natively supports keyed hashing.
50+
BLAKE2 also natively supports keyed hashing.
51+
4952
```C#
50-
Blake2b.ComputeHash(key, data);
53+
var hash = Blake2b.ComputeHash(key, data);
5154
```
5255

5356
### Incremental Hashing
5457

55-
Blake2 hashes can be incrementally updated if you do not have the data available all at once.
58+
BLAKE2 hashes can be incrementally updated if you do not have the data available all at once.
5659

5760
```C#
58-
async Task<byte[]> CalculateHashAsync(Stream data)
61+
async Task<byte[]> ComputeHashAsync(Stream data)
5962
{
6063
var incHash = Blake2b.CreateIncrementalHasher();
6164
var buffer = new byte[4096];
@@ -71,11 +74,30 @@ async Task<byte[]> CalculateHashAsync(Stream data)
7174
}
7275
```
7376

77+
### Allocation-Free Hashing
78+
79+
The output hash digest can be written to an existing buffer to avoid allocating a new array each time. This is especially useful when performing an iterative hash, as might be used in a [key derivation function](https://en.wikipedia.org/wiki/Key_derivation_function).
80+
81+
```C#
82+
byte[] DeriveBytes(string password, byte[] salt)
83+
{
84+
// Create key from password, then hash the salt using the key
85+
var pwkey = Blake2b.ComputeHash(System.Text.Encoding.UTF8.GetBytes(password));
86+
var hbuff = Blake2b.ComputeHash(pwkey, salt);
87+
88+
// Hash the hash lots of times, re-using the same buffer
89+
for (int i = 0; i < 1_000_000; i++)
90+
Blake2b.ComputeAndWriteHash(pwkey, hbuff, hbuff);
91+
92+
return hbuff;
93+
}
94+
```
95+
7496
### System.Security.Cryptography Interop
7597

7698
For interoperating with code that uses `System.Security.Cryptography` primitives, Blake2Fast can create a `HashAlgorithm` wrapper. The wrapper inherits from `HMAC` in case keyed hashing is required.
7799

78-
`HashAlgorithm` is less efficient than the above methods, so use it only when necessary.
100+
`HashAlgorithm` is less efficient than the above methods, so use it only when necessary for compatibility.
79101

80102
```C#
81103
byte[] WriteDataAndCalculateHash(byte[] data)

‎src/Blake2Fast/Blake2Fast.csproj‎

Lines changed: 11 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -6,35 +6,32 @@
66
<AssemblyTitle>Blake2Fast</AssemblyTitle>
77
<TargetFrameworks>netstandard1.0;netstandard1.3;netstandard2.0;netcoreapp2.0;netcoreapp2.1;net45</TargetFrameworks>
88
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
9-
<DebugType>pdbonly</DebugType>
10-
<DebugSymbols>true</DebugSymbols>
119
<LangVersion>latest</LangVersion>
12-
<VersionPrefix>0.1.0</VersionPrefix>
10+
<VersionPrefix>0.2.0</VersionPrefix>
1311
<Authors>Clinton Ingram</Authors>
1412
<Product>Blake2Fast</Product>
15-
<Description>Optimized implementations of the Blake2b and Blake2s hashing algorithms. Uses SSE2-SSE4.1 intrinsics support on .NET Core 2.1</Description>
13+
<Description>Optimized implementations of the BLAKE2b and BLAKE2s hashing algorithms. Uses SSE2-SSE4.1 Intrinsics support on .NET Core 2.1</Description>
1614
<Copyright>Copyright © 2018 Clinton Ingram</Copyright>
1715
<RepositoryType>git</RepositoryType>
1816
<RepositoryUrl>https://github.com/saucecontrol/Blake2Fast</RepositoryUrl>
1917
<PackageIconUrl>https://photosauce.net/icon64x64.png</PackageIconUrl>
2018
<PackageProjectUrl>https://github.com/saucecontrol/Blake2Fast</PackageProjectUrl>
21-
<PackageLicenseUrl>https://github.com/saucecontrol/Blake2Fast/license</PackageLicenseUrl>
19+
<PackageLicenseUrl>https://github.com/saucecontrol/Blake2Fast/blob/master/license</PackageLicenseUrl>
2220
<PackageReleaseNotes>See https://github.com/saucecontrol/Blake2Fast/releases</PackageReleaseNotes>
23-
<PackageTags>Blake2 Hash Blake2b Blake2s SSE SIMD HashAlgorithm HMAC</PackageTags>
24-
<GeneratePackageOnBuild>True</GeneratePackageOnBuild>
21+
<PackageTags>BLAKE2 Hash BLAKE2b BLAKE2s SSE SIMD HashAlgorithm HMAC</PackageTags>
2522
<AllowedOutputExtensionsInPackageBuildOutputFolder>$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb</AllowedOutputExtensionsInPackageBuildOutputFolder>
2623
</PropertyGroup>
2724

28-
<PropertyGroup Condition="'$(TargetFramework)'=='netcoreapp2.0' Or '$(TargetFramework)'=='netcoreapp2.1'">
29-
<DefineConstants>$(DefineConstants);IMPLICIT_BYTESPAN</DefineConstants>
30-
</PropertyGroup>
31-
32-
<PropertyGroup Condition="'$(TargetFramework)'=='netcoreapp2.1'">
33-
<DefineConstants>$(DefineConstants);FAST_SPAN;USE_INTRINSICS</DefineConstants>
25+
<PropertyGroup>
26+
<DefineConstants Condition="'$(TargetFramework)'=='netcoreapp2.0' Or '$(TargetFramework)'=='netcoreapp2.1'">$(DefineConstants);IMPLICIT_BYTESPAN</DefineConstants>
27+
<DefineConstants Condition="'$(TargetFramework)'=='netcoreapp2.1'">$(DefineConstants);FAST_SPAN;USE_INTRINSICS</DefineConstants>
3428
</PropertyGroup>
3529

3630
<PropertyGroup Condition="'$(Configuration)'=='Release'">
37-
<DocumentationFile>bin\$(Configuration)\$(TargetFrameWork)\SauceControl.Blake2Fast.xml</DocumentationFile>
31+
<DebugType>pdbonly</DebugType>
32+
<DebugSymbols>true</DebugSymbols>
33+
<GeneratePackageOnBuild>True</GeneratePackageOnBuild>
34+
<DocumentationFile>bin\$(Configuration)\$(TargetFrameWork)\$(AssemblyName).xml</DocumentationFile>
3835
</PropertyGroup>
3936

4037
<ItemGroup Condition="$(DefineConstants.Contains('USE_INTRINSICS'))">

‎src/Blake2Fast/Blake2b.cs‎

Lines changed: 65 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,35 @@
1-
#if !NETSTANDARD1_0
1+
using System;
2+
3+
#if !NETSTANDARD1_0
24
using System.Security.Cryptography;
35
#endif
46

57
#if FAST_SPAN
68
using ByteSpan = System.ReadOnlySpan<byte>;
9+
using WriteableByteSpan = System.Span<byte>;
710
#else
811
using ByteSpan = System.ArraySegment<byte>;
12+
using WriteableByteSpan = System.ArraySegment<byte>;
913
#endif
1014

1115
namespace SauceControl.Blake2Fast
1216
{
13-
/// <summary>Static helper methods for Blake2b hashing.</summary>
17+
/// <summary>Static helper methods for BLAKE2b hashing.</summary>
1418
public static class Blake2b
1519
{
20+
/// <summary>The default hash digest length in bytes. For BLAKE2b, this value is 64.</summary>
21+
public const int DefaultDigestLength = Blake2bContext.HashBytes;
22+
1623
/// <inheritdoc cref="ComputeHash(int, ByteSpan, ByteSpan)"/>
17-
public static byte[] ComputeHash(ByteSpan input) => ComputeHash(Blake2bContext.HashBytes, default, input);
24+
public static byte[] ComputeHash(ByteSpan input) => ComputeHash(DefaultDigestLength, default, input);
1825

1926
/// <inheritdoc cref="ComputeHash(int, ByteSpan, ByteSpan)"/>
2027
public static byte[] ComputeHash(int digestLength, ByteSpan input) => ComputeHash(digestLength, default, input);
2128

2229
/// <inheritdoc cref="ComputeHash(int, ByteSpan, ByteSpan)"/>
23-
public static byte[] ComputeHash(ByteSpan key, ByteSpan input) => ComputeHash(Blake2bContext.HashBytes, key, input);
30+
public static byte[] ComputeHash(ByteSpan key, ByteSpan input) => ComputeHash(DefaultDigestLength, key, input);
2431

25-
/// <summary>Perform an all-at-once Blake2b hash computation.</summary>
32+
/// <summary>Perform an all-at-once BLAKE2b hash computation.</summary>
2633
/// <remarks>If you have all the input available at once, this is the most efficient way to calculate the hash.</remarks>
2734
/// <param name="digestLength">The hash digest length in bytes. Valid values are 1 to 64.</param>
2835
/// <param name="key">0 to 64 bytes of input for initializing a keyed hash.</param>
@@ -36,16 +43,50 @@ public static byte[] ComputeHash(int digestLength, ByteSpan key, ByteSpan input)
3643
return ctx.Finish();
3744
}
3845

46+
/// <inheritdoc cref="ComputeAndWriteHash(ByteSpan, ByteSpan, WriteableByteSpan)" />
47+
public static void ComputeAndWriteHash(ByteSpan input, WriteableByteSpan output) => ComputeAndWriteHash(DefaultDigestLength, default, input, output);
48+
49+
/// <inheritdoc cref="ComputeAndWriteHash(int, ByteSpan, ByteSpan, WriteableByteSpan)" />
50+
public static void ComputeAndWriteHash(int digestLength, ByteSpan input, WriteableByteSpan output) => ComputeAndWriteHash(digestLength, default, input, output);
51+
52+
/// <summary>Perform an all-at-once BLAKE2b hash computation and write the hash digest to <paramref name="output" />.</summary>
53+
/// <remarks>If you have all the input available at once, this is the most efficient way to calculate the hash.</remarks>
54+
/// <param name="key">0 to 64 bytes of input for initializing a keyed hash.</param>
55+
/// <param name="input">The message bytes to hash.</param>
56+
/// <param name="output">Destination buffer into which the hash digest is written. The buffer must have a capacity of at least <see cref="DefaultDigestLength"/>(64) /> bytes.</param>
57+
public static void ComputeAndWriteHash(ByteSpan key, ByteSpan input, WriteableByteSpan output) => ComputeAndWriteHash(DefaultDigestLength, key, input, output);
58+
59+
/// <summary>Perform an all-at-once BLAKE2b hash computation and write the hash digest to <paramref name="output" />.</summary>
60+
/// <remarks>If you have all the input available at once, this is the most efficient way to calculate the hash.</remarks>
61+
/// <param name="digestLength">The hash digest length in bytes. Valid values are 1 to 64.</param>
62+
/// <param name="key">0 to 64 bytes of input for initializing a keyed hash.</param>
63+
/// <param name="input">The message bytes to hash.</param>
64+
/// <param name="output">Destination buffer into which the hash digest is written. The buffer must have a capacity of at least <paramref name="digestLength" /> bytes.</param>
65+
public static void ComputeAndWriteHash(int digestLength, ByteSpan key, ByteSpan input, WriteableByteSpan output)
66+
{
67+
#if FAST_SPAN
68+
if (output.Length < digestLength)
69+
#else
70+
if (output.Count < digestLength)
71+
#endif
72+
throw new ArgumentException($"Output buffer must have a capacity of at least {digestLength} bytes.", nameof(output));
73+
74+
var ctx = default(Blake2bContext);
75+
ctx.Init(digestLength, key);
76+
ctx.Update(input);
77+
ctx.TryFinish(output, out int _);
78+
}
79+
3980
/// <inheritdoc cref="CreateIncrementalHasher(int, ByteSpan)" />
40-
public static IBlake2Incremental CreateIncrementalHasher() => CreateIncrementalHasher(Blake2bContext.HashBytes, default(ByteSpan));
81+
public static IBlake2Incremental CreateIncrementalHasher() => CreateIncrementalHasher(DefaultDigestLength, default(ByteSpan));
4182

4283
/// <inheritdoc cref="CreateIncrementalHasher(int, ByteSpan)" />
4384
public static IBlake2Incremental CreateIncrementalHasher(int digestLength) => CreateIncrementalHasher(digestLength, default(ByteSpan));
4485

4586
/// <inheritdoc cref="CreateIncrementalHasher(int, ByteSpan)" />
46-
public static IBlake2Incremental CreateIncrementalHasher(ByteSpan key) => CreateIncrementalHasher(Blake2bContext.HashBytes, key);
87+
public static IBlake2Incremental CreateIncrementalHasher(ByteSpan key) => CreateIncrementalHasher(DefaultDigestLength, key);
4788

48-
/// <summary>Create and initialize an incremental Blake2b hash computation.</summary>
89+
/// <summary>Create and initialize an incremental BLAKE2b hash computation.</summary>
4990
/// <remarks>If you will recieve the input in segments rather than all at once, this is the most efficient way to calculate the hash.</remarks>
5091
/// <param name="digestLength">The hash digest length in bytes. Valid values are 1 to 64.</param>
5192
/// <param name="key">0 to 64 bytes of input for initializing a keyed hash.</param>
@@ -70,6 +111,18 @@ public static IBlake2Incremental CreateIncrementalHasher(int digestLength, ByteS
70111
/// <inheritdoc cref="ComputeHash(int, ByteSpan, ByteSpan)"/>
71112
public static byte[] ComputeHash(int digestLength, byte[] key, byte[] input) => ComputeHash(digestLength, key.AsByteSpan(), input.AsByteSpan());
72113

114+
/// <inheritdoc cref="ComputeAndWriteHash(ByteSpan, ByteSpan, WriteableByteSpan)" />
115+
public static void ComputeAndWriteHash(byte[] input, byte[] output) => ComputeAndWriteHash(DefaultDigestLength, default, input.AsByteSpan(), output.AsByteSpan());
116+
117+
/// <inheritdoc cref="ComputeAndWriteHash(int, ByteSpan, ByteSpan, WriteableByteSpan)" />
118+
public static void ComputeAndWriteHash(int digestLength, byte[] input, byte[] output) => ComputeAndWriteHash(digestLength, default, input.AsByteSpan(), output.AsByteSpan());
119+
120+
/// <inheritdoc cref="ComputeAndWriteHash(ByteSpan, ByteSpan, WriteableByteSpan)" />
121+
public static void ComputeAndWriteHash(byte[] key, byte[] input, byte[] output) => ComputeAndWriteHash(DefaultDigestLength, key.AsByteSpan(), input.AsByteSpan(), output.AsByteSpan());
122+
123+
/// <inheritdoc cref="ComputeAndWriteHash(int, ByteSpan, ByteSpan, WriteableByteSpan)" />
124+
public static void ComputeAndWriteHash(int digestLength, byte[] key, byte[] input, byte[] output) => ComputeAndWriteHash(digestLength, key.AsByteSpan(), input.AsByteSpan(), output.AsByteSpan());
125+
73126
/// <inheritdoc cref="CreateIncrementalHasher(int, ByteSpan)" />
74127
public static IBlake2Incremental CreateIncrementalHasher(byte[] key) => CreateIncrementalHasher(key.AsByteSpan());
75128

@@ -79,18 +132,18 @@ public static IBlake2Incremental CreateIncrementalHasher(int digestLength, ByteS
79132

80133
#if !NETSTANDARD1_0
81134
/// <inheritdoc cref="CreateHashAlgorithm(int)" />
82-
public static HashAlgorithm CreateHashAlgorithm() => CreateHashAlgorithm(Blake2bContext.HashBytes);
135+
public static HashAlgorithm CreateHashAlgorithm() => CreateHashAlgorithm(DefaultDigestLength);
83136

84-
/// <summary>Creates and initializes a <see cref="HashAlgorithm" /> instance that implements Blake2b hashing.</summary>
137+
/// <summary>Creates and initializes a <see cref="HashAlgorithm" /> instance that implements BLAKE2b hashing.</summary>
85138
/// <remarks>Use this only if you require an implementation of <see cref="HashAlgorithm" />. It is less efficient than the direct methods.</remarks>
86139
/// <param name="digestLength">The hash digest length in bytes. Valid values are 1 to 64.</param>
87140
/// <returns>A <see cref="HashAlgorithm" /> instance.</returns>
88141
public static HashAlgorithm CreateHashAlgorithm(int digestLength) => new Blake2HMAC(Blake2Algorithm.Blake2b, digestLength, default);
89142

90143
/// <inheritdoc cref="CreateHMAC(int, ByteSpan)" />
91-
public static HMAC CreateHMAC(ByteSpan key) => CreateHMAC(Blake2bContext.HashBytes, key);
144+
public static HMAC CreateHMAC(ByteSpan key) => CreateHMAC(DefaultDigestLength, key);
92145

93-
/// <summary>Creates and initializes an <see cref="HMAC" /> instance that implements Blake2b keyed hashing.</summary>
146+
/// <summary>Creates and initializes an <see cref="HMAC" /> instance that implements BLAKE2b keyed hashing. Uses BLAKE2's built-in support for keyed hashing rather than the normal 2-pass approach.</summary>
94147
/// <remarks>Use this only if you require an implementation of <see cref="HMAC" />. It is less efficient than the direct methods.</remarks>
95148
/// <param name="digestLength">The hash digest length in bytes. Valid values are 1 to 64.</param>
96149
/// <param name="key">0 to 64 bytes of input for initializing the keyed hash.</param>

‎src/Blake2Fast/Blake2bContext.cs‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -171,10 +171,6 @@ public void Update(ByteSpan input)
171171
}
172172
}
173173

174-
#if !IMPLICIT_BYTESPAN
175-
public void Update(byte[] input) => Update(input.AsByteSpan());
176-
#endif
177-
178174
private void finish(WriteableByteSpan hash)
179175
{
180176
if (this.f[0] != 0)
@@ -212,10 +208,13 @@ public byte[] Finish()
212208
return hash;
213209
}
214210

215-
#if FAST_SPAN
216211
public bool TryFinish(WriteableByteSpan output, out int bytesWritten)
217212
{
213+
#if FAST_SPAN
218214
if (output.Length < outlen)
215+
#else
216+
if (output.Count < outlen)
217+
#endif
219218
{
220219
bytesWritten = 0;
221220
return false;
@@ -225,6 +224,11 @@ public bool TryFinish(WriteableByteSpan output, out int bytesWritten)
225224
bytesWritten = (int)outlen;
226225
return true;
227226
}
227+
228+
#if !IMPLICIT_BYTESPAN
229+
public void Update(byte[] input) => Update(input.AsByteSpan());
230+
231+
public bool TryFinish(byte[] output, out int bytesWritten) => TryFinish(output.AsByteSpan(), out bytesWritten);
228232
#endif
229233
}
230234
}

0 commit comments

Comments
 (0)