-
Notifications
You must be signed in to change notification settings - Fork 66
perf/resilience: reduce OTP polling latency and add runtime resilience gates #493
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
DennisDyallo
merged 28 commits into
yubikit-composite-device-new
from
yubikit-performance
Jul 1, 2026
Merged
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
c197f25
test(perf): add YubiKit performance benchmarks
DennisDyallo 4b0e958
perf(otp): reduce HID write polling latency
DennisDyallo 0a741bd
fix(core): harden SmartCard listener recovery
DennisDyallo 10c2c04
test(core): add OTP polling resilience guard
DennisDyallo eb2ac82
test(core): add runtime resilience static scanner
DennisDyallo 21b8501
docs: document runtime resilience workflow
DennisDyallo dad4326
fix(core): dispose SmartCard listener contexts on stop
DennisDyallo 52d4d11
test(core): add runtime resilience fast runner
DennisDyallo b24077c
docs: defer runtime diagnostics project
DennisDyallo 0363a3d
docs: defer runtime audit skill
DennisDyallo 58ad07c
ci: add runtime resilience check
DennisDyallo 58c4ee9
docs: fold runtime resilience learnings forward
DennisDyallo 2a4c67b
docs(core): require resilience runner for loop changes
DennisDyallo b933d2a
feat(core): add Windows HID transport
DennisDyallo db41b6a
fix(core): document Windows HID access failures
DennisDyallo 32158c9
fix(fido2): align CTAP status values with CTAP 2.2
DennisDyallo 85d6f0b
fix(core): use full PIV application identifier
DennisDyallo 23bdbfb
docs(fido2): clarify CTAP status coverage
DennisDyallo d265372
chore: code style
DennisDyallo de5f0ed
fix(core): audit and fix Windows HID tests
DennisDyallo 51ac5b2
refactor: code style and warnings
DennisDyallo 630c697
Delete src/Core/tests/Yubico.YubiKit.Core.UnitTests/Sessions/Applicat…
DennisDyallo e8141ce
Merge pull request #501 from Yubico/yubikit-piv-9-byte-aid
DennisDyallo a0b78a6
refactor(fido2): remove PuvathRequired backwards compat alias
github-actions[bot] f9f649e
Merge pull request #505 from Yubico/yubikit-ctap-status-22-alignment
DennisDyallo fd7d983
fix(core): use ArgumentException for HID report buffer length validation
github-actions[bot] c2cae84
Merge pull request #496 from Yubico/yubikit-hid-windows
DennisDyallo 82b32a4
fix(core): preserve SmartCard context on stuck listener
DennisDyallo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
241 changes: 241 additions & 0 deletions
241
benchmarks/Yubico.YubiKit.PerformanceBenchmarks/Program.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,241 @@ | ||
| using BenchmarkDotNet.Attributes; | ||
| using BenchmarkDotNet.Columns; | ||
| using BenchmarkDotNet.Configs; | ||
| using BenchmarkDotNet.Diagnosers; | ||
| using BenchmarkDotNet.Environments; | ||
| using BenchmarkDotNet.Exporters; | ||
| using BenchmarkDotNet.Exporters.Csv; | ||
| using BenchmarkDotNet.Exporters.Json; | ||
| using BenchmarkDotNet.Jobs; | ||
| using BenchmarkDotNet.Reports; | ||
| using BenchmarkDotNet.Running; | ||
| using BenchmarkDotNet.Validators; | ||
| using Yubico.YubiKit.Core.Abstractions; | ||
| using Yubico.YubiKit.Core.Devices; | ||
| using Yubico.YubiKit.Core.Protocols.Fido.Hid; | ||
| using Yubico.YubiKit.Core.Protocols.SmartCard.Apdu; | ||
| using Yubico.YubiKit.Core.Sessions; | ||
| using Yubico.YubiKit.Core.Transports.Hid; | ||
| using Yubico.YubiKit.Core.Transports.SmartCard; | ||
| using Yubico.YubiKit.Management; | ||
|
|
||
| var config = YubiKitBenchmarkConfig.Create(); | ||
| BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args, config); | ||
|
|
||
| internal sealed class YubiKitBenchmarkConfig : ManualConfig | ||
| { | ||
| private YubiKitBenchmarkConfig() | ||
| { | ||
| var runId = DateTimeOffset.UtcNow.ToString("yyyyMMdd-HHmmss"); | ||
| ArtifactsPath = Path.Combine(FindRepoRoot(), "artifacts", "benchmarks", runId); | ||
|
|
||
| AddJob(Job.ShortRun | ||
| .WithRuntime(CoreRuntime.Core10_0) | ||
| .WithId("net10-short")); | ||
|
|
||
| AddColumnProvider(DefaultColumnProviders.Instance); | ||
| AddDiagnoser(MemoryDiagnoser.Default); | ||
| AddDiagnoser(ThreadingDiagnoser.Default); | ||
| AddDiagnoser(ExceptionDiagnoser.Default); | ||
| AddDiagnoser(new EventPipeProfiler(EventPipeProfile.CpuSampling)); | ||
| AddExporter(MarkdownExporter.GitHub); | ||
| AddExporter(HtmlExporter.Default); | ||
| AddExporter(CsvExporter.Default); | ||
| AddExporter(CsvMeasurementsExporter.Default); | ||
| AddExporter(JsonExporter.Full); | ||
| AddExporter(RPlotExporter.Default); | ||
| AddValidator(JitOptimizationsValidator.FailOnError); | ||
| AddLogger(DefaultConfig.Instance.GetLoggers().ToArray()); | ||
| } | ||
|
|
||
| public static IConfig Create() => new YubiKitBenchmarkConfig(); | ||
|
|
||
| private static string FindRepoRoot() | ||
| { | ||
| var current = AppContext.BaseDirectory; | ||
| while (!string.IsNullOrEmpty(current)) | ||
| { | ||
| if (Directory.Exists(Path.Combine(current, ".git"))) | ||
| return current; | ||
|
|
||
| current = Directory.GetParent(current)?.FullName; | ||
| } | ||
|
|
||
| return Directory.GetCurrentDirectory(); | ||
| } | ||
| } | ||
|
|
||
| public abstract class YubiKeyHardwareBenchmarkBase | ||
| { | ||
| protected IYubiKey Device { get; private set; } = null!; | ||
|
|
||
| protected void SetupDevice(ConnectionType requiredConnection) | ||
| { | ||
| Device = FindDevice(requiredConnection).GetAwaiter().GetResult(); | ||
| } | ||
|
|
||
| [GlobalCleanup] | ||
| public void Cleanup() => YubiKeyManager.Shutdown(); | ||
|
|
||
| private static async Task<IYubiKey> FindDevice(ConnectionType requiredConnection) | ||
| { | ||
| var devices = await YubiKeyManager.FindAllAsync(ConnectionType.All, forceRescan: true) | ||
| .ConfigureAwait(false); | ||
|
|
||
| return devices.FirstOrDefault(device => device.SupportsConnection(requiredConnection)) | ||
| ?? throw new InvalidOperationException( | ||
| $"No connected YubiKey exposes {requiredConnection}. Connect the YubiKey 5.8 device before running benchmarks."); | ||
| } | ||
| } | ||
|
|
||
| [MemoryDiagnoser] | ||
| [ThreadingDiagnoser] | ||
| [ExceptionDiagnoser] | ||
| public class YubiKeyDiscoveryBenchmarks | ||
| { | ||
| [GlobalSetup] | ||
| public void Setup() | ||
| { | ||
| _ = YubiKeyManager.FindAllAsync(ConnectionType.All, forceRescan: true) | ||
| .GetAwaiter() | ||
| .GetResult(); | ||
| } | ||
|
|
||
| [GlobalCleanup] | ||
| public void Cleanup() => YubiKeyManager.Shutdown(); | ||
|
|
||
| [Benchmark(Baseline = true)] | ||
| public async Task<int> CachedFindAll() | ||
| { | ||
| var devices = await YubiKeyManager.FindAllAsync(ConnectionType.All, forceRescan: false) | ||
| .ConfigureAwait(false); | ||
| return devices.Count; | ||
| } | ||
|
|
||
| [Benchmark] | ||
| public async Task<int> ForcedRescanFindAll() | ||
| { | ||
| var devices = await YubiKeyManager.FindAllAsync(ConnectionType.All, forceRescan: true) | ||
| .ConfigureAwait(false); | ||
| return devices.Count; | ||
| } | ||
|
|
||
| [Benchmark] | ||
| public async Task<int> ColdFindAll() | ||
| { | ||
| await YubiKeyManager.ShutdownAsync().ConfigureAwait(false); | ||
| var devices = await YubiKeyManager.FindAllAsync(ConnectionType.All, forceRescan: false) | ||
| .ConfigureAwait(false); | ||
| return devices.Count; | ||
| } | ||
| } | ||
|
|
||
| [MemoryDiagnoser] | ||
| [ThreadingDiagnoser] | ||
| [ExceptionDiagnoser] | ||
| public class SmartCardManagementBenchmarks : YubiKeyHardwareBenchmarkBase | ||
| { | ||
| [GlobalSetup] | ||
| public void Setup() => SetupDevice(ConnectionType.SmartCard); | ||
|
|
||
| [Benchmark(Baseline = true)] | ||
| public async Task<bool> ConnectSmartCard() | ||
| { | ||
| await using var connection = await Device.ConnectAsync<ISmartCardConnection>().ConfigureAwait(false); | ||
| return connection.SupportsExtendedApdu(); | ||
| } | ||
|
|
||
| [Benchmark] | ||
| public async Task<int> SelectManagementOverSmartCard() | ||
| { | ||
| var connection = await Device.ConnectAsync<ISmartCardConnection>().ConfigureAwait(false); | ||
| using var protocol = PcscProtocolFactory<ISmartCardConnection>.Create().Create(connection); | ||
| var response = await protocol.SelectAsync(ApplicationIds.Management).ConfigureAwait(false); | ||
| return response.Length; | ||
| } | ||
|
|
||
| [Benchmark] | ||
| public async Task<int> GetDeviceInfoOverSmartCard() | ||
| { | ||
| await using var session = await Device.CreateManagementSessionAsync( | ||
| preferredConnection: ConnectionType.SmartCard) | ||
| .ConfigureAwait(false); | ||
|
|
||
| var info = await session.GetDeviceInfoAsync().ConfigureAwait(false); | ||
| return info.SerialNumber ?? 0; | ||
| } | ||
| } | ||
|
|
||
| [MemoryDiagnoser] | ||
| [ThreadingDiagnoser] | ||
| [ExceptionDiagnoser] | ||
| public class FidoHidManagementBenchmarks : YubiKeyHardwareBenchmarkBase | ||
| { | ||
| [GlobalSetup] | ||
| public void Setup() => SetupDevice(ConnectionType.HidFido); | ||
|
|
||
| [Benchmark(Baseline = true)] | ||
| public async Task<int> ConnectFidoHid() | ||
| { | ||
| await using var connection = await Device.ConnectAsync<IFidoHidConnection>().ConfigureAwait(false); | ||
| return connection.PacketSize; | ||
| } | ||
|
|
||
| [Benchmark] | ||
| public async Task<int> CreateManagementSessionOverFidoHid() | ||
| { | ||
| await using var session = await Device.CreateManagementSessionAsync( | ||
| preferredConnection: ConnectionType.HidFido) | ||
| .ConfigureAwait(false); | ||
|
|
||
| return session.FirmwareVersion.Major; | ||
| } | ||
|
|
||
| [Benchmark] | ||
| public async Task<int> GetDeviceInfoOverFidoHid() | ||
| { | ||
| await using var session = await Device.CreateManagementSessionAsync( | ||
| preferredConnection: ConnectionType.HidFido) | ||
| .ConfigureAwait(false); | ||
|
|
||
| var info = await session.GetDeviceInfoAsync().ConfigureAwait(false); | ||
| return info.SerialNumber ?? 0; | ||
| } | ||
| } | ||
|
|
||
| [MemoryDiagnoser] | ||
| [ThreadingDiagnoser] | ||
| [ExceptionDiagnoser] | ||
| public class OtpHidManagementBenchmarks : YubiKeyHardwareBenchmarkBase | ||
| { | ||
| [GlobalSetup] | ||
| public void Setup() => SetupDevice(ConnectionType.HidOtp); | ||
|
|
||
| [Benchmark(Baseline = true)] | ||
| public async Task<int> ConnectOtpHid() | ||
| { | ||
| await using var connection = await Device.ConnectAsync<IOtpHidConnection>().ConfigureAwait(false); | ||
| return connection.FeatureReportSize; | ||
| } | ||
|
|
||
| [Benchmark] | ||
| public async Task<int> CreateManagementSessionOverOtpHid() | ||
| { | ||
| await using var session = await Device.CreateManagementSessionAsync( | ||
| preferredConnection: ConnectionType.HidOtp) | ||
| .ConfigureAwait(false); | ||
|
|
||
| return session.FirmwareVersion.Major; | ||
| } | ||
|
|
||
| [Benchmark] | ||
| public async Task<int> GetDeviceInfoOverOtpHid() | ||
| { | ||
| await using var session = await Device.CreateManagementSessionAsync( | ||
| preferredConnection: ConnectionType.HidOtp) | ||
| .ConfigureAwait(false); | ||
|
|
||
| var info = await session.GetDeviceInfoAsync().ConfigureAwait(false); | ||
| return info.SerialNumber ?? 0; | ||
| } | ||
| } |
23 changes: 23 additions & 0 deletions
23
benchmarks/Yubico.YubiKit.PerformanceBenchmarks/Yubico.YubiKit.PerformanceBenchmarks.csproj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <OutputType>Exe</OutputType> | ||
| <TargetFramework>net10.0</TargetFramework> | ||
| <RootNamespace>Yubico.YubiKit.PerformanceBenchmarks</RootNamespace> | ||
| <AssemblyName>Yubico.YubiKit.PerformanceBenchmarks</AssemblyName> | ||
| <IsPackable>false</IsPackable> | ||
| <PlatformTarget>AnyCPU</PlatformTarget> | ||
| <DebugType>pdbonly</DebugType> | ||
| <DebugSymbols>true</DebugSymbols> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="BenchmarkDotNet" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="../../src/Core/src/Yubico.YubiKit.Core.csproj" /> | ||
| <ProjectReference Include="../../src/Management/src/Yubico.YubiKit.Management.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # Runtime Resilience Learnings | ||
|
|
||
| This folder records phase closeouts for the runtime resilience workstream. Each phase captures what we found, how we got better at finding it, what worked, what did not, what review changed, and what remains risky. | ||
|
|
||
| ## Phases | ||
|
|
||
| - [Phase 1: SmartCard Listener Recovery](phase-01-smartcard-listener.md) | ||
| - [Phase 2: OTP HID Ready-To-Write Polling](phase-02-otp-polling.md) | ||
| - [Phase 3: Static Runtime-Resilience Scanner](phase-03-static-scanner.md) | ||
| - [Phase 4: SmartCard Context Leak Invariant](phase-04-smartcard-context-leak.md) | ||
| - [Phase 5: Minimal Fast Runner](phase-05-minimal-fast-runner.md) | ||
| - [Phase 6: Diagnostics Project Deferred](phase-06-diagnostics-project-deferred.md) | ||
| - [Phase 7: Audit Skill Deferred](phase-07-audit-skill-deferred.md) | ||
|
|
||
| ## Closeout Rule | ||
|
|
||
| Each future phase should close in this order: | ||
|
|
||
| 1. Implement the smallest useful slice. | ||
| 2. Verify with focused and broader project gates. | ||
| 3. Run cross-vendor review when the change affects runtime behavior or proof quality. | ||
| 4. Write the phase learning document. | ||
| 5. Commit the phase before starting the next one. | ||
|
|
||
| ## What The Next Work Should Do | ||
|
|
||
| - Use `dotnet toolchain.cs -- resilience --fast` as the default runtime-resilience verification command. | ||
| - Start from a concrete bug seed or missing invariant, not from a desire to add infrastructure. | ||
| - Prefer no-hardware proof first: fake seams, deterministic timing/resource-release invariants, or seeded scanner fixtures. | ||
| - Use BenchmarkDotNet for discovery and evidence, then convert proven foreground regressions into cheap gates. | ||
| - Keep the diagnostics project and audit skill deferred until the workflow grows beyond the current single-command runner. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.