perf(sync): single-shot DCV retry — drop per-order challenge polling

PerformDcvIfNeededAsync now accepts an optional waitForChallengeSecondsOverride
parameter. Enroll continues to pass null, preserving the configured
DcvWaitForChallengeSeconds budget (default 60s) for one-shot end-to-end
issuance. TryRunDcvDuringSyncAsync passes 0, which forces a single
TrackOrder call per pending order — if CERTInext hasn't yet exposed the DCV
slot, sync moves on and the next sync cycle will pick the order up.

Motivation: with the previous code, every pending order in
Synchronize/GetSingleRecord could spend up to 60s polling for the DCV
challenge to materialize. On accounts with many pending orders this scaled
poorly — a live DCV-on sync against ~150 pending orders took 2:35.

After: the same sync completes in ~2s of plugin time per record (one
TrackOrder per pending order, no per-order delay loop). Already-issued
orders were and remain skipped (the gate at Synchronize line 719 only
runs the retry for EXTERNALVALIDATION).

Tests:
- New SyncDcvRetry_DoesSingleShotTrackOrder_WhenChallengeNotReady pins
  the single-shot behaviour: even with DcvWaitForChallengeSeconds=60 in
  config, the sync path makes exactly ONE TrackOrder call for an order
  whose domainVerification is null and exits in <10s wall-clock.
- Existing happy-path tests still exercise the Enroll budget (which keeps
  the full polling behaviour) via the null override.

146/146 unit tests pass.

Author: spbsoluble

CERTInext.Tests/CERTInextCAPluginDcvTests.cs

@@ -395,6 +395,65 @@ public void SetDomainValidatorFactory_SecondCall_OverridesFirst()
                 "the most recent SetDomainValidatorFactory call must replace the earlier one");
         }
 
+        // ---------------------------------------------------------------------------
+        // Sync path is single-shot for the DCV challenge wait
+        // ---------------------------------------------------------------------------
+
+        [Fact]
+        public async Task SyncDcvRetry_DoesSingleShotTrackOrder_WhenChallengeNotReady()
+        {
+            // Sync MUST NOT poll the configured DcvWaitForChallengeSeconds budget per
+            // pending order — that would scale O(orders × 60s) per cycle and tie up
+            // gateway threads for minutes per sync.  When TrackOrder returns null
+            // domainVerification, sync exits immediately and lets the next sync cycle
+            // pick the order up.
+            var mock = NewMock();
+
+            // High config budget — would normally drive 6+ polls × 5s waits.  The sync
+            // override of 0 must prevent that.
+            var config = DcvConfig(dcvWaitForChallengeSeconds: 60);
+
+            mock.Setup(c => c.TrackOrderAsync(MockCertificateData.DcvOrderId, It.IsAny<CancellationToken>()))
+                .ReturnsAsync(new TrackOrderResponse
+                {
+                    OrderDetails = new TrackOrderResponseDetails
+                    {
+                        OrderStatusId       = "1",
+                        CertificateStatusId = "1",
+                        DomainVerification  = null
+                    }
+                });
+
+            // GetSingleRecord calls GetCertificateAsync first to materialize the record;
+            // the sync-DCV-retry kicks in afterwards.  The pending response keeps the
+            // retry path engaged so we exercise the override.
+            mock.Setup(c => c.GetCertificateAsync(MockCertificateData.DcvOrderId, It.IsAny<CancellationToken>()))
+                .ReturnsAsync(MockCertificateData.PendingCertRecord(MockCertificateData.DcvOrderId));
+
+            var validator = new FakeDomainValidator();
+            var plugin = BuildPlugin(mock.Object, new FakeDomainValidatorFactory(validator), config);
+
+            var sw = System.Diagnostics.Stopwatch.StartNew();
+            // GetSingleRecord calls TryRunDcvDuringSyncAsync internally — which is the
+            // sync-style path with waitForChallengeSecondsOverride=0.
+            var record = await plugin.GetSingleRecord(MockCertificateData.DcvOrderId);
+            sw.Stop();
+
+            record.Should().NotBeNull();
+            // The 0-budget single shot must complete well under the 60s config budget.
+            // Use a generous 10s ceiling to tolerate slow CI hosts; the actual cost is
+            // ~1 TrackOrder.  Without the override we'd be ≥60s.
+            sw.Elapsed.Should().BeLessThan(TimeSpan.FromSeconds(10),
+                "sync's DCV retry must be single-shot, not poll the configured challenge budget");
+
+            mock.Verify(c => c.TrackOrderAsync(MockCertificateData.DcvOrderId, It.IsAny<CancellationToken>()),
+                Times.Exactly(1),
+                "PerformDcvIfNeededAsync's single-shot challenge check must make exactly ONE " +
+                "TrackOrder call when waitForChallengeSecondsOverride=0 and the slot is null. " +
+                "Without the override, the polling loop would issue many more calls within " +
+                "the 60s budget.");
+        }
+
         // ---------------------------------------------------------------------------
         // Failure modes
         // ---------------------------------------------------------------------------

CERTInext/CERTInextCAPlugin.cs

@@ -1059,7 +1059,15 @@ public DomainValidatorConfigProvider(Dictionary<string, object> config)
         ///   * a bounded DCV timeout linked to the caller's cancellation token,
         ///   * swallowing of non-cancellation exceptions so a single bad order does not
         ///     halt a 12-hour sync — the order will be retried on the next cycle.
-        /// Returns <c>true</c> when DCV actually executed, <c>false</c> when skipped.
+        ///
+        /// Uses a single-shot challenge check (<c>waitForChallengeSeconds=0</c>) by default
+        /// because sync runs periodically: if CERTInext hasn't yet exposed the DCV slot for
+        /// this order, the next sync cycle will pick it up.  Waiting per-order during sync
+        /// scales poorly — a single pending order's 60s budget becomes minutes of wasted
+        /// gateway thread time across an account with many orders.  See PR #2 discussion.
+        ///
+        /// Returns <c>true</c> when DCV actually executed (or DCV is already complete),
+        /// <c>false</c> when skipped.
         /// </summary>
         private async Task<bool> TryRunDcvDuringSyncAsync(string orderNumber, CancellationToken ct)
         {
@@ -1081,10 +1089,11 @@ private async Task<bool> TryRunDcvDuringSyncAsync(string orderNumber, Cancellati
                 dcvCts.CancelAfter(TimeSpan.FromMinutes(timeoutMinutes));
 
                 _logger.LogInformation(
-                    "Attempting deferred DCV during sync/refresh. OrderNumber={OrderNumber}, DcvTimeoutMinutes={Timeout}",
+                    "Attempting deferred DCV during sync/refresh (single-shot challenge check). " +
+                    "OrderNumber={OrderNumber}, DcvTimeoutMinutes={Timeout}",
                     orderNumber, timeoutMinutes);
 
-                return await PerformDcvIfNeededAsync(orderNumber, dcvCts.Token);
+                return await PerformDcvIfNeededAsync(orderNumber, dcvCts.Token, waitForChallengeSecondsOverride: 0);
             }
             catch (OperationCanceledException) when (ct.IsCancellationRequested)
             {
@@ -1110,15 +1119,26 @@ private async Task<bool> TryRunDcvDuringSyncAsync(string orderNumber, Cancellati
         ///
         /// Rule: if the order is already issued we never attempt DCV — it would be a no-op
         /// at best and could confuse the CA at worst.
+        ///
+        /// <paramref name="waitForChallengeSecondsOverride"/> lets the sync path force a
+        /// single-shot challenge check (pass <c>0</c>) so a sync cycle doesn't spend up to
+        /// <c>DcvWaitForChallengeSeconds</c> per pending order waiting for CERTInext to
+        /// expose the DCV slot — sync runs periodically, so unexposed orders are picked up
+        /// on the next cycle instead.  Enroll passes <c>null</c> to keep the full configured
+        /// budget (user-visible latency benefits from a one-shot end-to-end finish).
         /// </summary>
-        private async Task<bool> PerformDcvIfNeededAsync(string orderNumber, CancellationToken ct)
+        private async Task<bool> PerformDcvIfNeededAsync(
+            string orderNumber,
+            CancellationToken ct,
+            int? waitForChallengeSecondsOverride = null)
         {
             // Poll TrackOrder until CERTInext exposes the DCV challenge (domainVerification
             // populated) OR the cert reaches a terminal state OR the wait budget expires.
             // Under concurrent enrollment load CERTInext sometimes takes a few seconds to
             // materialize the slot after GenerateOrderSSL returns — without this wait a
             // race-condition order skips DCV entirely and waits for the next sync cycle.
-            int waitBudgetSeconds = _config.GetEffectiveDcvWaitForChallengeSeconds();
+            int waitBudgetSeconds = waitForChallengeSecondsOverride
+                ?? _config.GetEffectiveDcvWaitForChallengeSeconds();
             // Challenge-wait poll interval is clamped to [1s, 5s] so it's responsive even
             // when an admin has set DcvPropagationDelaySeconds high for slow zones (that
             // setting governs how long we wait *after* publishing a TXT record, which is a