Add adaptive embedding throughput shaping for Azure 429 limits (#1115)

## Why
The previous retry-only fix still failed under sustained S0 throttling:
large embedding requests kept exhausting retries at the same payload
size. We need throughput shaping so rebuilds can continue progressing
under rate limits instead of stalling at repeated 429 exhaustion.

## What changed
- Added adaptive batch downshifting in embedding rebuilds:
  - starts at configured max batch size
- on 429/RateLimitReached, splits throttled batches and retries smaller
sub-batches
- reuses the smaller successful size for subsequent requests in the same
run
  - fails clearly if batch size 1 still exhausts retries
- Added explicit request pacing controls:
  - `AIOptions:EmbeddingRetry:MaxEmbeddingBatchSize` (default 2048)
  - `AIOptions:EmbeddingRetry:MinInterRequestDelayMs` (default 250)
- embedding requests are serialized and paced between calls to reduce
sustained RPM pressure
- Hardened Retry-After parsing:
  - supports `retry-after`, `retry-after-ms`, `x-ms-retry-after-ms`
- supports extracting `retry after N seconds` from exception message
text
- Added coarse progress logging during rebuilds (not per call):
  - logs start configuration
  - logs progress at 10% milestones when total count is known
  - falls back to every 500 chunks when total count is unknown
  - includes current adaptive batch size in progress messages

## Validation
- `dotnet build
EssentialCSharp.Chat.Shared/EssentialCSharp.Chat.Common.csproj -c
Release --nologo`
- `dotnet test
EssentialCSharp.Chat.Tests/EssentialCSharp.Chat.Tests.csproj -c Release
--no-restore -v q`

Both passed.

Author: BenjaminMichaelis

EssentialCSharp.Chat.Shared/Extensions/ServiceCollectionExtensions.cs

@@ -100,8 +100,8 @@ public static IServiceCollection AddAzureOpenAIServices(
         // Configure AI options from configuration
         services.Configure<AIOptions>(configuration.GetSection("AIOptions"));
 
-        // Configure retry options from configuration section
-        // Environment variables like EmbeddingRetry:MaxRetries will override defaults
+        // Configure retry options from configuration section.
+        // Environment variables can override via AIOptions__EmbeddingRetry__*.
         services.AddOptions<EmbeddingRetryOptions>()
             .Bind(configuration.GetSection(EmbeddingRetryOptions.SectionPath))
             .ValidateDataAnnotations()

EssentialCSharp.Chat.Shared/Models/EmbeddingRetryOptions.cs

@@ -34,6 +34,20 @@ public sealed class EmbeddingRetryOptions
     [Range(1, 600000)]
     public int MaxDelayMs { get; set; } = 60000;
 
+    /// <summary>
+    /// Maximum embedding request payload size sent per API call.
+    /// The service may adaptively downshift below this value when throttled.
+    /// </summary>
+    [Range(1, 2048)]
+    public int MaxEmbeddingBatchSize { get; set; } = 2048;
+
+    /// <summary>
+    /// Minimum delay between embedding API requests in milliseconds.
+    /// This adds request pacing to reduce sustained rate-limit pressure.
+    /// </summary>
+    [Range(0, 600000)]
+    public int MinInterRequestDelayMs { get; set; } = 250;
+
     /// <summary>
     /// Exponential backoff multiplier. Each retry delay is multiplied by this value.
     /// For example, with baseDelay=1000ms and multiplier=2.0:
@@ -74,6 +88,15 @@ public void Validate()
         if (BaseDelayMs > MaxDelayMs)
             throw new InvalidOperationException("BaseDelayMs must be less than or equal to MaxDelayMs.");
 
+        if (MaxEmbeddingBatchSize <= 0)
+            throw new InvalidOperationException("MaxEmbeddingBatchSize must be positive.");
+
+        if (MaxEmbeddingBatchSize > 2048)
+            throw new InvalidOperationException("MaxEmbeddingBatchSize cannot exceed Azure embedding API limit (2048).");
+
+        if (MinInterRequestDelayMs < 0)
+            throw new InvalidOperationException("MinInterRequestDelayMs must be non-negative.");
+
         if (BackoffMultiplier < 1.0)
             throw new InvalidOperationException("BackoffMultiplier must be >= 1.0.");