add(tts): return concatenation manifests and retry helper

Author: Tom-Ryder

Sources/AgentRunKit/Documentation.docc/AgentRunKit.md

@@ -104,6 +104,7 @@ For a complete walkthrough, see <doc:GettingStarted>.
 - ``VertexGoogleClient``
 - ``ResponsesAPIClient``
 - ``RetryPolicy``
+- ``HTTPDataRetry``
 - ``GoogleAuthService``
 
 ### Provider Capabilities

Sources/AgentRunKit/Documentation.docc/Articles/MultimodalAndAudio.md

@@ -118,6 +118,7 @@ These methods cover different use cases:
 | `generate(text:voice:options:)` | `Data` | Single request, no chunking |
 | `stream(text:voice:options:)` | `AsyncThrowingStream<TTSSegment, Error>` | Chunked, yields ordered ``TTSSegment`` values as they complete |
 | `generateAll(text:voice:options:)` | `Data` | Chunked, concatenates all segments into one `Data` |
+| `generateWithManifest(text:voice:options:)` | ``TTSConcatenationResult`` | Like `generateAll` but also returns a per-segment manifest of chunk, encoding, and timing |
 | `chunks(for:)` | `[TTSChunk]` | The chunk plan this client will use, without invoking the provider |
 
 ```swift
@@ -134,10 +135,25 @@ for try await segment in tts.stream(text: longArticle) {
 // Full concatenated output
 let fullAudio = try await tts.generateAll(text: longArticle, options: TTSOptions(speed: 1.25))
 
+// Concatenated audio plus a per-segment manifest
+let result = try await tts.generateWithManifest(text: longArticle, options: TTSOptions(responseFormat: .pcm))
+for entry in result.manifest {
+    if let range = entry.timing.byteRangeInConcatenatedAudio {
+        print("chunk \(entry.chunk.index): bytes \(range) of result.audio")
+    }
+}
+
 // Forecast the chunk plan without generating audio
 let plan = tts.chunks(for: longArticle)
 ```
 
+`generateWithManifest` populates ``TTSSegmentTiming/byteRangeInConcatenatedAudio`` for `pcm`
+output today. Other formats leave it `nil` until the framework can compute byte ranges defensibly.
+`generateAll` is implemented on top of the same path and returns `result.audio`.
+
+`stream` segments always carry ``TTSSegmentTiming/uncomputed`` timing. Per-segment audio is the
+raw chunk bytes, and final container offsets are only meaningful after concatenation.
+
 ### TTSOptions
 
 ``TTSOptions`` controls per-request parameters:
@@ -147,13 +163,26 @@ let plan = tts.chunks(for: longArticle)
 
 ### How Chunking Works
 
-The chunker splits input text on sentence boundaries using `NLTokenizer`. Sentences are packed into chunks up to the provider's `maxChunkCharacters` limit. Oversized sentences fall back to word-level, then character-level splitting. ``TTSClient`` dispatches up to `maxConcurrent` chunk requests in parallel using a task group. Results are buffered and yielded in original order.
+The chunker splits input text on sentence boundaries using `NLTokenizer`. Sentences are packed up to
+the provider's `maxChunkCharacters` limit. Oversized sentences fall back to word-level, then
+character-level splitting.
+
+``TTSClient`` dispatches up to `maxConcurrent` chunk requests in parallel. Results are buffered and
+yielded in original order.
 
-Each ``TTSSegment`` aggregates a ``TTSChunk`` (the unit of input text), a ``TTSAudioEncoding`` (the encoding ``TTSClient`` requested from the provider), a ``TTSSegmentTiming`` (audio-time metadata; both fields are `nil` until the framework computes them), and the audio bytes. The chunk, encoding, and timing are the canonical access path; flat computed properties on ``TTSSegment`` (`index`, `total`, `text`, `sourceRange`) forward to the chunk for log statements that need only those fields. For force-split chunks, `text` normalizes whitespace to single spaces while `sourceRange` covers the discontiguous span of the words it contains, preserving left-to-right monotonicity for caller-side highlighting and forced alignment.
+Each ``TTSSegment`` carries a ``TTSChunk``, a ``TTSAudioEncoding``, a ``TTSSegmentTiming``, and the
+audio bytes. The chunk, encoding, and timing fields are the canonical access path; flat properties
+on ``TTSSegment`` forward to the chunk for compact logging.
 
-``TTSClient/chunks(for:)`` returns the same ``TTSChunk`` values the stream will emit, without calling the provider. Use it to forecast chunk identity before generation or to drive offline planning.
+For force-split chunks, `text` normalizes whitespace to single spaces while `sourceRange` covers the
+span of the words it contains. That keeps ranges monotonic for caller-side highlighting and forced
+alignment.
 
-``TTSConcatenationResult`` and ``TTSManifestEntry`` describe the shape of a manifest-aware concatenation that pairs the audio bytes with a per-segment manifest of chunk, encoding, and timing.
+``TTSClient/chunks(for:)`` returns the same ``TTSChunk`` values the stream will emit, without calling
+the provider. Use it to forecast chunk identity before generation or to drive offline planning.
+
+``TTSConcatenationResult`` and ``TTSManifestEntry`` pair concatenated audio bytes with a per-segment
+manifest of chunk, encoding, and timing.
 
 For MP3 output, the concatenator strips ID3v2 headers, Xing/Info frames, and ID3v1 tails from interior segments for clean concatenation.
 
@@ -185,6 +214,19 @@ let provider = MyTTSProvider(config: TTSProviderConfig(
 let tts = TTSClient(provider: provider)
 ```
 
+For HTTP-backed providers, ``HTTPDataRetry`` exposes the same retry primitive
+``OpenAITTSProvider`` uses: exponential backoff with jitter and `Retry-After`-aware handling of
+429 responses. Pass a ``RetryPolicy`` and receive `(Data, HTTPURLResponse)` on success or a
+``TransportError`` on failure; cancellation propagates through `CancellationError`.
+
+```swift
+let (data, response) = try await HTTPDataRetry.perform(
+    urlRequest: request,
+    session: .shared,
+    retryPolicy: .default
+)
+```
+
 ## See Also
 
 - <doc:AgentAndChat>
@@ -204,3 +246,4 @@ let tts = TTSClient(provider: provider)
 - ``TTSManifestEntry``
 - ``TTSConcatenationResult``
 - ``TTSOptions``
+- ``HTTPDataRetry``

Sources/AgentRunKit/LLM/Transport/HTTPDataRetry.swift

@@ -0,0 +1,23 @@
+import Foundation
+
+/// A public retry facade shared by ``OpenAITTSProvider`` and custom HTTP-backed providers.
+public enum HTTPDataRetry: Sendable {
+    /// Sends the request with retry, throwing bare ``TransportError`` or `CancellationError`.
+    public static func perform(
+        urlRequest: URLRequest,
+        session: URLSession,
+        retryPolicy: RetryPolicy
+    ) async throws -> (Data, HTTPURLResponse) {
+        do {
+            return try await HTTPRetry.performData(
+                urlRequest: urlRequest,
+                session: session,
+                retryPolicy: retryPolicy
+            )
+        } catch is CancellationError {
+            throw CancellationError()
+        } catch let AgentError.llmError(transportError) {
+            throw transportError
+        }
+    }
+}

Sources/AgentRunKit/LLM/Transport/HTTPRetry.swift

@@ -26,6 +26,9 @@ enum HTTPRetry {
             do {
                 (data, response) = try await session.data(for: urlRequest)
             } catch {
+                if isCancellation(error) {
+                    throw CancellationError()
+                }
                 lastError = TransportError.networkError(error)
                 continue
             }
@@ -79,6 +82,9 @@ enum HTTPRetry {
             do {
                 (bytes, response) = try await session.bytes(for: urlRequest)
             } catch {
+                if isCancellation(error) {
+                    throw CancellationError()
+                }
                 lastError = TransportError.networkError(error)
                 continue
             }
@@ -165,6 +171,13 @@ enum HTTPRetry {
         return nil
     }
 
+    static func isCancellation(_ error: any Error) -> Bool {
+        if error is CancellationError {
+            return true
+        }
+        return (error as? URLError)?.code == .cancelled
+    }
+
     static func collectErrorBody(from bytes: URLSession.AsyncBytes) async -> String {
         await withTaskGroup(of: String?.self) { group in
             group.addTask {

Sources/AgentRunKit/TTS/OpenAITTSProvider.swift

@@ -52,18 +52,12 @@ public struct OpenAITTSProvider: TTSProvider, Sendable {
             encoding: context.encoding
         )
 
-        do {
-            let (data, _) = try await HTTPRetry.performData(
-                urlRequest: urlRequest, session: session, retryPolicy: retryPolicy
-            )
-            return data
-        } catch is CancellationError {
-            throw CancellationError()
-        } catch let AgentError.llmError(transportError) {
-            throw transportError
-        } catch {
-            throw TransportError.other(String(describing: error))
-        }
+        let (data, _) = try await HTTPDataRetry.perform(
+            urlRequest: urlRequest,
+            session: session,
+            retryPolicy: retryPolicy
+        )
+        return data
     }
 
     func buildURLRequest(

Sources/AgentRunKit/TTS/TTSClient.swift

@@ -94,19 +94,54 @@ public struct TTSClient<P: TTSProvider>: Sendable {
         voice: String? = nil,
         options: TTSOptions = TTSOptions()
     ) async throws -> Data {
+        try await generateWithManifest(text: text, voice: voice, options: options).audio
+    }
+
+    /// Synthesizes the input and returns concatenated audio plus a per-segment manifest; chunk failure throws.
+    public func generateWithManifest(
+        text: String,
+        voice: String? = nil,
+        options: TTSOptions = TTSOptions()
+    ) async throws -> TTSConcatenationResult {
         var segments: [TTSSegment] = []
         for try await segment in stream(text: text, voice: voice, options: options) {
             segments.append(segment)
         }
 
         let effectiveFormat = options.responseFormat ?? provider.config.defaultFormat
-        if effectiveFormat == .mp3 {
-            return MP3Concatenator.concatenate(segments.map(\.audio))
+        let audio: Data = if effectiveFormat == .mp3 {
+            MP3Concatenator.concatenate(segments.map(\.audio))
+        } else {
+            Self.appendingConcatenation(segments.map(\.audio))
         }
 
-        var result = Data()
+        var manifest: [TTSManifestEntry] = []
+        manifest.reserveCapacity(segments.count)
+        var pcmCursor = 0
         for segment in segments {
-            result.append(segment.audio)
+            let timing: TTSSegmentTiming
+            if effectiveFormat == .pcm {
+                let lower = pcmCursor
+                pcmCursor += segment.audio.count
+                timing = TTSSegmentTiming(byteRangeInConcatenatedAudio: lower ..< pcmCursor)
+            } else {
+                timing = .uncomputed
+            }
+            manifest.append(TTSManifestEntry(
+                chunk: segment.chunk,
+                encoding: segment.encoding,
+                timing: timing
+            ))
+        }
+
+        return TTSConcatenationResult(audio: audio, manifest: manifest)
+    }
+
+    private static func appendingConcatenation(_ audioSegments: [Data]) -> Data {
+        var result = Data()
+        result.reserveCapacity(audioSegments.reduce(0) { $0 + $1.count })
+        for audioSegment in audioSegments {
+            result.append(audioSegment)
         }
         return result
     }