What would you like to be added?
Provide a simple way to control client-side timeouts for server requests made by the CLI (e.g., generateContentStream, generateContent, and any other backend API calls).
Introduce a Mendel (experiment) flag to control these timeouts globally.
Additionally, control the total waiting time for a server response, including retries. If the cumulative duration exceeds a defined threshold (e.g., thresholdSec), the system should stop retrying and cancel the operation.
Why is this needed?
Currently, timeouts are not globally configurable for all server requests, and retries can potentially continue for a long time without a cap on the total duration. Providing these controls allows for better resource management and prevents sessions from hanging indefinitely in suboptimal network conditions or during server unavailability.
Additional context?
The Mendel flag should allow for experimentation and gradual rollout of these global timeout settings.
The cumulative duration check should be integrated into the core retry logic (e.g., retryWithBackoff in packages/core/src/utils/retry.ts) or the base client configuration so that it applies uniformly to all network requests.
What would you like to be added?
Provide a simple way to control client-side timeouts for server requests made by the CLI (e.g.,
generateContentStream,generateContent, and any other backend API calls).Introduce a Mendel (experiment) flag to control these timeouts globally.
Additionally, control the total waiting time for a server response, including retries. If the cumulative duration exceeds a defined threshold (e.g.,
thresholdSec), the system should stop retrying and cancel the operation.Why is this needed?
Currently, timeouts are not globally configurable for all server requests, and retries can potentially continue for a long time without a cap on the total duration. Providing these controls allows for better resource management and prevents sessions from hanging indefinitely in suboptimal network conditions or during server unavailability.
Additional context?
The Mendel flag should allow for experimentation and gradual rollout of these global timeout settings.
The cumulative duration check should be integrated into the core retry logic (e.g.,
retryWithBackoffinpackages/core/src/utils/retry.ts) or the base client configuration so that it applies uniformly to all network requests.