Skip to content

Commit 87e5292

Browse files
alexylonalexylon
committed
Retry only 5xx; bump LLM timeout to 30 min; Morph falls back to edit_file on any failure
1 parent 2b5db57 commit 87e5292

6 files changed

Lines changed: 209 additions & 126 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ All notable changes to Sofos are documented in this file.
3030

3131
### Changed
3232

33+
- **LLM request timeout raised to 30 minutes; retries dropped from Anthropic and OpenAI, kept only for Morph. Morph ceiling raised to 10 min and now falls back to `edit_file` on any failure.** Previously the three provider clients all shared a single `REQUEST_TIMEOUT = 300s` from `build_http_client`, and every call went through `with_retries` which retried up to three times on any transport failure (timeout, DNS, connection refused) plus 5xx. For Opus 4.7 adaptive thinking at high effort, the 5 min ceiling didn't fit — the client-level `.send()` timed out at ~300s — and the retry then replayed the same long thinking twice more before failing. `REQUEST_TIMEOUT` is now 30 min and applies to Anthropic + OpenAI (reqwest's `.timeout()` is a total-operation deadline, not an idle one, so it has to cover minutes of silent thinking before the first token arrives). Morph has its own `MORPH_REQUEST_TIMEOUT = 600s` client-level ceiling plus an outer 10-min `tokio::time::timeout` in the tool dispatcher (previously 30s, which was too aggressive for large files or backend stalls); any Morph failure — timeout, transport, 4xx, or 5xx — now falls back to a prompt-level hint that steers the model at `edit_file`, the deterministic diff-based editor, rather than propagating as a tool error and stalling the loop. The primary Anthropic and OpenAI endpoints now use a new `send_once` helper that classifies the response but does not retry — a timeout, connect error, or 5xx is surfaced to the user immediately rather than quietly re-running an expensive call. `with_retries` is kept for Morph (5xx only, up to 2 retries with jittered backoff). Response classification moved into `ApiCallError { Transport, ServerError, ClientError }` with body text preserved on error, so the final `SofosError::Api` message still carries the server's explanation. The old `check_response_status` and `is_retryable_error` helpers (the latter had a dead 5xx branch — `reqwest::Client::send()` returns `Ok(Response)` for 5xx, so the retry loop never saw one) were removed.
3334
- **"User declined" prompt phrasing** now nudges the model to pivot rather than retry. The old `Command blocked by user: 'X'` read like a hard policy block and invited the model to reissue the same command. Replaced with `User declined 'X'. Propose a different approach or ask the user to clarify rather than retrying the same command.` at all three rejection sites.
3435
- **`/think` command wording aligned.** The startup banner used `Extended thinking: enabled`; `/think on` printed `Extended thinking enabled.`; `/think off` printed `Extended thinking disabled.`. Consolidated on `Extended thinking: enabled` / `Extended thinking: disabled` everywhere.
3536
- **`read_file` output cap raised to ~256 KB** (64k tokens). Previously `read_file` shared the ~64 KB / 16k-token cap with `execute_bash` and `search_code`, which clipped mid-sized source files — generated code, JSON fixtures, long prompt templates — and forced the model into an extra range-reads round trip against the 200-iteration tool-loop budget. `execute_bash` stdout/stderr and `search_code` keep the 16k-token cap, since verbose test output and broad ripgrep patterns benefit from being forced to narrow rather than handing the model noise.

‎src/api/anthropic.rs‎

Lines changed: 3 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ impl AnthropicClient {
4747
headers.insert("anthropic-version", HeaderValue::from_static(API_VERSION));
4848
headers.insert("anthropic-beta", HeaderValue::from_static(ANTHROPIC_BETA));
4949

50-
let client = utils::build_http_client(headers)?;
50+
let client = utils::build_http_client(headers, utils::REQUEST_TIMEOUT)?;
5151

5252
Ok(Self { client })
5353
}
@@ -87,16 +87,8 @@ impl AnthropicClient {
8787
let url = format!("{}/messages", API_BASE);
8888
let request = Self::prepare_request(request);
8989

90-
let client = self.client.clone();
91-
let response = utils::with_retries("Anthropic", || {
92-
let client = client.clone();
93-
let url = url.clone();
94-
let request = request.clone();
95-
async move { client.post(&url).json(&request).send().await }
96-
})
97-
.await?;
90+
let response = utils::send_once("Anthropic", self.client.post(&url).json(&request)).await?;
9891

99-
let response = utils::check_response_status(response).await?;
10092
let result = response.json::<CreateMessageResponse>().await?;
10193
Ok(result)
10294
}
@@ -117,23 +109,7 @@ impl AnthropicClient {
117109

118110
let url = format!("{}/messages", API_BASE);
119111

120-
let client = self.client.clone();
121-
let response = utils::with_retries("Anthropic", || {
122-
let client = client.clone();
123-
let url = url.clone();
124-
let request = request.clone();
125-
async move {
126-
client
127-
.post(&url)
128-
.json(&request)
129-
.timeout(utils::STREAMING_REQUEST_TIMEOUT)
130-
.send()
131-
.await
132-
}
133-
})
134-
.await?;
135-
136-
let response = utils::check_response_status(response).await?;
112+
let response = utils::send_once("Anthropic", self.client.post(&url).json(&request)).await?;
137113

138114
let mut byte_stream = response.bytes_stream();
139115
let mut buffer = String::new();

‎src/api/morph.rs‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ impl MorphClient {
5454
.map_err(|e| SofosError::Config(format!("Invalid Morph API key: {}", e)))?,
5555
);
5656

57-
let client = utils::build_http_client(headers)?;
57+
let client = utils::build_http_client(headers, utils::MORPH_REQUEST_TIMEOUT)?;
5858

5959
Ok(Self {
6060
client,
@@ -95,11 +95,10 @@ impl MorphClient {
9595
let client = client.clone();
9696
let url = url.clone();
9797
let request = request.clone();
98-
async move { client.post(&url).json(&request).send().await }
98+
async move { utils::send_classified(client.post(&url).json(&request)).await }
9999
})
100100
.await?;
101101

102-
let response = utils::check_response_status(response).await?;
103102
let result: MorphResponse = response.json().await?;
104103

105104
let choice = result

‎src/api/openai.rs‎

Lines changed: 2 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ impl OpenAIClient {
2121
.map_err(|e| SofosError::Config(format!("Invalid OpenAI API key: {}", e)))?,
2222
);
2323

24-
let client = utils::build_http_client(headers)?;
24+
let client = utils::build_http_client(headers, utils::REQUEST_TIMEOUT)?;
2525

2626
Ok(Self { client })
2727
}
@@ -104,16 +104,8 @@ impl OpenAIClient {
104104
eprintln!("======================================\n");
105105
}
106106

107-
let client = self.client.clone();
108-
let response = utils::with_retries("OpenAI", || {
109-
let client = client.clone();
110-
let url = url.clone();
111-
let body = body.clone();
112-
async move { client.post(&url).json(&body).send().await }
113-
})
114-
.await?;
107+
let response = utils::send_once("OpenAI", self.client.post(&url).json(&body)).await?;
115108

116-
let response = utils::check_response_status(response).await?;
117109
let response_text = response.text().await?;
118110

119111
if std::env::var("SOFOS_DEBUG").is_ok() {

0 commit comments

Comments
 (0)