Skip to content

Commit a4a51b8

Browse files
acking-youclaude
acking-you
and
claude
authored
feat(llm-access): add standalone Codex image gateway (#52)
* feat(llm-access): add standalone Codex image gateway * fix(llm-access): address codex image gateway review * feat(llm-access): track codex image key usage * feat(llm-access): share codex image gateway * refactor(codex-image): tidy shared gateway and document the dual-mode design Apply low-risk review improvements and add the detailed comments the shared gateway was missing: - Decouple the per-key image usage rollup from the response hot path (fire-and-forget spawn) so the account/key concurrency permits release and the image bytes return without waiting on a control-plane write. The rollup is an admin-visibility metric, not a billing/quota gate. - Reclaim ImageKeyLimiter map entries once a key is idle and carries no min-start-interval clock (stamp last_start only when an interval is configured), fixing the asymmetric unbounded growth vs ImageAccountLimiter. - Rebuild hydrate_codex_image_route with struct-update (..candidate) so new ProviderCodexRoute fields are carried through automatically. - Strip the standalone/direct Codex image toggles in normalize_kiro_key_patch so Codex-only flags never persist onto a Kiro key. - Document the dual-mode gateway, the two per-key gates, the dispatch/failover flow, the limiter semantics, and the deployment caveats (0032 backfill, path-split Caddy routing, per-process image-log dir). - Convert unwrap()/unwrap_err() in the new gateway tests to expect/expect_err for the workspace unwrap_used lint under -D warnings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(codex-image): dedupe helpers and tighten request-id/base64 hot paths Reduce duplication and tidy conventions in the shared image gateway: - Route the gateway's and session.rs's copies of the header-value helper to the existing shared llm_access_codex::request::extract_header_value, and use the shared codex_user_agent directly instead of a local wrapper. - Consolidate now_ms() (duplicated in gateway.rs + logging.rs) and the poison-safe lock_unpoisoned() (a limiter local plus two inline copies) into a small crate-internal util module. - Make request ids collision-free within a millisecond via a per-process atomic counter, and count base64 padding over bytes instead of chars to skip UTF-8 decoding of multi-megabyte image payloads. - Refresh the crate module doc to describe the shared dual-entrypoint design. No behavior change; verified via clippy -D warnings and the codex-image / codex / provider test suites. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(codex-image): trace requests and record integrated-mode usage Optimize observability and analytics for the Codex image gateway. Tracing: emit a structured info! line on every completed image request (request_id, mode, key, account, endpoint, status, failover, duration, image_count, usage_tokens), a warn! on account exhaustion, an info! on per-key limit rejection, and a debug! per candidate dispatch, so a request's status is traceable end to end. Integrated-mode usage: when running inside the main Codex API binary (IntegratedCodexApi) emit one UsageEvent per request via the usage journal, so image traffic lands in the DuckDB usage log alongside text/Kiro requests. gpt-image usage tokens are folded into the output and billable columns (Codex bills image generation by token, so it counts toward the key quota); failures carry the upstream error message/class for inline display. The standalone binary has no usage worker and keeps relying on the JSONL image log + per-key image rollup. Image visualization: add a response_image_count field to UsageEvent, persisted through the journal wire and DuckDB (migration 0005, appended at the tail of the summary/detail selects so no positional index shifts), surfaced on the public/admin usage DTOs, and rendered in the usage UI as a "生图 xN" badge plus an image-count line in the token column. Round-trip tests cover the new field through both the journal postcard codec and the DuckDB summary/detail decode paths. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(usage): preserve image metrics during duckdb compaction --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 3350d78 commit a4a51b8

79 files changed

Lines changed: 4407 additions & 103 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Cargo.lock‎

Lines changed: 22 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ members = [
1515
"crates/llm-access",
1616
"crates/llm-access-core",
1717
"crates/llm-access-codex",
18+
"crates/llm-access-codex-image",
1819
"crates/llm-access-kiro",
1920
"crates/llm-access-migrations",
2021
"crates/llm-access-store",

‎crates/frontend/src/api.rs‎

Lines changed: 109 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5972,7 +5972,8 @@ pub struct PublicLlmGatewayUsageEventView {
59725972
pub credit_usage_missing: bool,
59735973
pub client_ip: String,
59745974
pub ip_region: String,
5975-
/// Inline error message surfaced directly in the list (no detail view needed).
5975+
/// Inline error message surfaced directly in the list (no detail view
5976+
/// needed).
59765977
#[serde(default)]
59775978
pub error_message: Option<String>,
59785979
/// Stable upstream error class for failed requests, when classified.
@@ -5981,6 +5982,9 @@ pub struct PublicLlmGatewayUsageEventView {
59815982
/// Whether this event belongs to a permanently rejected Codex session.
59825983
#[serde(default)]
59835984
pub session_blocked: bool,
5985+
/// Number of images returned by a Codex image generation/edit request.
5986+
#[serde(default)]
5987+
pub response_image_count: Option<i64>,
59845988
pub created_at: i64,
59855989
}
59865990

@@ -6066,6 +6070,10 @@ const fn default_true() -> bool {
60666070
true
60676071
}
60686072

6073+
const fn default_codex_image_generation_max_concurrency() -> u64 {
6074+
3
6075+
}
6076+
60696077
fn default_kiro_pool_strategy() -> String {
60706078
llm_store::default_kiro_pool_strategy()
60716079
}
@@ -6095,6 +6103,12 @@ pub struct AdminLlmGatewayKeyView {
60956103
pub usage_output_tokens: u64,
60966104
pub usage_credit_total: f64,
60976105
pub usage_credit_missing_events: u64,
6106+
#[serde(default)]
6107+
pub codex_image_usage_tokens: u64,
6108+
#[serde(default)]
6109+
pub codex_image_usage_missing_events: u64,
6110+
#[serde(default)]
6111+
pub codex_image_last_used_at: Option<i64>,
60986112
pub remaining_billable: i64,
60996113
pub last_used_at: Option<i64>,
61006114
pub created_at: i64,
@@ -6113,6 +6127,12 @@ pub struct AdminLlmGatewayKeyView {
61136127
#[serde(default)]
61146128
pub codex_strict_session_rejection_enabled: bool,
61156129
#[serde(default = "default_true")]
6130+
pub codex_image_generation_enabled: bool,
6131+
#[serde(default = "default_true")]
6132+
pub codex_image_standalone_generation_enabled: bool,
6133+
#[serde(default)]
6134+
pub codex_image_direct_generation_enabled: bool,
6135+
#[serde(default = "default_true")]
61166136
pub kiro_request_validation_enabled: bool,
61176137
#[serde(default = "default_true")]
61186138
pub kiro_cache_estimation_enabled: bool,
@@ -6170,6 +6190,10 @@ pub struct AdminLlmGatewayKeysSummaryView {
61706190
pub usage_billable_tokens_sum: u64,
61716191
pub usage_credit_total: f64,
61726192
pub usage_credit_missing_events: u64,
6193+
#[serde(default)]
6194+
pub codex_image_usage_tokens_sum: u64,
6195+
#[serde(default)]
6196+
pub codex_image_usage_missing_events: u64,
61736197
}
61746198

61756199
/// Combined admin payload for the key inventory screen.
@@ -6324,7 +6348,8 @@ pub struct AdminLlmGatewayUsageEventView {
63246348
pub client_ip: String,
63256349
pub ip_region: String,
63266350
pub last_message_content: Option<String>,
6327-
/// Inline error message surfaced directly in the list (no detail view needed).
6351+
/// Inline error message surfaced directly in the list (no detail view
6352+
/// needed).
63286353
#[serde(default)]
63296354
pub error_message: Option<String>,
63306355
/// Stable upstream error class for failed requests, when classified.
@@ -6333,6 +6358,9 @@ pub struct AdminLlmGatewayUsageEventView {
63336358
/// Whether this event belongs to a permanently rejected Codex session.
63346359
#[serde(default)]
63356360
pub session_blocked: bool,
6361+
/// Number of images returned by a Codex image generation/edit request.
6362+
#[serde(default)]
6363+
pub response_image_count: Option<i64>,
63366364
pub created_at: i64,
63376365
}
63386366

@@ -6382,6 +6410,8 @@ pub struct AdminLlmGatewayUsageEventDetailView {
63826410
pub error_message: Option<String>,
63836411
pub error_class: Option<String>,
63846412
pub session_blocked: bool,
6413+
#[serde(default)]
6414+
pub response_image_count: Option<i64>,
63856415
pub error_body: Option<String>,
63866416
pub response_body: Option<String>,
63876417
pub created_at: i64,
@@ -8956,6 +8986,9 @@ pub async fn create_admin_llm_gateway_key(
89568986
usage_output_tokens: 0,
89578987
usage_credit_total: 0.0,
89588988
usage_credit_missing_events: 0,
8989+
codex_image_usage_tokens: 0,
8990+
codex_image_usage_missing_events: 0,
8991+
codex_image_last_used_at: None,
89598992
remaining_billable: quota_billable_limit as i64,
89608993
last_used_at: None,
89618994
created_at: 0,
@@ -8985,6 +9018,9 @@ pub async fn create_admin_llm_gateway_key(
89859018
uses_global_kiro_billable_model_multipliers: true,
89869019
codex_fast_enabled: true,
89879020
codex_strict_session_rejection_enabled: false,
9021+
codex_image_generation_enabled: true,
9022+
codex_image_standalone_generation_enabled: true,
9023+
codex_image_direct_generation_enabled: false,
89889024
kiro_candidate_credit_summary: None,
89899025
})
89909026
}
@@ -9032,6 +9068,9 @@ pub struct PatchAdminLlmGatewayKeyRequest<'a> {
90329068
pub request_min_start_interval_ms: Option<u64>,
90339069
pub codex_fast_enabled: Option<bool>,
90349070
pub codex_strict_session_rejection_enabled: Option<bool>,
9071+
pub codex_image_generation_enabled: Option<bool>,
9072+
pub codex_image_standalone_generation_enabled: Option<bool>,
9073+
pub codex_image_direct_generation_enabled: Option<bool>,
90359074
pub kiro_request_validation_enabled: Option<bool>,
90369075
pub kiro_cache_estimation_enabled: Option<bool>,
90379076
pub kiro_zero_cache_debug_enabled: Option<bool>,
@@ -9068,6 +9107,9 @@ pub async fn patch_admin_llm_gateway_key(
90689107
request.request_min_start_interval_ms,
90699108
request.codex_fast_enabled,
90709109
request.codex_strict_session_rejection_enabled,
9110+
request.codex_image_generation_enabled,
9111+
request.codex_image_standalone_generation_enabled,
9112+
request.codex_image_direct_generation_enabled,
90719113
request.kiro_request_validation_enabled,
90729114
request.kiro_cache_estimation_enabled,
90739115
request.kiro_zero_cache_debug_enabled,
@@ -9179,6 +9221,24 @@ pub async fn patch_admin_llm_gateway_key(
91799221
serde_json::Value::Bool(enabled),
91809222
);
91819223
}
9224+
if let Some(enabled) = request.codex_image_generation_enabled {
9225+
body.insert(
9226+
"codex_image_generation_enabled".to_string(),
9227+
serde_json::Value::Bool(enabled),
9228+
);
9229+
}
9230+
if let Some(enabled) = request.codex_image_standalone_generation_enabled {
9231+
body.insert(
9232+
"codex_image_standalone_generation_enabled".to_string(),
9233+
serde_json::Value::Bool(enabled),
9234+
);
9235+
}
9236+
if let Some(enabled) = request.codex_image_direct_generation_enabled {
9237+
body.insert(
9238+
"codex_image_direct_generation_enabled".to_string(),
9239+
serde_json::Value::Bool(enabled),
9240+
);
9241+
}
91829242
if let Some(kiro_request_validation_enabled) = request.kiro_request_validation_enabled {
91839243
body.insert(
91849244
"kiro_request_validation_enabled".to_string(),
@@ -9855,6 +9915,10 @@ pub struct AccountSummaryView {
98559915
pub auto_refresh_enabled: bool,
98569916
pub request_max_concurrency: Option<u64>,
98579917
pub request_min_start_interval_ms: Option<u64>,
9918+
#[serde(default)]
9919+
pub codex_image_generation_enabled: bool,
9920+
#[serde(default = "default_codex_image_generation_max_concurrency")]
9921+
pub codex_image_generation_max_concurrency: u64,
98589922
pub proxy_mode: String,
98599923
pub proxy_config_id: Option<String>,
98609924
pub effective_proxy_source: String,
@@ -9883,6 +9947,9 @@ impl Default for AccountSummaryView {
98839947
auto_refresh_enabled: true,
98849948
request_max_concurrency: None,
98859949
request_min_start_interval_ms: None,
9950+
codex_image_generation_enabled: false,
9951+
codex_image_generation_max_concurrency: default_codex_image_generation_max_concurrency(
9952+
),
98869953
proxy_mode: "inherit".to_string(),
98879954
proxy_config_id: None,
98889955
effective_proxy_source: "binding".to_string(),
@@ -10248,6 +10315,9 @@ pub async fn import_admin_llm_gateway_account(
1024810315
auto_refresh_enabled: true,
1024910316
request_max_concurrency: None,
1025010317
request_min_start_interval_ms: None,
10318+
codex_image_generation_enabled: false,
10319+
codex_image_generation_max_concurrency: default_codex_image_generation_max_concurrency(
10320+
),
1025110321
proxy_mode: "inherit".to_string(),
1025210322
proxy_config_id: None,
1025310323
effective_proxy_source: "binding".to_string(),
@@ -10347,6 +10417,8 @@ pub struct PatchAdminLlmGatewayAccountInput {
1034710417
pub proxy_config_id: Option<String>,
1034810418
pub request_max_concurrency: Option<u64>,
1034910419
pub request_min_start_interval_ms: Option<u64>,
10420+
pub codex_image_generation_enabled: Option<bool>,
10421+
pub codex_image_generation_max_concurrency: Option<u64>,
1035010422
pub request_max_concurrency_unlimited: bool,
1035110423
pub request_min_start_interval_ms_unlimited: bool,
1035210424
}
@@ -10373,6 +10445,10 @@ pub async fn patch_admin_llm_gateway_account(
1037310445
auto_refresh_enabled: input.auto_refresh_enabled.unwrap_or(true),
1037410446
request_max_concurrency: input.request_max_concurrency,
1037510447
request_min_start_interval_ms: input.request_min_start_interval_ms,
10448+
codex_image_generation_enabled: input.codex_image_generation_enabled.unwrap_or(false),
10449+
codex_image_generation_max_concurrency: input
10450+
.codex_image_generation_max_concurrency
10451+
.unwrap_or_else(default_codex_image_generation_max_concurrency),
1037610452
proxy_mode: input
1037710453
.proxy_mode
1037810454
.clone()
@@ -10430,6 +10506,9 @@ pub async fn refresh_admin_llm_gateway_account(name: &str) -> Result<AccountSumm
1043010506
auto_refresh_enabled: true,
1043110507
request_max_concurrency: None,
1043210508
request_min_start_interval_ms: None,
10509+
codex_image_generation_enabled: false,
10510+
codex_image_generation_max_concurrency: default_codex_image_generation_max_concurrency(
10511+
),
1043310512
proxy_mode: "inherit".to_string(),
1043410513
proxy_config_id: None,
1043510514
effective_proxy_source: "binding".to_string(),
@@ -10492,6 +10571,9 @@ pub async fn refresh_admin_llm_gateway_account_auth(
1049210571
auto_refresh_enabled: true,
1049310572
request_max_concurrency: None,
1049410573
request_min_start_interval_ms: None,
10574+
codex_image_generation_enabled: false,
10575+
codex_image_generation_max_concurrency: default_codex_image_generation_max_concurrency(
10576+
),
1049510577
proxy_mode: "inherit".to_string(),
1049610578
proxy_config_id: None,
1049710579
effective_proxy_source: "binding".to_string(),
@@ -10563,6 +10645,9 @@ pub async fn consume_admin_llm_gateway_account_rate_limit_reset_credit(
1056310645
auto_refresh_enabled: true,
1056410646
request_max_concurrency: None,
1056510647
request_min_start_interval_ms: None,
10648+
codex_image_generation_enabled: false,
10649+
codex_image_generation_max_concurrency:
10650+
default_codex_image_generation_max_concurrency(),
1056610651
proxy_mode: "inherit".to_string(),
1056710652
proxy_config_id: None,
1056810653
effective_proxy_source: "binding".to_string(),
@@ -11196,6 +11281,9 @@ pub async fn create_admin_kiro_key(
1119611281
usage_output_tokens: 0,
1119711282
usage_credit_total: 0.0,
1119811283
usage_credit_missing_events: 0,
11284+
codex_image_usage_tokens: 0,
11285+
codex_image_usage_missing_events: 0,
11286+
codex_image_last_used_at: None,
1119911287
remaining_billable: quota_billable_limit as i64,
1120011288
last_used_at: None,
1120111289
created_at: 0,
@@ -11225,6 +11313,9 @@ pub async fn create_admin_kiro_key(
1122511313
uses_global_kiro_billable_model_multipliers: true,
1122611314
codex_fast_enabled: true,
1122711315
codex_strict_session_rejection_enabled: false,
11316+
codex_image_generation_enabled: false,
11317+
codex_image_standalone_generation_enabled: false,
11318+
codex_image_direct_generation_enabled: false,
1122811319
kiro_candidate_credit_summary: None,
1122911320
})
1123011321
}
@@ -11960,6 +12051,20 @@ mod tests {
1196012051

1196112052
assert!(!key.kiro_full_request_logging_enabled);
1196212053
assert!(!key.kiro_remote_media_resolution_enabled);
12054+
assert!(key.codex_image_generation_enabled);
12055+
assert!(key.codex_image_standalone_generation_enabled);
12056+
assert!(!key.codex_image_direct_generation_enabled);
12057+
}
12058+
12059+
#[test]
12060+
fn admin_gateway_key_view_defaults_codex_image_usage_to_zero() {
12061+
let key: AdminLlmGatewayKeyView =
12062+
serde_json::from_str(r#"{"id":"k","name":"K","provider_type":"codex"}"#)
12063+
.expect("key should parse");
12064+
12065+
assert_eq!(key.codex_image_usage_tokens, 0);
12066+
assert_eq!(key.codex_image_usage_missing_events, 0);
12067+
assert_eq!(key.codex_image_last_used_at, None);
1196312068
}
1196412069

1196512070
#[test]
@@ -12084,6 +12189,8 @@ mod tests {
1208412189
.expect("account summary should parse");
1208512190

1208612191
assert_eq!(account.rate_limit_reset_credits_available, Some(2));
12192+
assert!(!account.codex_image_generation_enabled);
12193+
assert_eq!(account.codex_image_generation_max_concurrency, 3);
1208712194
}
1208812195

1208912196
#[test]

‎crates/frontend/src/pages/admin_kiro_gateway.rs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1899,6 +1899,9 @@ fn kiro_key_editor_card(props: &KiroKeyEditorCardProps) -> Html {
18991899
request_min_start_interval_ms: None,
19001900
codex_fast_enabled: None,
19011901
codex_strict_session_rejection_enabled: None,
1902+
codex_image_generation_enabled: None,
1903+
codex_image_standalone_generation_enabled: None,
1904+
codex_image_direct_generation_enabled: None,
19021905
kiro_request_validation_enabled: Some(kiro_request_validation_enabled_value),
19031906
kiro_cache_estimation_enabled: Some(kiro_cache_estimation_enabled_value),
19041907
kiro_zero_cache_debug_enabled: Some(kiro_zero_cache_debug_enabled_value),
@@ -2006,6 +2009,9 @@ fn kiro_key_editor_card(props: &KiroKeyEditorCardProps) -> Html {
20062009
request_min_start_interval_ms: None,
20072010
codex_fast_enabled: None,
20082011
codex_strict_session_rejection_enabled: None,
2012+
codex_image_generation_enabled: None,
2013+
codex_image_standalone_generation_enabled: None,
2014+
codex_image_direct_generation_enabled: None,
20092015
kiro_request_validation_enabled: None,
20102016
kiro_cache_estimation_enabled: Some(kiro_cache_estimation_enabled_value),
20112017
kiro_zero_cache_debug_enabled: Some(kiro_zero_cache_debug_enabled_value),

0 commit comments

Comments
 (0)