`extra_body` parameter

[devm777]

Would be great to have an extra_body parameter like the OpenAI Python client does

[64bit]

This can be achieved with byot feature

For example:

#[derive(Serialize)]
struct CreateChatCompletionRequestExtra {
  // https://serde.rs/attr-flatten.html
  #[serde(flatten)] 
  pub request: CreateChatCompletionRequest, // original request type
  #[serde(flatten)]
  pub extra_body: serde_json::Value, // or this can be your custom type
}

let request = CreateChatCompletionRequestExtra { ... fields ... };

let response: CreateChatCompletionResponse = client.chat().create_byot(&request).await?;

[devm777]

Oh interesting! Didn't realize it was possible to extend types like that

extra_body would still be a useful addition to this library to match the behaviour of official OpenAI client SDKs. And this parameter used by many providers like Gemini API and Cerebras to pass provider-specific config

[bigoldcat123]

I think extra_body would also be valuable on the response side.

For example:

 pub struct ChatCompletionResponseMessage { 
    // existing fields...
      #[serde(flatten)]     pub extra: serde_json::Value
} 
pub pub struct CreateChatCompletionRequest {
    // existing fields...
      #[serde(flatten)]     pub extra: serde_json::Value
}

While BYOT can technically solve this, it has a different tradeoff. Once users switch to a fully custom response type, they often lose the convenience and type safety of the existing SDK models just to access 1-2 provider-specific fields.

Without built-in support, users end up doing things like:

if let Some(content) = response["choices"][0]["message"]["content"].as_str() {
     println!("{}", content); 
} 

which means giving up the strongly-typed response structure for the entire payload because of a few extra fields.

[64bit]

the extra_body doesnt have to be serde_json::Value it can be any type as long as trait requirements are met mostly Serialize.