Skip to content

Commit 4e09045

Browse files
authored
✨ Add Google temporary chat mode with metadata fallback (#135)
* Add Google temporary chat mode with metadata fallback * Refine chat mode config and fallback detection * added break like suggested by copilot * Add summary compaction for temporary and fallback flows * added file option for context * simplified some logic
1 parent c64dee6 commit 4e09045

4 files changed

Lines changed: 293 additions & 23 deletions

File tree

README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,30 @@ To use Gemini-FastAPI, you need to extract your Gemini session cookies:
211211

212212
Each client entry can be configured with a different proxy to work around rate limits. Omit the `proxy` field or set it to `null` or an empty string to keep a direct connection.
213213

214+
### Chat Session Mode
215+
216+
You can control whether requests use normal Google chats or Google's temporary chat mode:
217+
218+
```yaml
219+
gemini:
220+
chat_mode: "normal" # "normal" (reuse metadata) or "temporary" (Google temporary chat, not saved to account)
221+
max_chars_per_request: 1000000
222+
oversized_context_strategy: "compaction" # "compaction" or "file"
223+
```
224+
225+
When `chat_mode` is set to `temporary`, the server applies an internal effective input limit of 90% of `max_chars_per_request`.
226+
When context exceeds the effective budget, handling is controlled by `oversized_context_strategy`:
227+
- `compaction`: summarize older turns and keep recent turns verbatim.
228+
- `file`: attach oversized context as `message.txt` and process it from file.
229+
230+
Environment variable equivalents:
231+
232+
```bash
233+
export CONFIG_GEMINI__CHAT_MODE="temporary"
234+
export CONFIG_GEMINI__MAX_CHARS_PER_REQUEST=1000000
235+
export CONFIG_GEMINI__OVERSIZED_CONTEXT_STRATEGY="compaction"
236+
```
237+
214238
### Custom Models
215239

216240
You can define custom models in `config/config.yaml` or via environment variables.

0 commit comments

Comments
 (0)