fix(core): strip thoughts from scrubbed history turns and resolve thought leakage#27971
fix(core): strip thoughts from scrubbed history turns and resolve thought leakage#27971amelidev wants to merge 15 commits into
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a critical issue where the Gemini model's internal reasoning thoughts were leaking into the plain-text conversation history. By introducing a preprocessing step to strip these thoughts and subsequently coalescing the remaining turns, the changes ensure that the model history remains clean and compliant with API requirements, preventing the model from confusing its own scratchpad with user-facing dialogue. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
📊 PR Size: size/L
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces changes to filter out internal model thoughts from the chat history during history hardening and scrubbing, preventing thought leakage. It adds a stripThoughts helper function in historyHardening.ts and updates scrubContents to exclude thought parts, accompanied by comprehensive unit tests. The reviewer noted an inconsistency in how thoughts are detected between stripThoughts (strict boolean check) and scrubContents (truthy check), suggesting a unified truthy check to ensure robust thought detection.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request removes peer dependency flags from package-lock.json and implements logic to strip internal model thoughts from the conversation history in packages/core. This includes adding a stripThoughts step in hardenHistory, filtering out thought parts in scrubContents, and adding corresponding unit tests. The review feedback suggests adding defensive checks to ensure that parts are valid objects before using the 'in' operator to prevent runtime TypeErrors, and using the nullish coalescing operator (??) instead of logical OR (||) when providing default values for optional properties.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request removes "peer": true from several dependencies in package-lock.json and implements a mechanism to strip internal model thoughts (where part.thought === true) from conversational history in packages/core. This includes adding a stripThoughts pass in hardenHistory and filtering out thought parts in scrubContents, along with corresponding unit tests. The feedback suggests extracting the inline thought check into a reusable type guard isThoughtPart with a local interface and a detailed comment, in accordance with repository guidelines for using undocumented SDK properties, and using this helper consistently across the codebase.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request removes peer dependency flags from several packages in package-lock.json and implements a mechanism to strip internal model thoughts (where part.thought === true) from chat history in packages/core. Specifically, a new stripThoughts helper has been introduced in historyHardening.ts to filter out thought parts and discard any resulting empty turns during history hardening and scrubbing. Corresponding unit tests have been added to verify this behavior and prevent thought leakage. There are no review comments to address, and I have no additional feedback to provide.
gemini-cli
Bot
added
the
status/need-issue
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request removes the "peer": true flag from various dependencies in package-lock.json and introduces a mechanism to strip internal thought parts (where part.thought === true) from chat history. Specifically, it adds a stripThoughts function in historyHardening.ts that filters out thought parts and removes empty turns, integrating it into hardenHistory and scrubContents. Additionally, new unit tests are added in geminiChat.test.ts and historyHardening.test.ts to verify this behavior. There are no review comments, and I have no feedback to provide.
|
Hi there! Thank you for your interest in contributing to Gemini CLI. To ensure we maintain high code quality and focus on our prioritized roadmap, we only guarantee review and consideration of pull requests for issues that are explicitly labeled as 'help wanted'. This PR will be closed in 7 days if it remains without that designation. We encourage you to find and contribute to existing 'help wanted' issues in our backlog! Thank you for your understanding. |
…and type safety - Cast synthetic model thought parts in geminiChat.test.ts using as unknown as Part for strict type safety and consistency. - Extract isInternalThought helper function with no type casting or ESLint suppression required. - Optimize stripThoughts using .some() to avoid unnecessary allocations on thought-free turns. - Remove redundant trailing filter in stripThoughts. - Consolidate multi-pass loop in scrubHistory into a single, high-performance pass that scrubs and coalesces adjacent turns immutably. - Ensure strict immutability in coalesce and scrubHistory by completely replacing the turn object instead of mutating in place.
|
Closing this pull request as it does not have an associated tracked issue linked, and there has been no progress since the nudge. |
|
Hello @rmedranollamas |
|
Thank you very much! |
|
Reopening this PR . |
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces changes to strip internal thought parts from the conversation history and handle empty turns in packages/core. It updates hardenHistory, scrubHistory, and scrubContents to filter out parts where thought is true, and includes corresponding unit tests. The reviewer pointed out that isInternalThought strictly checks for part.thought === true instead of truthiness, which could lead to thought leakage if the API returns truthy values other than true.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request removes peer dependency flags from package-lock.json and introduces history hardening logic to strip internal thought parts from conversation history. The core changes add a stripThoughts helper and update history scrubbing and coalescing functions, supported by new unit tests. The review feedback highlights a potential issue in scrubContents where stripping all thought parts from a Content object could result in an empty parts array, potentially triggering a 400 Bad Request from the Gemini API, and suggests filtering out empty content objects.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request removes peer dependency flags from several packages in package-lock.json and implements logic in packages/core to strip internal thought parts from chat history and contents. Specifically, a new stripThoughts pass is integrated into hardenHistory, and both scrubHistory and scrubContents are updated to filter out thought parts and remove empty turns. Unit tests have been added to verify these changes. The reviewer's feedback correctly identifies a TypeScript compilation issue in historyHardening.ts where part.thought is accessed on the Part type without casting it to ThoughtPart.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request implements the filtering of internal thought parts from chat history and contents to prevent thought leakage and ensure compatibility with strict APIs. It introduces a stripThoughts helper in historyHardening.ts to remove thought parts before coalescing, updates scrubHistory and scrubContents to exclude thought parts and clean up empty turns, and adds comprehensive unit tests to validate these changes. As there are no review comments, I have no feedback to provide.
5 participants
Summary
This PR resolves a Thought Leakage issue where the Gemini model's internal monologues/reasoning thoughts leak into plain-text history turns, confusing the model in subsequent turns and causing it to emulate scratchpad thoughts or enter infinite loop monologues.
Details
scrubContentsinsidepackages/core/src/utils/historyHardening.tsto actively filter out any parts wherepart.thoughtis truthy, before map-scrubbing the remaining fields. This ensures nested executions (e.g. subagents) do not leak raw thoughts as plain conversational text.scrubHistoryto runcoalesceon the scrubbed turns. This safely discards any turns made empty by thought stripping while maintaining the Gemini API's strict role-alternation invariant.stripThoughtsto check for truthyp.thoughtproperties rather than strictlyp.thought === true, matching how thoughts are represented.thought leakage in getHistoryTurns) inpackages/core/src/core/geminiChat.test.tsto reproduce and confirm the fix.Related Issues
Resolves the issue documented in
issue.md.How to Validate
Run the targeted and whole history hardening test suites using Vitest:
npx vitest run packages/core/src/core/geminiChat.test.ts -t "thought leakage in getHistoryTurns" npx vitest run packages/core/src/utils/historyHardening.test.tsAll tests pass perfectly in under 15 seconds.
Pre-Merge Checklist