v0.29.0

v0.29.0

🎉 Overview

v0.29.0 replaces the simple message-dropping truncation strategy with a new VLM-based SummarizingTruncationStrategy that summarizes older conversation history to preserve context while staying within token limits. It also fixes mouse scroll coordinate scaling issues, improves scroll tool descriptions with OS-specific guidance, removes get and locate from the default agent tools, hardens the move_mouse tool against malformed coordinate inputs, and makes base64 image truncation in html reports more robust.

✨ New Features

• SummarizingTruncationStrategy — new default truncation strategy that uses the VLM to summarize older conversation history instead of dropping messages, with prompt caching support during summarization for cost efficiency by @philipph-askui in #257
• SlidingImageWindowSummarizingTruncationStrategy (experimental) — extends summarization with dynamic image removal from older messages to reduce network traffic and latencies while staying compatible with prompt caching by @philipph-askui in #257
• truncation_strategy init parameter on ComputerAgent, AndroidAgent, and Agent — allows passing a custom truncation strategy with auto-injection of conversation dependencies (vlm_provider, reporter, callbacks) by @philipph-askui in #257

🔧 Improvements

• Mouse scroll tool description now includes OS-dependent scroll guidance (start with dy=150/dy=-150, macOS direction info) by @programminx-askui in #260
• truncate_content in reporting replaced by truncate_base64_images — only base64 image data is replaced with placeholders, leaving all other content (prompts, tool outputs) untouched by @philipph-askui in #259
• move_mouse tool now robustly parses coordinates when the agent passes them as strings or comma-separated values, with clearer tool description and improved error messages by @philipph-askui in #262

🐛 Bug Fixes

• Fix incorrect coordinate scaling on mouse scroll deltas — ComputerAgentOsFacade.mouse_scroll no longer applies display scaling to scroll amounts (SOLENG-332) by @programminx-askui in #260

⚠️ Breaking Changes

• SimpleTruncationStrategy and SimpleTruncationStrategyFactory removed — replaced by SummarizingTruncationStrategy as the new default
• Conversation constructor parameter truncation_strategy_factory replaced by truncation_strategy (a strategy instance instead of a factory)
• get and locate tools removed from Agent's default tool list — they are no longer auto-added when an agent_os is provided
• mouse_scroll parameters renamed from x/y to dx/dy across all AgentOs implementations (AskUiControllerClient, PlaywrightAgentOs, ComputerAgentOsFacade, ComputerAgent)
• truncate_content function in reporting.py removed — replaced by truncate_base64_images

Full Changelog: v0.28.0...v0.29.0