Skip to content

feat(context): add result offload plugin#974

Merged
lizradway merged 7 commits intostrands-agents:mainfrom
lizradway:resultoffload
May 4, 2026
Merged

feat(context): add result offload plugin#974
lizradway merged 7 commits intostrands-agents:mainfrom
lizradway:resultoffload

Conversation

@lizradway
Copy link
Copy Markdown
Member

@lizradway lizradway commented Apr 30, 2026

Description

TypeScript port of sdk-python#2162 and sdk-python#2222. Adds a ContextOffloader plugin that proactively intercepts oversized tool results via AfterToolCallEvent, stores them in an external storage backend, and replaces the in-context result with a truncated preview plus per-block storage references.

What it does:

  • Estimates token count of tool results using model.countTokens() (same approach as Python's count_tokens)
  • When a result exceeds maxResultTokens (default: 2,500), offloads each content block to storage
  • Replaces text/JSON blocks with a truncated preview and stored references
  • Replaces image/video/document blocks with metadata placeholders and stored references
  • Registers a retrieve_offloaded_content tool by default so the agent can fetch full content on demand (can be disabled via includeRetrievalTool: false)
  • Retrieval tool returns native content blocks (ImageBlock, DocumentBlock) for binary content
  • Storage references are actionable — FileStorage returns file paths, S3Storage returns s3:// URIs — so agents can use existing tools (file_read, bash) directly
  • Gracefully degrades — if storage fails, the original result is preserved

Includes:

  • Storage interface with InMemoryStorage, FileStorage, and S3Storage implementations (mirrors all three Python backends)
  • ContextOffloader plugin implementing the Plugin interface
  • 39 unit tests covering offloading behavior, storage backends, retrieval tool, constructor validation, and edge cases

Usage:

import { Agent } from '@strands-agents/sdk'
import { ContextOffloader, InMemoryStorage } from '@strands-agents/sdk/vended-plugins/context-offloader'

const agent = new Agent({
  model,
  plugins: [new ContextOffloader({ storage: new InMemoryStorage() })],
})

Related Issues

Documentation PR

strands-agents/docs#772 (Python docs — TS docs TBD)

Type of Change

New feature

Testing

How have you tested the change?

  • I ran npm run check
  • 39 new unit tests (plugin behavior, InMemoryStorage, FileStorage, retrieval tool)
  • All existing tests pass with no regressions
  • TypeScript build compiles cleanly

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@github-actions github-actions Bot added the strands-running <strands-managed> Whether or not an agent is currently running label Apr 30, 2026
Comment thread strands-ts/src/vended-plugins/context-offloader/storage.ts
Comment thread strands-ts/src/vended-plugins/context-offloader/index.ts
Comment thread strands-ts/src/vended-plugins/context-offloader/plugin.ts Outdated
Comment thread strands-ts/src/vended-plugins/context-offloader/storage.ts
Comment thread strands-ts/src/vended-plugins/context-offloader/plugin.ts
Comment thread strands-ts/src/vended-plugins/context-offloader/plugin.ts Outdated
Comment thread strands-ts/src/vended-plugins/context-offloader/storage.ts Outdated
Comment thread strands-ts/src/vended-plugins/context-offloader/storage.ts Outdated
@github-actions
Copy link
Copy Markdown
Contributor

Assessment: Comment

Well-structured plugin with solid error handling, good graceful degradation, and comprehensive test coverage. A few items to address before merging:

Review Categories
  • Bug: decodeStoredContent doesn't handle video/* content types, causing binary corruption when retrieving offloaded video content through the retrieval tool.
  • Type Safety: The as JSONValue cast in the retrieval tool callback hides the fact that ImageBlock/DocumentBlock (non-JSONValue types) are returned — worth considering the intended behavior.
  • Documentation: All exported types lack the required TSDoc documentation (interface properties, class docs, module doc). The skills plugin provides a good reference for the expected level.
  • Concurrency: FileStorage._saveMetadata is called on every store(), but _handleToolResult stores blocks via Promise.all — concurrent writes to the same metadata file may cause data loss.
  • API Review: This introduces a new public Storage interface and ContextOffloader plugin. Consider whether the needs-api-review label is appropriate given the moderate scope.

The plugin design is clean, the graceful degradation on storage failure is well thought out, and the test coverage is thorough across storage backends.

Comment thread strands-ts/src/vended-plugins/context-offloader/storage.ts Outdated
@github-actions github-actions Bot removed the strands-running <strands-managed> Whether or not an agent is currently running label Apr 30, 2026
@github-actions github-actions Bot added the strands-running <strands-managed> Whether or not an agent is currently running label Apr 30, 2026
Comment thread strands-ts/src/vended-plugins/context-offloader/plugin.ts Outdated
Comment thread strands-ts/src/vended-plugins/context-offloader/plugin.ts Outdated
@github-actions
Copy link
Copy Markdown
Contributor

Assessment: Approve

All previously raised issues have been addressed well — TSDoc added throughout, the video/* decoding gap fixed, S3Storage simplified, FileStorage metadata race condition resolved with the write chain, and _prefix made readonly.

Remaining minor items
  • Testing: Add a retrieval test for video/* content (since it was just added, worth verifying with a test)
  • Style: Inline import() type annotations for format types can be moved to top-level imports; logger message on line 193 should be lowercase per logging style guide

Solid work iterating on the feedback — the plugin is in good shape.

@github-actions github-actions Bot removed the strands-running <strands-managed> Whether or not an agent is currently running label Apr 30, 2026
@github-actions github-actions Bot added the strands-running <strands-managed> Whether or not an agent is currently running label Apr 30, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Assessment: Approve

All feedback from previous review rounds has been addressed. The code is clean, well-documented, and thoroughly tested. No remaining issues — ready to merge.

@github-actions github-actions Bot removed the strands-running <strands-managed> Whether or not an agent is currently running label Apr 30, 2026
@lizradway lizradway marked this pull request as ready for review April 30, 2026 19:24
@github-actions github-actions Bot added the strands-running <strands-managed> Whether or not an agent is currently running label Apr 30, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Assessment: Approve

No changes since last review. All previously identified issues have been resolved across three review rounds. The PR is ready to merge.

@github-actions github-actions Bot removed the strands-running <strands-managed> Whether or not an agent is currently running label Apr 30, 2026
opieter-aws
opieter-aws previously approved these changes Apr 30, 2026
@github-actions github-actions Bot added the strands-running <strands-managed> Whether or not an agent is currently running label May 4, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

Assessment: Approve

The new commit cleanly removes the LocalAgent model cast now that model is a required property on the interface. The code is simpler and fully type-safe — no issues.

One note: the PR description's "Known limitations" section still references the cast workaround and links to #938. Consider updating it since that limitation is now resolved.

@github-actions github-actions Bot removed the strands-running <strands-managed> Whether or not an agent is currently running label May 4, 2026
@lizradway lizradway added this pull request to the merge queue May 4, 2026
Merged via the queue into strands-agents:main with commit 48639b3 May 4, 2026
15 of 17 checks passed
@lizradway lizradway deleted the resultoffload branch May 4, 2026 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants