Skip to content

Worker: memory management improvements#1366

Draft
josephjclark wants to merge 3 commits intomainfrom
worker-memory
Draft

Worker: memory management improvements#1366
josephjclark wants to merge 3 commits intomainfrom
worker-memory

Conversation

@josephjclark
Copy link
Copy Markdown
Collaborator

Responding to a spate of OOMkills and lost runs: some experiments with memory management

Fixes #897

AI Usage

Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):

  • I have used Claude Code
  • I have used another model
  • I have not used AI

You can read more details in our
Responsible AI Policy

@github-project-automation github-project-automation bot moved this to New Issues in Core Apr 13, 2026
@josephjclark
Copy link
Copy Markdown
Collaborator Author

josephjclark commented Apr 14, 2026

I don't understand why this is failing in tests 🤔

Running around in circles. Even in code where the json streaming function isn't called it seems to fail.

I've also bumped into a possible issue with circular structures. It makes me nervous - shouldn't the JSON.stringify function throw if it hits a circular structure? That's something that could be hurting us right now in production

EDIT: Well we do seem to handle circular structures - the runtime must be dealing with that before events are emitted

{
  "x": {
    "x": "[Circular]"
  },
  "data": {}
}

@josephjclark
Copy link
Copy Markdown
Collaborator Author

managed to fix one of the failing tests but still got fails. Will have to look into this later. Claude unable to help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New Issues

Development

Successfully merging this pull request may close these issues.

Worker: Runs get lost if the dataclip is >150mb

2 participants