Skip to content

[Bug] Memory Contamination due to Scratchpad Reuse #28654

@mmoffitt

Description

@mmoffitt

Describe the issue

I believe there is an issue where ONNX Runtime is improperly reusing scratchpad memory between sessions without initializing values back to zeros. As a result, one observes different behavior depending on the order in which models are executed, and can even inspect the internal state of models which have previously run.

Version info:

% pip3 show onnxruntime          
Name: onnxruntime
Version: 1.26.0
Summary: ONNX Runtime is a runtime accelerator for Machine Learning models
Home-page: https://onnxruntime.ai
Author: Microsoft Corporation
Author-email: onnxruntime@microsoft.com
License: MIT License
Location: /Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/site-packages
Requires: flatbuffers, numpy, packaging, protobuf

To reproduce

Reproducing the bug
See the attached ort_bug.zip, which contains ort_bug.py + A.onnx + B.onnx

Observed behavior:
Executing model A before B:

% python3 ort_bug.py A.onnx B.onnx
A.onnx yields 1.5
B.onnx yields 1.5

Executing model B before A:

% python3 ort_bug.py B.onnx A.onnx
B.onnx yields 1.0
A.onnx yields 1.0

ort_bug.zip

Urgency

It is fairly urgent -- we are using ONNX Runtime to evaluate submissions to the 2026 NeuroGolf Championship, which concludes in roughly one month.

Platform

Mac

OS Version

Version 26.5 (25F71)

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.26.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions