Refactor EPContext data callbacks for EP-owned I/O#28621
Open
GopalakrishnanN wants to merge 6 commits into
Open
Refactor EPContext data callbacks for EP-owned I/O#28621GopalakrishnanN wants to merge 6 commits into
GopalakrishnanN wants to merge 6 commits into
Conversation
added 6 commits
May 15, 2026 17:24
Contributor
There was a problem hiding this comment.
Pull request overview
Refactors EPContext data I/O so execution providers can retrieve user-provided read/write callbacks (and associated state) from an opaque OrtEpContextConfig, allowing EP-owned fallback behavior for file/mmap handling.
Changes:
- Bumps ORT version/API surface to 1.28.0 / ORT_API_VERSION 28 and adds new C/C++ APIs for EPContext data callbacks.
- Introduces
OrtEpContextConfigin the EP API to extract and later retrieve EPContext read/write callbacks from session/compile options. - Updates the example plugin EP and focused EPContext tests to validate callback plumbing for embedded vs external EPContext data.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| VERSION_NUMBER | Bump runtime version to 1.28.0. |
| onnxruntime/test/framework/ep_plugin_provider_test.cc | Adds unit tests for EPContext config/callback accessors and argument validation. |
| onnxruntime/test/autoep/test_execution.cc | Adds AutoEP coverage for external EPContext data using read/write callbacks. |
| onnxruntime/test/autoep/library/example_plugin_ep/ep.h | Extends example EP config and plumbs Ort::EpContextConfig ownership into EP instance. |
| onnxruntime/test/autoep/library/example_plugin_ep/ep.cc | Implements callback-or-file fallback logic for EPContext external data; writes external EPContext via callback when provided. |
| onnxruntime/test/autoep/library/example_plugin_ep/ep_factory.cc | Extracts OrtEpContextConfig during EP creation and passes it to the EP. |
| onnxruntime/core/session/plugin_ep/ep_api.h | Declares new EP API entrypoints for EPContext config + callback getters. |
| onnxruntime/core/session/plugin_ep/ep_api.cc | Implements EPContext config handle and callback getter APIs; appends to OrtEpApi. |
| onnxruntime/core/session/ort_apis.h | Adds declaration for SessionOptions_SetEpContextDataReadFunc in the core API surface. |
| onnxruntime/core/session/onnxruntime_c_api.cc | Bumps internal OrtApi struct instance to ort_api_1_to_28 and appends new API function pointer. |
| onnxruntime/core/session/model_compilation_options.h | Adds ModelCompilationOptions::SetEpContextDataWriteFunc. |
| onnxruntime/core/session/model_compilation_options.cc | Stores EPContext write callback into model generation options. |
| onnxruntime/core/session/compile_api.h | Adds compile API entrypoint for setting EPContext data write callback. |
| onnxruntime/core/session/compile_api.cc | Implements compile API entrypoint and appends it to OrtCompileApi. |
| onnxruntime/core/session/abi_session_options.cc | Implements SessionOptions_SetEpContextDataReadFunc storage. |
| onnxruntime/core/framework/session_options.h | Adds storage for EPContext read callback + state in internal SessionOptions. |
| onnxruntime/core/framework/ep_context_options.h | Adds EpContextDataWriteFuncHolder to model generation options. |
| onnxruntime/core/framework/ep_context_options.cc | Adds accessor to retrieve EPContext write callback holder. |
| include/onnxruntime/core/session/onnxruntime_ep_c_api.h | Public EP API additions: OrtEpContextConfig handle + callback getter APIs. |
| include/onnxruntime/core/session/onnxruntime_cxx_inline.h | Adds inline C++ wrappers for Ort::EpContextConfig and new setter wrappers. |
| include/onnxruntime/core/session/onnxruntime_cxx_api.h | Declares Ort::EpContextConfig RAII wrapper and new SessionOptions/ModelCompilationOptions methods. |
| include/onnxruntime/core/session/onnxruntime_c_api.h | Bumps ORT_API_VERSION to 28 and adds callback typedefs + new C/Compile API functions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
4919
to
4924
| &OrtApis::GetMemPatternEnabled, | ||
| &OrtApis::GetSessionExecutionMode, | ||
| // End of Version 27 - DO NOT MODIFY ABOVE (see above text for more information) | ||
|
|
||
| &OrtApis::SessionOptions_SetEpContextDataReadFunc, | ||
| }; |
Comment on lines
394
to
401
| &OrtCompileAPI::ModelCompilationOptions_SetInputModel, | ||
| // End of Version 24 - DO NOT MODIFY ABOVE | ||
| // End of Version 25 - DO NOT MODIFY ABOVE | ||
| // End of Version 26 - DO NOT MODIFY ABOVE | ||
| // End of Version 27 - DO NOT MODIFY ABOVE | ||
|
|
||
| &OrtCompileAPI::ModelCompilationOptions_SetEpContextDataWriteFunc, | ||
| }; |
Comment on lines
1353
to
1361
| // End of Version 25 - DO NOT MODIFY ABOVE | ||
| // End of Version 26 - DO NOT MODIFY ABOVE | ||
| // End of Version 27 - DO NOT MODIFY ABOVE | ||
|
|
||
| &OrtExecutionProviderApi::SessionOptions_GetEpContextConfig, | ||
| &OrtExecutionProviderApi::ReleaseEpContextConfig, | ||
| &OrtExecutionProviderApi::EpContextConfig_GetEpContextDataReadFunc, | ||
| &OrtExecutionProviderApi::EpContextConfig_GetEpContextDataWriteFunc, | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Validation: