Skip to content

Refactor EPContext data callbacks for EP-owned I/O#28621

Open
GopalakrishnanN wants to merge 6 commits into
microsoft:mainfrom
GopalakrishnanN:gokrishnan/CryptoSupportEPOwnedIO
Open

Refactor EPContext data callbacks for EP-owned I/O#28621
GopalakrishnanN wants to merge 6 commits into
microsoft:mainfrom
GopalakrishnanN:gokrishnan/CryptoSupportEPOwnedIO

Conversation

@GopalakrishnanN
Copy link
Copy Markdown
Contributor

Summary:

  • replace OrtEpApi EPContext read/write helper APIs with callback getter APIs so EPs own file/mmap fallback behavior
  • add an Ort::EpContextConfig RAII wrapper and update the example plugin EP to use it
  • bump the new API surface to ORT API version 28 and update focused EPContext tests

Validation:

  • lintrunner -a on touched files: passed
  • built onnxruntime_test_all: passed
  • built onnxruntime_autoep_test: passed
  • onnxruntime_test_all.exe --gtest_filter=PluginExecutionProviderTest.EpContext*: 6 tests passed
  • onnxruntime_autoep_test.exe --gtest_filter=OrtEpLibrary.EpContext: 9 tests passed
  • onnxruntime_autoep_test.exe: 59 tests passed

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors EPContext data I/O so execution providers can retrieve user-provided read/write callbacks (and associated state) from an opaque OrtEpContextConfig, allowing EP-owned fallback behavior for file/mmap handling.

Changes:

  • Bumps ORT version/API surface to 1.28.0 / ORT_API_VERSION 28 and adds new C/C++ APIs for EPContext data callbacks.
  • Introduces OrtEpContextConfig in the EP API to extract and later retrieve EPContext read/write callbacks from session/compile options.
  • Updates the example plugin EP and focused EPContext tests to validate callback plumbing for embedded vs external EPContext data.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
VERSION_NUMBER Bump runtime version to 1.28.0.
onnxruntime/test/framework/ep_plugin_provider_test.cc Adds unit tests for EPContext config/callback accessors and argument validation.
onnxruntime/test/autoep/test_execution.cc Adds AutoEP coverage for external EPContext data using read/write callbacks.
onnxruntime/test/autoep/library/example_plugin_ep/ep.h Extends example EP config and plumbs Ort::EpContextConfig ownership into EP instance.
onnxruntime/test/autoep/library/example_plugin_ep/ep.cc Implements callback-or-file fallback logic for EPContext external data; writes external EPContext via callback when provided.
onnxruntime/test/autoep/library/example_plugin_ep/ep_factory.cc Extracts OrtEpContextConfig during EP creation and passes it to the EP.
onnxruntime/core/session/plugin_ep/ep_api.h Declares new EP API entrypoints for EPContext config + callback getters.
onnxruntime/core/session/plugin_ep/ep_api.cc Implements EPContext config handle and callback getter APIs; appends to OrtEpApi.
onnxruntime/core/session/ort_apis.h Adds declaration for SessionOptions_SetEpContextDataReadFunc in the core API surface.
onnxruntime/core/session/onnxruntime_c_api.cc Bumps internal OrtApi struct instance to ort_api_1_to_28 and appends new API function pointer.
onnxruntime/core/session/model_compilation_options.h Adds ModelCompilationOptions::SetEpContextDataWriteFunc.
onnxruntime/core/session/model_compilation_options.cc Stores EPContext write callback into model generation options.
onnxruntime/core/session/compile_api.h Adds compile API entrypoint for setting EPContext data write callback.
onnxruntime/core/session/compile_api.cc Implements compile API entrypoint and appends it to OrtCompileApi.
onnxruntime/core/session/abi_session_options.cc Implements SessionOptions_SetEpContextDataReadFunc storage.
onnxruntime/core/framework/session_options.h Adds storage for EPContext read callback + state in internal SessionOptions.
onnxruntime/core/framework/ep_context_options.h Adds EpContextDataWriteFuncHolder to model generation options.
onnxruntime/core/framework/ep_context_options.cc Adds accessor to retrieve EPContext write callback holder.
include/onnxruntime/core/session/onnxruntime_ep_c_api.h Public EP API additions: OrtEpContextConfig handle + callback getter APIs.
include/onnxruntime/core/session/onnxruntime_cxx_inline.h Adds inline C++ wrappers for Ort::EpContextConfig and new setter wrappers.
include/onnxruntime/core/session/onnxruntime_cxx_api.h Declares Ort::EpContextConfig RAII wrapper and new SessionOptions/ModelCompilationOptions methods.
include/onnxruntime/core/session/onnxruntime_c_api.h Bumps ORT_API_VERSION to 28 and adds callback typedefs + new C/Compile API functions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 4919 to 4924
&OrtApis::GetMemPatternEnabled,
&OrtApis::GetSessionExecutionMode,
// End of Version 27 - DO NOT MODIFY ABOVE (see above text for more information)

&OrtApis::SessionOptions_SetEpContextDataReadFunc,
};
Comment on lines 394 to 401
&OrtCompileAPI::ModelCompilationOptions_SetInputModel,
// End of Version 24 - DO NOT MODIFY ABOVE
// End of Version 25 - DO NOT MODIFY ABOVE
// End of Version 26 - DO NOT MODIFY ABOVE
// End of Version 27 - DO NOT MODIFY ABOVE

&OrtCompileAPI::ModelCompilationOptions_SetEpContextDataWriteFunc,
};
Comment on lines 1353 to 1361
// End of Version 25 - DO NOT MODIFY ABOVE
// End of Version 26 - DO NOT MODIFY ABOVE
// End of Version 27 - DO NOT MODIFY ABOVE

&OrtExecutionProviderApi::SessionOptions_GetEpContextConfig,
&OrtExecutionProviderApi::ReleaseEpContextConfig,
&OrtExecutionProviderApi::EpContextConfig_GetEpContextDataReadFunc,
&OrtExecutionProviderApi::EpContextConfig_GetEpContextDataWriteFunc,
};
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants