Skip to content

Commit c0bb9f2

Browse files
author
Gopalakrishnan Nallasamy
committed
Make EPContext data callback APIs experimental
Convert the six EPContext read/write callback C APIs from stable APIs to experimental APIs using the mechanism introduced in PR #28746: - OrtApi_SessionOptions_SetEpContextDataReadFunc - OrtCompileApi_ModelCompilationOptions_SetEpContextDataWriteFunc - OrtEpApi_SessionOptions_GetEpContextConfig - OrtEpApi_ReleaseEpContextConfig - OrtEpApi_EpContextConfig_GetEpContextDataReadFunc - OrtEpApi_EpContextConfig_GetEpContextDataWriteFunc Remove the C++ convenience wrappers and move the auxiliary types (OrtEpContextConfig, OrtReadNamedBufferFunc, OrtWriteNamedBufferFunc) into onnxruntime_experimental_c_api.h. Update the example plugin EP, sample helper utilities, and tests to use the generated experimental accessors.
1 parent a2704c7 commit c0bb9f2

23 files changed

Lines changed: 490 additions & 469 deletions

include/onnxruntime/core/session/onnxruntime_c_api.h

Lines changed: 0 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -606,63 +606,6 @@ typedef OrtStatus*(ORT_API_CALL* OrtWriteBufferFunc)(_In_ void* state,
606606
_In_ const void* buffer,
607607
_In_ size_t buffer_num_bytes);
608608

609-
/** \brief Function called to write named binary data.
610-
*
611-
* This callback is currently used for EPContext binary data, but its contract is intentionally generic so future APIs
612-
* can reuse it for other named data payloads. The callback is called synchronously by the component that receives it.
613-
* ORT does not own or retain buffer after the callback returns. ORT does not serialize invocations made by different
614-
* EP instances or worker threads.
615-
*
616-
* Each callback invocation represents one complete write operation for name. The callback signature does not
617-
* provide an offset, sequence number, or final-chunk marker, so the component invoking the callback must define any
618-
* chunked ordering and completion contract with the application. Current EPContext use should prefer a single callback
619-
* invocation per EPContext binary unless chunking semantics are documented by the EP.
620-
*
621-
* The application's implementation can process the data in any way (e.g., encrypt and store, upload to cloud storage,
622-
* or compress) before persisting it.
623-
*
624-
* \param[in] state Opaque pointer holding the user's state. ORT does not own or manage this pointer. The application
625-
* must keep it valid for the duration required by the API that accepted the callback and must provide
626-
* any synchronization required if it can be used concurrently.
627-
* \param[in] name The file name or logical data identifier as a null-terminated UTF-8 string.
628-
* \param[in] buffer The buffer containing data to write.
629-
* \param[in] buffer_num_bytes The size of the buffer in bytes.
630-
*
631-
* \return OrtStatus* Write status. Return nullptr on success.
632-
* Use CreateStatus to provide error info with ORT_FAIL as the error code.
633-
* ORT will release the OrtStatus* if not null.
634-
*/
635-
typedef OrtStatus*(ORT_API_CALL* OrtWriteNamedBufferFunc)(_In_ void* state,
636-
_In_ const char* name,
637-
_In_ const void* buffer,
638-
_In_ size_t buffer_num_bytes);
639-
640-
/** \brief Function called to read named binary data.
641-
*
642-
* This callback is currently used for EPContext binary data, but its contract is intentionally generic so future APIs
643-
* can reuse it for other named data payloads. The application reads, processes (e.g., decrypts, decompresses,
644-
* downloads), and returns the requested data. ORT provides an allocator so the application can allocate the output
645-
* buffer directly. The callback is called synchronously by the component that receives it. ORT does not serialize
646-
* invocations made by different EP instances or worker threads.
647-
*
648-
* \param[in] state Opaque pointer holding the user's state. ORT does not own or manage this pointer. The application
649-
* must keep it valid for the duration required by the API that accepted the callback and must provide
650-
* any synchronization required if it can be used concurrently.
651-
* \param[in] name The file name or logical data identifier to read as a null-terminated UTF-8 string.
652-
* \param[in] allocator ORT-provided allocator. The application must use this to allocate the output buffer.
653-
* \param[out] buffer Set by the implementation to the allocated buffer containing the output data.
654-
* \param[out] data_size Set by the implementation to the size of the output data in bytes.
655-
*
656-
* \return OrtStatus* Read status. Return nullptr on success.
657-
* Use CreateStatus to provide error info with ORT_FAIL as the error code.
658-
* ORT will release the OrtStatus* if not null.
659-
*/
660-
typedef OrtStatus*(ORT_API_CALL* OrtReadNamedBufferFunc)(_In_ void* state,
661-
_In_ const char* name,
662-
_In_ OrtAllocator* allocator,
663-
_Outptr_ void** buffer,
664-
_Out_ size_t* data_size);
665-
666609
/** \brief Function called by ORT to allow user to specify how an initializer should be saved, that is, either
667610
* written to an external file or stored within the model. ORT calls this function for every initializer when
668611
* generating a model.
@@ -7594,30 +7537,6 @@ struct OrtApi {
75947537
* \since Version 1.28.
75957538
*/
75967539
ORT_API_T(OrtExperimentalFnPtr, GetExperimentalFunction, _In_ const char* name);
7597-
7598-
/** \brief Registers a callback to provide EPContext binary data during session load.
7599-
*
7600-
* When loading a compiled model with external (non-embedded) EPContext binary data, an execution provider can
7601-
* retrieve this callback from OrtEpContextConfig and call it instead of reading the binary data from disk.
7602-
*
7603-
* Reading happens at session load, so this callback is configured on OrtSessionOptions. The corresponding write
7604-
* callback runs only at compile time and is configured on OrtModelCompilationOptions via
7605-
* OrtCompileApi::ModelCompilationOptions_SetEpContextDataWriteFunc.
7606-
*
7607-
* The state pointer is stored as-is and is not owned by ORT. It must remain valid while any session or EP created
7608-
* from these options may call the callback. If the same state may be used by multiple EPs or threads, the application
7609-
* is responsible for synchronization.
7610-
*
7611-
* \param[in] options The OrtSessionOptions instance.
7612-
* \param[in] read_func The OrtReadNamedBufferFunc callback.
7613-
* \param[in] state Opaque state passed to read_func. Can be NULL.
7614-
*
7615-
* \snippet{doc} snippets.dox OrtStatus Return Value
7616-
*
7617-
* \since Version 1.28.
7618-
*/
7619-
ORT_API2_STATUS(SessionOptions_SetEpContextDataReadFunc, _Inout_ OrtSessionOptions* options,
7620-
_In_ OrtReadNamedBufferFunc read_func, _In_opt_ void* state);
76217540
};
76227541

76237542
/*
@@ -8460,31 +8379,6 @@ struct OrtCompileApi {
84608379
ORT_API2_STATUS(ModelCompilationOptions_SetInputModel,
84618380
_In_ OrtModelCompilationOptions* model_compile_options,
84628381
_In_ const OrtModel* model);
8463-
8464-
/** \brief Sets a callback for writing EPContext binary data during compilation.
8465-
*
8466-
* When EPContext embed mode is disabled, execution providers can retrieve this callback from OrtEpContextConfig and
8467-
* call it instead of writing EPContext binary data directly to disk.
8468-
*
8469-
* Writing happens only at compile time, so this callback is configured on OrtModelCompilationOptions. The
8470-
* corresponding read callback runs at session load and is configured on OrtSessionOptions via
8471-
* OrtApi::SessionOptions_SetEpContextDataReadFunc.
8472-
*
8473-
* The state pointer is stored as-is and is not owned by ORT. It must remain valid for the duration of the compile
8474-
* operation that may call the callback. If the same state may be used by multiple EPs or threads, the application is
8475-
* responsible for synchronization.
8476-
*
8477-
* \param[in] model_compile_options The OrtModelCompilationOptions instance.
8478-
* \param[in] write_func The OrtWriteNamedBufferFunc callback used to write EPContext bytes.
8479-
* \param[in] state Opaque state passed to write_func. Can be NULL.
8480-
*
8481-
* \snippet{doc} snippets.dox OrtStatus Return Value
8482-
*
8483-
* \since Version 1.28.
8484-
*/
8485-
ORT_API2_STATUS(ModelCompilationOptions_SetEpContextDataWriteFunc,
8486-
_In_ OrtModelCompilationOptions* model_compile_options,
8487-
_In_ OrtWriteNamedBufferFunc write_func, _In_opt_ void* state);
84888382
};
84898383

84908384
/**

include/onnxruntime/core/session/onnxruntime_cxx_api.h

Lines changed: 0 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -672,7 +672,6 @@ ORT_DEFINE_RELEASE(Value);
672672
ORT_DEFINE_RELEASE(ValueInfo);
673673

674674
ORT_DEFINE_RELEASE_FROM_API_STRUCT(ModelCompilationOptions, GetCompileApi);
675-
ORT_DEFINE_RELEASE_FROM_API_STRUCT(EpContextConfig, GetEpApi);
676675
ORT_DEFINE_RELEASE_FROM_API_STRUCT(EpDevice, GetEpApi);
677676
ORT_DEFINE_RELEASE_FROM_API_STRUCT(KernelDef, GetEpApi);
678677
ORT_DEFINE_RELEASE_FROM_API_STRUCT(KernelDefBuilder, GetEpApi);
@@ -804,7 +803,6 @@ struct AllocatedFree {
804803

805804
struct AllocatorWithDefaultOptions;
806805
struct Env;
807-
struct EpContextConfig;
808806
struct EpDevice;
809807
struct ExternalInitializerInfo;
810808
struct Graph;
@@ -1207,21 +1205,6 @@ struct EpDevice : detail::EpDeviceImpl<OrtEpDevice> {
12071205
ConstKeyValuePairs ep_metadata = {}, ConstKeyValuePairs ep_options = {});
12081206
};
12091207

1210-
/** \brief Owning wrapper around ::OrtEpContextConfig. */
1211-
struct EpContextConfig : detail::Base<OrtEpContextConfig> {
1212-
explicit EpContextConfig(std::nullptr_t) {} ///< No instance is created
1213-
explicit EpContextConfig(OrtEpContextConfig* p) : Base<OrtEpContextConfig>{p} {} ///< Take ownership
1214-
1215-
/// \brief Wraps OrtEpApi::SessionOptions_GetEpContextConfig
1216-
explicit EpContextConfig(const OrtSessionOptions* session_options);
1217-
1218-
/// \brief Wraps OrtEpApi::EpContextConfig_GetEpContextDataReadFunc
1219-
std::pair<OrtReadNamedBufferFunc, void*> GetEpContextDataReadFunc() const;
1220-
1221-
/// \brief Wraps OrtEpApi::EpContextConfig_GetEpContextDataWriteFunc
1222-
std::pair<OrtWriteNamedBufferFunc, void*> GetEpContextDataWriteFunc() const;
1223-
};
1224-
12251208
/** \brief Validate a compiled model's compatibility for one or more EP devices.
12261209
*
12271210
* Throws on error. Returns the resulting compatibility status.
@@ -1705,8 +1688,6 @@ struct SessionOptionsImpl : ConstSessionOptionsImpl<T> {
17051688
const std::vector<char*>& external_initializer_file_buffer_array,
17061689
const std::vector<size_t>& external_initializer_file_lengths); ///< Wraps OrtApi::AddExternalInitializersFromFilesInMemory
17071690

1708-
SessionOptionsImpl& SetEpContextDataReadFunc(OrtReadNamedBufferFunc read_func, void* state); ///< Wraps OrtApi::SessionOptions_SetEpContextDataReadFunc
1709-
17101691
SessionOptionsImpl& AppendExecutionProvider_CPU(int use_arena); ///< Wraps OrtApi::SessionOptionsAppendExecutionProvider_CPU
17111692
SessionOptionsImpl& AppendExecutionProvider_CUDA(const OrtCUDAProviderOptions& provider_options); ///< Wraps OrtApi::SessionOptionsAppendExecutionProvider_CUDA
17121693
SessionOptionsImpl& AppendExecutionProvider_CUDA_V2(const OrtCUDAProviderOptionsV2& provider_options); ///< Wraps OrtApi::SessionOptionsAppendExecutionProvider_CUDA_V2
@@ -1808,9 +1789,6 @@ struct ModelCompilationOptions : detail::Base<OrtModelCompilationOptions> {
18081789
///< Wraps OrtApi::ModelCompilationOptions_SetOutputModelWriteFunc
18091790
ModelCompilationOptions& SetOutputModelWriteFunc(OrtWriteBufferFunc write_func, void* state);
18101791

1811-
///< Wraps OrtCompileApi::ModelCompilationOptions_SetEpContextDataWriteFunc
1812-
ModelCompilationOptions& SetEpContextDataWriteFunc(OrtWriteNamedBufferFunc write_func, void* state);
1813-
18141792
ModelCompilationOptions& SetEpContextBinaryInformation(const ORTCHAR_T* output_directory,
18151793
const ORTCHAR_T* model_name); ///< Wraps OrtApi::ModelCompilationOptions_SetEpContextBinaryInformation
18161794
ModelCompilationOptions& SetFlags(uint32_t flags); ///< Wraps OrtApi::ModelCompilationOptions_SetFlags

include/onnxruntime/core/session/onnxruntime_cxx_inline.h

Lines changed: 0 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -769,24 +769,6 @@ inline EpDevice::EpDevice(OrtEpFactory& ep_factory, ConstHardwareDevice& hardwar
769769
ThrowOnError(GetEpApi().CreateEpDevice(&ep_factory, hardware_device, ep_metadata, ep_options, &p_));
770770
}
771771

772-
inline EpContextConfig::EpContextConfig(const OrtSessionOptions* session_options) {
773-
ThrowOnError(GetEpApi().SessionOptions_GetEpContextConfig(session_options, &p_));
774-
}
775-
776-
inline std::pair<OrtReadNamedBufferFunc, void*> EpContextConfig::GetEpContextDataReadFunc() const {
777-
OrtReadNamedBufferFunc read_func = nullptr;
778-
void* state = nullptr;
779-
ThrowOnError(GetEpApi().EpContextConfig_GetEpContextDataReadFunc(this->p_, &read_func, &state));
780-
return {read_func, state};
781-
}
782-
783-
inline std::pair<OrtWriteNamedBufferFunc, void*> EpContextConfig::GetEpContextDataWriteFunc() const {
784-
OrtWriteNamedBufferFunc write_func = nullptr;
785-
void* state = nullptr;
786-
ThrowOnError(GetEpApi().EpContextConfig_GetEpContextDataWriteFunc(this->p_, &write_func, &state));
787-
return {write_func, state};
788-
}
789-
790772
namespace detail {
791773
template <typename T>
792774
inline std::string EpAssignedSubgraphImpl<T>::GetEpName() const {
@@ -1353,12 +1335,6 @@ inline ModelCompilationOptions& ModelCompilationOptions::SetOutputModelWriteFunc
13531335
return *this;
13541336
}
13551337

1356-
inline ModelCompilationOptions& ModelCompilationOptions::SetEpContextDataWriteFunc(
1357-
OrtWriteNamedBufferFunc write_func, void* state) {
1358-
Ort::ThrowOnError(GetCompileApi().ModelCompilationOptions_SetEpContextDataWriteFunc(this->p_, write_func, state));
1359-
return *this;
1360-
}
1361-
13621338
inline ModelCompilationOptions& ModelCompilationOptions::SetEpContextEmbedMode(
13631339
bool embed_ep_context_in_model) {
13641340
Ort::ThrowOnError(GetCompileApi().ModelCompilationOptions_SetEpContextEmbedMode(
@@ -1702,13 +1678,6 @@ inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AddExternalInitializersFrom
17021678
return *this;
17031679
}
17041680

1705-
template <typename T>
1706-
inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::SetEpContextDataReadFunc(OrtReadNamedBufferFunc read_func,
1707-
void* state) {
1708-
ThrowOnError(GetApi().SessionOptions_SetEpContextDataReadFunc(this->p_, read_func, state));
1709-
return *this;
1710-
}
1711-
17121681
template <typename T>
17131682
inline SessionOptionsImpl<T>& SessionOptionsImpl<T>::AppendExecutionProvider_CPU(int use_arena) {
17141683
ThrowOnError(OrtSessionOptionsAppendExecutionProvider_CPU(this->p_, use_arena));

include/onnxruntime/core/session/onnxruntime_ep_c_api.h

Lines changed: 0 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@ extern "C" {
1818
* @{
1919
*/
2020
ORT_RUNTIME_CLASS(Ep);
21-
ORT_RUNTIME_CLASS(EpContextConfig);
2221
ORT_RUNTIME_CLASS(EpFactory);
2322
ORT_RUNTIME_CLASS(EpGraphSupportInfo);
2423
ORT_RUNTIME_CLASS(MemoryDevice); // opaque class to wrap onnxruntime::OrtDevice
@@ -2078,75 +2077,6 @@ struct OrtEpApi {
20782077
ORT_API2_STATUS(ProfilingEventsContainer_AddEvents, _In_ OrtProfilingEventsContainer* events_container,
20792078
_In_reads_(num_events) const OrtProfilingEvent* const* events,
20802079
_In_ size_t num_events);
2081-
2082-
/** \brief Get the EPContext configuration from session options.
2083-
*
2084-
* Extracts EPContext-related data I/O callbacks from the session options into an opaque OrtEpContextConfig handle.
2085-
* The EP should call this during CreateEp() while session_options is still valid, and store the returned handle for
2086-
* use during Compile(). The returned config is always non-NULL and must be released with ReleaseEpContextConfig.
2087-
*
2088-
* The returned handle owns only ORT's copy of callback function pointers and opaque state pointer values. It does not
2089-
* own the application-provided state. The application is responsible for keeping callback state valid and
2090-
* synchronized while an EP may call callbacks retrieved from this config.
2091-
*
2092-
* \param[in] session_options The OrtSessionOptions instance.
2093-
* \param[out] config The extracted OrtEpContextConfig.
2094-
*
2095-
* \snippet{doc} snippets.dox OrtStatus Return Value
2096-
*
2097-
* \since Version 1.28.
2098-
*/
2099-
ORT_API2_STATUS(SessionOptions_GetEpContextConfig,
2100-
_In_ const OrtSessionOptions* session_options,
2101-
_Outptr_ OrtEpContextConfig** config);
2102-
2103-
/** \brief Release an OrtEpContextConfig instance.
2104-
*
2105-
* \param[in] input The OrtEpContextConfig instance to release. May be NULL.
2106-
*
2107-
* \since Version 1.28.
2108-
*/
2109-
ORT_CLASS_RELEASE(EpContextConfig);
2110-
2111-
/** \brief Get the application-provided EPContext data read callback.
2112-
*
2113-
* Returns the OrtReadNamedBufferFunc and opaque state pointer registered via
2114-
* OrtApi::SessionOptions_SetEpContextDataReadFunc. If no callback was registered, *read_func and *state are set to
2115-
* NULL. The EP is responsible for calling the callback when present and for using its own normal read path when no
2116-
* callback is present.
2117-
*
2118-
* \param[in] config The OrtEpContextConfig from SessionOptions_GetEpContextConfig.
2119-
* \param[out] read_func The registered read callback, or NULL if none was registered.
2120-
* \param[out] state Opaque state pointer passed to read_func, or NULL if none was registered.
2121-
*
2122-
* \snippet{doc} snippets.dox OrtStatus Return Value
2123-
*
2124-
* \since Version 1.28.
2125-
*/
2126-
ORT_API2_STATUS(EpContextConfig_GetEpContextDataReadFunc,
2127-
_In_ const OrtEpContextConfig* config,
2128-
_Out_ OrtReadNamedBufferFunc* read_func,
2129-
_Out_ void** state);
2130-
2131-
/** \brief Get the application-provided EPContext data write callback.
2132-
*
2133-
* Returns the OrtWriteNamedBufferFunc and opaque state pointer registered via
2134-
* OrtCompileApi::ModelCompilationOptions_SetEpContextDataWriteFunc. If no callback was registered, *write_func and
2135-
* *state are set to NULL. The EP is responsible for calling the callback when present and for using its own normal
2136-
* write path when no callback is present.
2137-
*
2138-
* \param[in] config The OrtEpContextConfig from SessionOptions_GetEpContextConfig.
2139-
* \param[out] write_func The registered write callback, or NULL if none was registered.
2140-
* \param[out] state Opaque state pointer passed to write_func, or NULL if none was registered.
2141-
*
2142-
* \snippet{doc} snippets.dox OrtStatus Return Value
2143-
*
2144-
* \since Version 1.28.
2145-
*/
2146-
ORT_API2_STATUS(EpContextConfig_GetEpContextDataWriteFunc,
2147-
_In_ const OrtEpContextConfig* config,
2148-
_Out_ OrtWriteNamedBufferFunc* write_func,
2149-
_Out_ void** state);
21502080
};
21512081

21522082
/**

0 commit comments

Comments
 (0)