Skip to content

Experimental_C_API.md

Latest commit

 

History

History
254 lines (192 loc) · 11.8 KB

Experimental_C_API.md

File metadata and controls

254 lines (192 loc) · 11.8 KB

Experimental C API Design

Problem Statement

The ORT C API (OrtApi struct) provides a stable binary interface with strict backward compatibility guarantees: functions are append-only, never reordered or removed, and versioned so that older clients work against newer libraries.

This stability comes at a cost: new public APIs cannot be iterated on. Once a function is added to the stable struct, its signature and slot are permanent. We need a mechanism to expose new APIs for experimental/preview usage before committing them to the stable surface.

Requirements:

  • Allow test usage of new public APIs before promotion to the stable API
  • No stability guarantee for experimental functions
  • Minimal impact on the stable API surface
  • Reasonable ergonomics for C and C++ consumers
  • Functions may persist across multiple releases if unchanged
  • Clean promotion path to the stable API

Approaches Considered

Approach A: Versioned Experimental Struct (Exact Match)

Add a stable API entry point that returns a const OrtExperimentalApi* struct, similar to the existing sub-API pattern (GetCompileApi, GetEpApi, etc.). The struct would require an exact version match—any change to the struct layout bumps the version, and the runtime only satisfies the exact version it was built with.

Pros:

  • Same ergonomics as the existing stable API (typed struct, IDE autocomplete)
  • Full type safety with no casts after the initial retrieval
  • Familiar pattern for existing ORT API consumers

Cons:

  • A struct version bump due to any function changing breaks all consumers, even those only using unchanged functions
  • Mimics the stable API pattern but intentionally breaks its contract—semantically confusing
  • Requires either per-version headers or a single "latest only" header
  • Doesn't naturally support functions that persist unchanged across releases
  • If a user doesn't know the runtime ORT version, they'd have to probe multiple versions

Approach B: Name-Based Function Pointer Lookup

Add a single stable API entry point that retrieves an experimental function pointer by name. A companion header provides typedefs, name constants, and (for C++) typed accessor helpers.

Pros:

  • Each function is independently addressable—unchanged functions keep resolving across releases
  • Signature changes use a new name (API version-introduced suffix guarantees uniqueness); old name can be removed independently
  • Minimal stable API cost (one slot)
  • The instability contract is semantically clear: "is this specific thing available?"
  • Promotion to stable is clean: move to OrtApi, optionally keep the name as a redirect
  • Adding/removing experimental functions doesn't affect unrelated consumers

Cons:

  • Requires one cast from the generic function pointer to the correct function pointer type
  • Less discoverable without the companion header
  • String-based lookup has minor runtime cost (irrelevant—done once at init)

Chosen Approach: Name-Based Lookup with Typed C++ Helpers

The name-based approach (B) is the better fit because:

  1. Individual function longevity: Experimental functions may persist unchanged for several releases before promotion. The struct approach breaks all consumers on any change; name-based lookup only affects the specific function that changed.

  2. Honest contract: A struct looks stable but isn't. A per-function lookup makes instability semantically obvious.

  3. Simpler maintenance: No struct layout to track, no version bumps to coordinate. Just add/remove entries from a registration table.

  4. Ergonomics are solvable: The C++ wrapper with macro-generated typed accessors provides essentially the same user experience as a struct, minus one initial cast.

Design Details

Stable API Addition

One function pointer slot added to OrtApi:

// Generic function pointer type used as an opaque handle.
// Cast to the correct function pointer type before calling.
typedef void (ORT_API_CALL* OrtExperimentalFnPtr)(void);

// Returns nullptr if the named function is not available in this build.
OrtExperimentalFnPtr(ORT_API_CALL* GetExperimentalFunction)(_In_ const char* name) NO_EXCEPTION;

Using a function pointer type (rather than void*) ensures that casting to the correct function pointer type and back is well-defined in both C and C++ per the standard. Consumers must cast the returned value to the exact typedef before calling—calling through any other type is undefined behavior. Including ORT_API_CALL in the typedef matches the calling convention of all ORT API functions, which avoids compiler warnings when casting between the generic and typed pointers.

Single Source of Truth: The .inc File

All experimental functions are declared in one X-macro include file. The first argument is the ORT API version in which the function was introduced. The macro mechanically constructs the lookup name as <Name>_ExpSinceV<API Version>, guaranteeing uniqueness by construction—no two entries can collide unless they share both the same version and the same base name, which is trivially avoided during review.

// onnxruntime_experimental_api.inc
//
// ORT_EXPERIMENTAL_FUNC(SinceVersion, Name, ReturnType, Params...)

ORT_EXPERIMENTAL_FUNC(22, OrtApi_SomeNewThing, OrtStatusPtr,
    _In_ const OrtSession* session, _Out_ int64_t* result)

ORT_EXPERIMENTAL_FUNC(22, OrtApi_AnotherThing, OrtStatusPtr,
    _In_ const OrtEnv* env, _In_ const char* name, _Out_ OrtValue** out)

Experimental Consumer Header (generated from .inc)

A single header serves both C and C++ experimental API consumers. The C section provides typedefs and name constants; the C++ section (guarded by #ifdef __cplusplus) adds typed inline accessors in the Ort::Experimental namespace.

// onnxruntime_experimental_api.h
#pragma once

// Declare any new, auxiliary opaque types required by the experimental APIs in this header too.

// --- C: function pointer typedefs and name constants ---
#define ORT_EXPERIMENTAL_FUNC(VER, NAME, RET, ...)                                                   \
  typedef RET(ORT_API_CALL* OrtExperimental_##NAME##_ExpSinceV##VER##_Fn)(__VA_ARGS__) NO_EXCEPTION; \
  static const char* const kOrtExperimental_##NAME##_ExpSinceV##VER = #NAME "_ExpSinceV" #VER;
#include "onnxruntime_experimental_api.inc"
#undef ORT_EXPERIMENTAL_FUNC

// Produces (for SinceVersion=22, Name=OrtApi_SomeNewThing):
//   typedef OrtStatusPtr(ORT_API_CALL* OrtExperimental_OrtApi_SomeNewThing_ExpSinceV22_Fn)(
//       ...) NO_EXCEPTION;
//   static const char* const kOrtExperimental_OrtApi_SomeNewThing_ExpSinceV22 =
//       "OrtApi_SomeNewThing_ExpSinceV22";

#ifdef __cplusplus
namespace Ort::Experimental {

// --- C++: typed inline accessors (reuses the C typedefs above) ---
#define ORT_EXPERIMENTAL_FUNC(VER, NAME, RET, ...)                                                  \
  inline OrtExperimental_##NAME##_ExpSinceV##VER##_Fn Get_##NAME##_ExpSinceV##VER##_Fn(             \
      const OrtApi* api) {                                                                          \
    return reinterpret_cast<OrtExperimental_##NAME##_ExpSinceV##VER##_Fn>(                          \
        api->GetExperimentalFunction(kOrtExperimental_##NAME##_ExpSinceV##VER));                    \
  }
#include "onnxruntime_experimental_api.inc"
#undef ORT_EXPERIMENTAL_FUNC

}  // namespace Ort::Experimental

// Produces (for SinceVersion=22, Name=OrtApi_SomeNewThing):
// namespace Ort::Experimental {
//   inline OrtExperimental_OrtApi_SomeNewThing_ExpSinceV22_Fn
//   Get_OrtApi_SomeNewThing_ExpSinceV22_Fn(const OrtApi* api) {
//     return reinterpret_cast<OrtExperimental_OrtApi_SomeNewThing_ExpSinceV22_Fn>(
//         api->GetExperimentalFunction(kOrtExperimental_OrtApi_SomeNewThing_ExpSinceV22));
//   }
// }
#endif  // __cplusplus

C usage:

OrtExperimental_OrtApi_SomeNewThing_ExpSinceV22_Fn fn =
    (OrtExperimental_OrtApi_SomeNewThing_ExpSinceV22_Fn)api->GetExperimentalFunction(
        kOrtExperimental_OrtApi_SomeNewThing_ExpSinceV22);
if (fn) {
  OrtStatusPtr status = fn(session, &result);
}

C++ usage:

if (auto* fn = Ort::Experimental::Get_OrtApi_SomeNewThing_ExpSinceV22_Fn(api)) {
  Ort::Status status(fn(session, &result));
}

Implementation Side (generated from .inc)

// experimental_api.cc

// Function implementations use the full constructed name.
ORT_API_STATUS_IMPL(OrtExperimentalApis::OrtApi_SomeNewThing_ExpSinceV22,
                    _In_ const OrtSession* session, _Out_ int64_t* result) {
  API_IMPL_BEGIN
  // ...
  API_IMPL_END
}

// Registration table (auto-generated from .inc)
struct ExperimentalEntry {
  std::string_view name;
  OrtExperimentalFnPtr fn;
};

static const ExperimentalEntry kExperimentalFunctions[] = {
#define ORT_EXPERIMENTAL_FUNC(VER, NAME, ...) \
  { #NAME "_ExpSinceV" #VER, reinterpret_cast<OrtExperimentalFnPtr>(&OrtExperimentalApis::NAME##_ExpSinceV##VER) },
#include "onnxruntime_experimental_api.inc"
#undef ORT_EXPERIMENTAL_FUNC
};

// Lookup implementation
ORT_API(OrtExperimentalFnPtr, OrtApis::GetExperimentalFunction, _In_ const char* name) {
  if (name == nullptr) return nullptr;
  std::string_view target(name);
  for (const auto& entry : kExperimentalFunctions) {
    if (entry.name == target) return entry.fn;
  }
  return nullptr;
}

Lifecycle Rules

  1. Adding an experimental function: Add one line to the .inc file with the current ORT API version, implement it.
  2. Removing a function: Delete the line from the .inc. No retirement tracking is needed—the versioned name is inherently unique and cannot be accidentally reused.
  3. Changing a signature: Add a new entry with the current ORT API version (producing a new unique name) and optionally delete the old entry. Both can coexist if the old signature is still supported.
  4. Promoting to stable: Add the function to the stable API struct (append-only, name drops the _ExpSinceV<ver> suffix). Delete the experimental entry. Optionally keep the experimental entry for a transitional period, resolving it as a redirect.

Naming Convention

Experimental function names follow the pattern <TargetStruct>_<Name>_ExpSinceV<API Version>. The API version is the ORT API version in which the function was first introduced. The target struct prefix indicates the intended promotion destination. This naming scheme guarantees uniqueness by construction—a signature change requires a new .inc entry at the current API version, which produces a distinct name.

Examples:

  • OrtApi_SomeNewThing_ExpSinceV22 — introduced in API v22, destined for OrtApi
  • OrtApi_SomeNewThing_ExpSinceV23 — signature changed in API v23, replaces the v22 entry
  • OrtEpApi_SomeNewEpThing_ExpSinceV22 — destined for OrtEpApi
  • OrtCompileApi_SomeNewCompileThing_ExpSinceV22 — destined for OrtCompileApi

At promotion, the stable struct member drops the _ExpSinceV<API Version> suffix (e.g., the stable slot is named SomeNewThing in OrtApi).

Names are flat strings matched exactly. No separate retirement tracking is needed because the version suffix makes accidental name reuse impossible.

Rejected: Enumeration Helper

A runtime enumeration API (GetExperimentalFunctionNames) was considered but rejected. Any consumer that wants to call an experimental function already needs the header for the typedef—and the header is the enumeration. A consumer without the header could discover names at runtime but couldn't safely call them (no type information). This would cost a stable API slot for no practical benefit.

Open Questions

  • Should the Python/C#/Java bindings expose experimental functions, or keep them C/C++ only initially?
    • We can start with C/C++ and prove it out first.
  • Should we document an "epoch" expectation (e.g., "experimental functions are expected to be promoted or removed within 2 releases")?
    • We can set a general expectation without enforcing it.