Expand model package with authoring tools and schema versioning (#28989)

### Description

Builds out the standalone `model_package/` C library so a single library
covers the full lifecycle of an ONNX Runtime model package: inspection,
authoring, content-addressed shared assets, commit, prune, and
validation.
The library remains free of any dependency on ONNX Runtime itself.

**Public C API (`model_package/include/model_package.h`)**

- Lifecycle: `ModelPackage_Open` / `ModelPackage_New` /
`ModelPackage_Close`,
with `ModelPackageOpenOptions` controlling external-path access, symlink
  following, and strict unknown-field handling.
- Inspection: a POD `ModelPackageInfo` tree (`ModelPackage_Info`) plus
  by-name lookups for components, variants, and per-namespace
  `executor_info` entries, and round-trip JSON getters that preserve
  fields unknown to the current build.
- Path resolution: `ModelPackage_ResolveStringRef` implements the
package's
  resolution rules — relative paths anchored at a base directory,
  `sha256:<hex>[/sub/path]` for shared-asset content, and portable vs
  installed confinement (absolute paths and `..` segments only allowed
  under `layout: "installed"`).
- Shared assets: SHA-256 directory hashing,
`ModelPackage_AddSharedAsset`
  (with reproducible-build URI check and an optional `copy_in` staging
  mode), `ModelPackage_RemoveSharedAsset`, and
  `ModelPackage_ResolveAssetUri`. Assets under
  `<package_root>/shared_assets/` are auto-discovered at `Open`.
- Authoring: inline/external component setters, variant upsert/remove,
  per-namespace executor_info setters (inline and external), and
  package-level metadata / layout / `additional_metadata` setters.
  Mutations invalidate cached pointers in the mutated scope and its
  descendants.
- Commit / prune / validate: `ModelPackage_Commit` writes the in-memory
  model to disk either in place or to a fresh `dest_root` ("save as"),
  with `PRESERVE` or `DENSE` write modes; `ModelPackage_Prune` reclaims
  unreferenced files under `shared_assets/` and tracked orphan
  variant/component directories left behind by removals; and
  `ModelPackage_Validate` runs schema, path-reachability, asset-rehash,
  and unknown-field checks and returns a JSON report.
- Errors: opaque `ModelPackageStatus*` with a stable additive
  `ModelPackageErrorCode` enum (IO, schema, version, path confinement,
  asset missing, asset hash mismatch, not found, invalid arg, state).

**Internal layout**

The implementation is split into focused translation units:
`manifest_parser`, `model_package_impl`, `authoring`,
`commit_prune_validate`, `path_resolver`, `asset_hasher`, and an
in-tree `sha256`. Shared error plumbing lives in `status_impl.h`.

**ONNX Runtime integration (`onnxruntime/core/session/model_package/`)**

The ORT-side glue is wired onto the library through the C inspection
and path-resolution entry points. `model_package_context` now translates
the library's info tree into ORT-internal structs and parses the
`executor_info["ort"]` payload (`model_file`, `external_data`,
`session_options`, `provider_options`). When a variant declares
`external_data`, `CreateSessionForModelPackage` loads the model from a
memory mapping and passes the resolved folder to the session via
`kOrtSessionOptionsModelExternalInitializersFileFolderPath`, so external
initializers — including those backed by a shared asset — are picked up
at `Initialize` time.

The experimental `OrtModelPackageApi_*_SinceV28` C entries introduced in
#28990 are unchanged.

**Documentation and tests**

- `model_package/README.md` documents the on-disk layout, manifest and
  component schemas, shared-asset rules, path resolution, the authoring
  flow, and commit / prune / validate semantics.
- `onnxruntime/core/session/model_package/README.md` documents the ORT
  consumer-side glue: the `executor_info["ort"]` schema, the variant
  selection algorithm, the session-creation contract, and the registered
  experimental C entries.
- New library tests cover inspection, authoring, asset hashing, and
  commit (`model_package/tests/`). The ORT integration tests in
  `onnxruntime/test/autoep/test_model_package.cc` are reworked against
  the current C API surface.

### Motivation and Context

ORT needs a single library that owns the model package format end to
end — not just reading it, but producing it, validating it, and
maintaining it on disk with content-addressed shared assets.
Consolidating this behind one C API lets ORT, publisher tooling, and
third-party consumers share the same parser, path-resolution rules, and
on-disk invariants without each reimplementing them, and keeps the
library independent of the ORT session runtime.

---------

Co-authored-by: jambayk <jambayk@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: jambayk <jambay@github.com>

Author: 3 people

cmake/onnxruntime_session.cmake

@@ -13,10 +13,8 @@ file(GLOB onnxruntime_session_srcs CONFIGURE_DEPENDS
 
 # Standalone model package library (parsing/inspection with no ORT dependency).
 # Compiled as a static library and linked into onnxruntime_session.
-# NOTE: ORT intentionally uses the library's internal C++ types directly (model_package::ParsePackage,
-# model_package_internal.h) rather than going through its public C API (ModelPackage_*). This avoids
-# double-wrapping (ORT C API -> standalone C API -> C++ internals). The public C API exists for
-# external consumers (GenAI, FL) who link against the standalone library independently.
+# ORT uses the standalone library's public C API (model_package.h) and translates
+# the C handles into ORT-internal C++ types inside core/session/model_package/.
 set(MODEL_PACKAGE_LIB_DIR "${REPO_ROOT}/model_package")
 if(NOT onnxruntime_MINIMAL_BUILD)
   set(MODEL_PACKAGE_BUILD_SHARED OFF CACHE BOOL "" FORCE)
@@ -59,7 +57,7 @@ onnxruntime_add_include_to_target(onnxruntime_session onnxruntime_common onnxrun
 target_link_libraries(onnxruntime_session PRIVATE onnxruntime_lora)
 if(TARGET model_package)
   target_link_libraries(onnxruntime_session PRIVATE model_package)
-  target_include_directories(onnxruntime_session PRIVATE ${MODEL_PACKAGE_LIB_DIR}/include ${MODEL_PACKAGE_LIB_DIR}/src)
+  target_include_directories(onnxruntime_session PRIVATE ${MODEL_PACKAGE_LIB_DIR}/include)
 endif()
 if(onnxruntime_ENABLE_INSTRUMENT)
   target_compile_definitions(onnxruntime_session PUBLIC ONNXRUNTIME_ENABLE_INSTRUMENT)

include/onnxruntime/core/session/onnxruntime_experimental_c_api.inc

@@ -66,6 +66,7 @@ ORT_EXPERIMENTAL_API(28, OrtStatusPtr, OrtApi_ExperimentalApiTest, _Out_ int64_t
 //    - OrtModelPackageApi_ModelPackage_GetVariantCount
 //    - OrtModelPackageApi_ModelPackage_GetVariantNames
 //    - OrtModelPackageApi_ModelPackage_GetVariantEpName
+//    - OrtModelPackageApi_ModelPackage_ResolveStringRef
 // 4) Select a component and resolve variant:
 //    - OrtModelPackageApi_SelectComponent
 // 5) Query selected variant info (optional):
@@ -212,6 +213,33 @@ ORT_EXPERIMENTAL_API(28, OrtStatusPtr, OrtModelPackageApi_ModelPackage_GetVarian
                      _In_ const char* variant_name,
                      _Outptr_result_maybenull_ const char** out_ep)
 
+/** \brief Resolve a path reference declared inside the package against the model_package rules.
+ *
+ * Handles the path forms a package may use:
+ *   - "sha256:<hex>" or "sha256:<hex>/<tail>": a content-addressed shared asset. Resolves to
+ *     the asset's on-disk directory (honoring manifest shared_assets overrides), optionally
+ *     joined with the confined tail. Errors if the asset is not declared/discovered.
+ *   - any other value: a relative path resolved against `base_dir` (or the package root when
+ *     `base_dir` is NULL), with portable-layout confinement (no absolute paths, no "..").
+ *
+ * When `must_exist` is non-zero the resolved path must exist on disk. `out_path` is owned by
+ * `ctx` and remains valid until the next call to this function on the same context.
+ *
+ * \param[in] ctx The package context returned by OrtModelPackageApi_CreateModelPackageContext.
+ * \param[in] base_dir Base directory for relative inputs. May be NULL to use the package root.
+ * \param[in] input The path reference to resolve.
+ * \param[in] must_exist Non-zero to require that the resolved path exists on disk.
+ * \param[out] out_path Receives the resolved path string.
+ *
+ * \snippet{doc} snippets.dox OrtStatus Return Value
+ */
+ORT_EXPERIMENTAL_API(28, OrtStatusPtr, OrtModelPackageApi_ModelPackage_ResolveStringRef,
+                     _In_ const OrtModelPackageContext* ctx,
+                     _In_opt_ const char* base_dir,
+                     _In_ const char* input,
+                     _In_ int must_exist,
+                     _Outptr_ const char** out_path)
+
 /** \brief Select a component model and return an opaque component instance.
  *
  * The variant selection is also performed during this call based on the component

model_package/CMakeLists.txt

@@ -52,8 +52,13 @@ endif()
 # ─────────────────────────────────────────────────────────────────────────────
 
 set(MODEL_PACKAGE_SOURCES
-  src/api.cc
-  src/parser.cc
+  src/asset_hasher.cc
+  src/authoring.cc
+  src/commit_prune_validate.cc
+  src/manifest_parser.cc
+  src/model_package_impl.cc
+  src/path_resolver.cc
+  src/sha256.cc
 )
 
 if(MODEL_PACKAGE_BUILD_SHARED)
@@ -91,6 +96,45 @@ install(TARGETS model_package
   RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
 )
 
-install(FILES include/model_package_api.h
+install(FILES include/model_package_api.h include/model_package.h
   DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}
 )
+
+# ─────────────────────────────────────────────────────────────────────────────
+# Tests
+# ─────────────────────────────────────────────────────────────────────────────
+
+if(MODEL_PACKAGE_BUILD_TESTS)
+  enable_testing()
+  add_executable(test_inspection tests/test_inspection.cc)
+  target_link_libraries(test_inspection PRIVATE model_package nlohmann_json::nlohmann_json)
+  target_include_directories(test_inspection PRIVATE
+    ${CMAKE_CURRENT_SOURCE_DIR}/include
+    ${CMAKE_CURRENT_SOURCE_DIR}/src
+  )
+  add_test(NAME inspection COMMAND test_inspection)
+
+  add_executable(test_asset_hashing tests/test_asset_hashing.cc)
+  target_link_libraries(test_asset_hashing PRIVATE model_package nlohmann_json::nlohmann_json)
+  target_include_directories(test_asset_hashing PRIVATE
+    ${CMAKE_CURRENT_SOURCE_DIR}/include
+    ${CMAKE_CURRENT_SOURCE_DIR}/src
+  )
+  add_test(NAME asset_hashing COMMAND test_asset_hashing)
+
+  add_executable(test_authoring tests/test_authoring.cc)
+  target_link_libraries(test_authoring PRIVATE model_package nlohmann_json::nlohmann_json)
+  target_include_directories(test_authoring PRIVATE
+    ${CMAKE_CURRENT_SOURCE_DIR}/include
+    ${CMAKE_CURRENT_SOURCE_DIR}/src
+  )
+  add_test(NAME authoring COMMAND test_authoring)
+
+  add_executable(test_commit tests/test_commit.cc)
+  target_link_libraries(test_commit PRIVATE model_package nlohmann_json::nlohmann_json)
+  target_include_directories(test_commit PRIVATE
+    ${CMAKE_CURRENT_SOURCE_DIR}/include
+    ${CMAKE_CURRENT_SOURCE_DIR}/src
+  )
+  add_test(NAME commit COMMAND test_commit)
+endif()