Skip to content

Commit 063244d

Browse files
authored
"Embedding" (simple) API layer (#1701)
## Summary Introduces a new **`Babylon::Embedding`** layer that gives host apps a small, opinionated `Runtime` + `View` API for embedding Babylon Native, and migrates every Playground host (Android, iOS, visionOS, macOS, UWP, Win32, X11) onto it. The goal is to make platform integration a job that takes tens — not hundreds — of lines of code, and to do that without leaking `AppRuntime`, `Graphics::Device`, plugin headers, polyfill plumbing, or threading rules into the host. Two layers ship together: 1. **Core C++ API** (`Embedding/Include/Shared/Babylon/Embedding/{Runtime,View,RuntimeOptions,LogLevel}.h`) — cross-platform, single source of truth for the lifecycle, threading, suspend/resume, input forwarding, and XR window management. 2. **Per-platform language facades** that wrap the core API in idiomatic types for the host language: - **Android**: JNI shared library (`BabylonNativeEmbedding.cpp`) + a host-authored Java facade (`BabylonNative.java`) - **Apple**: `BNRuntime` / `BNView` Obj‑C classes (Swift-friendly via the bridging header) ## What's new ### `Embedding` core (`Embedding/Source`, `Embedding/Include/Shared`) Two value types with explicit ownership and a documented contract: ```cpp Babylon::Embedding::RuntimeOptions options{}; options.enableDebugger = true; options.log = [](LogLevel level, std::string_view msg) { /* … */ }; Babylon::Embedding::Runtime runtime{options}; runtime.LoadScript("app:///Scripts/babylon.max.js"); runtime.LoadScript("app:///Scripts/scene.js"); Babylon::Embedding::View view{runtime, nativeWindowHandle}; view.Resize(width, height, CoordinateUnits::Physical); // driven from the host's draw callback: view.RenderFrame(); ``` What the layer takes care of for you: - **Runtime construction**: spins up `AppRuntime`, `JsRuntime`, polyfills (Console, Window, XMLHttpRequest, WebSocket, URL, Canvas, TextDecoder, Performance, Scheduling), and non‑GPU plugins. - **GPU deferral**: `Graphics::Device`, `NativeEngine`, `NativeInput`, `NativeCanvas`, `TestUtils`, and (optionally) `NativeXr` / `ShaderCache` are constructed lazily on the **first** `View::Resize`. Subsequent View attaches just rebind the Device. - **Script queuing**: `LoadScript` / `Eval` / `RunOnJsThread` are safe to call before the first `View` attach; queued and flushed when the engine comes up. - **Suspend / Resume**: reference-counted, closes the in-flight frame, pauses JS timers, optionally persists the shader cache. - **Input**: `OnPointer*` / `OnMouse*` accept either `Physical` or `Logical` pixel units; the View handles DPR conversion. - **XR window**: `Runtime::SetXrWindow` hands off a secondary surface for NativeXr. - **Logging**: single `RuntimeOptions::log` sink for `console.{log,warn,error}`, `Babylon::DebugTrace`, and uncaught JS exceptions. ### Android facade `Embedding/Android/` builds the JNI shared library `libBabylonNativeEmbedding.so` (replacing the old `BabylonNativeJNI.cpp`). It ships the **native side only** — each host authors its own Java declarations that match the JNI ABI. The Playground's facade (`Apps/Playground/Android/BabylonNative/…/com/babylonjs/embedding/BabylonNative.java`, replacing the old `Wrapper.java`) is a thin `static`-method mirror of the C++ API: - **Process lifecycle**: `setContext`, `setCurrentActivity`, `pause`, `resume`, `requestPermissionsResult` - **Runtime**: `runtimeCreate` (optionally with a nested `RuntimeOptions`), `runtimeDestroy`, `runtimeLoadScript`, `runtimeEval`, `runtimeSetXrSurface`, `runtimeIsXrActive` - **View**: `viewAttach`, `viewDetach`, `viewRenderFrame`, `viewResize`, `viewPointerDown` / `viewPointerMove` / `viewPointerUp` A separate `com.library.babylonnative.BabylonView` helper wires `SurfaceHolder.Callback` and touch events to those calls. ### Apple facade `Embedding/Apple/` — Obj‑C wrappers (`BNRuntime`, `BNView`, `BNViewDelegate`) usable from Obj‑C and from Swift via the bridging header. Used by Playground iOS, macOS, and visionOS. `BNRuntimeNative.h` exposes the underlying `Babylon::Embedding::Runtime*` for callers that need the C++ API. ## Host app migrations Every Playground host now runs on top of the Embedding layer: | Host | Before | After | |---|---|---| | Android | `BabylonNativeJNI.cpp` + `Wrapper.java` | `BabylonNative` Java facade over the JNI layer | | iOS | `LibNativeBridge.{h,mm}` + ad-hoc Swift glue | `BNRuntime` / `BNView`, ~75-line `ViewController.swift` | | macOS | `ViewController.mm` doing its own AppContext plumbing | Same `BNRuntime` / `BNView` types as iOS | | visionOS | `LibNativeBridge.{h,mm}` + custom App.swift threading | `BNRuntime` / `BNView` + bridging header | | Win32 | `AppContext` + manual `UpdateSize` / mouse plumbing | `Runtime` + `View` with `WM_POINTER` → `View::On*` | | UWP | C++/WinRT `AppContext` + manual swap-chain plumbing | `Runtime` + `View` from `IInspectable` | | X11 | `AppContext` + X11 event handlers | `Runtime` + `View` with `View::OnMouse*` | The shared `AppContext.{h,cpp}` is gone — each host now constructs a `Runtime` + `View` directly. ## Lifecycle contract - **First View attach is the heavy step.** Hosts MUST call `Resize` at least once with the surface's pixel dimensions before the first `RenderFrame` — the Device is built on the first Resize. - **One View per Runtime at a time.** `View` constructor throws `std::runtime_error` on double-attach. - **Destroy Views before their Runtime.** `~View` clears the back-reference, so LIFO destruction works; `~Runtime` asserts (debug builds) if a View is still attached. - **Frame thread**: `RenderFrame`, `Resize`, `Suspend`, `Resume`, `LoadScript`, `Eval`, `RunOnJsThread`, View construction/destruction. - **Any thread**: `OnPointer*` / `OnMouse*` / `IsSuspended` / `IsXrActive` / `SetXrWindow`. ## File scope ``` Embedding/ Include/Shared/Babylon/Embedding/{Runtime,View,RuntimeOptions,LogLevel}.h Source/{Runtime,View}.cpp + RuntimeImpl.h Android/ — JNI shared library (Java facade lives per-host; see Playground) Apple/ — BNRuntime / BNView / BNViewDelegate Apps/Playground/ Win32/ UWP/ X11/ macOS/ iOS/ visionOS/ Android/ (all migrated) ``` ## Closed open items - [x] AI-generated comment cleanup - [x] Naming (renamed to "Embedding") - [x] All Playground hosts migrated (Android, iOS, visionOS, macOS, UWP, Win32, X11) - [x] `RunOnJsThread` now has an `afterScriptLoad` parameter - [x] `Runtime` / `View` are regular movable value types - [x] Removed the planning `.md` from the repo root - [ ] Real fix for DeviceImpl.cpp issue
1 parent b320866 commit 063244d

70 files changed

Lines changed: 4365 additions & 1537 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/instructions/babylon-native-debugging.instructions.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ For pure RenderDoc CLI usage (any app, not BN-specific), see
2424
|---|---|
2525
| `Apps/Playground/Shared/CommandLine.{h,cpp}` | Argument parser. Single source of truth for supported flags. |
2626
| `Apps/Playground/Shared/Diagnostics.{h,cpp}` | Crash handler, `DumpFailure`, finish-line, exit-code tracking. |
27-
| `Apps/Playground/Shared/AppContext.cpp` | Wires `UnhandledExceptionHandler` + `console.error` into `DumpFailure`; injects `_playgroundOptions` into JS. |
27+
| `Apps/Playground/Win32/App.cpp` (and the other per-host `App.*`) | Wires the `RuntimeOptions::log` callback into `DumpFailure` (`JS CONSOLE ERROR` for `LogLevel::Error`, `UNCAUGHT JS ERROR` for `LogLevel::Fatal`) and injects `_playgroundOptions` into JS via `Runtime::RunOnJsThread`. |
2828
| `Apps/Playground/Scripts/validation_native.js` | Test runner. Reads `_playgroundOptions`, picks tests, calls `TestUtils.captureNextFrame()`. Reference-image load failures arrive via `BABYLON.Tools.LoadFile`'s `onLoadFileError` and are tagged with `MISSING_REFERENCE_IMAGE:`. |
2929
| `Apps/Playground/Scripts/config.json` | Test catalog. Each entry has `title`, `playgroundId`/`scriptToRun`, `referenceImage`, optional `excludeFromAutomaticTesting`/`reason`/`onlyVisual`/`renderCount`/`capture`/`threshold`/`errorRatio`. |
3030
| `Plugins/TestUtils/Source/TestUtils.cpp` | Native side of `TestUtils.captureNextFrame()` -- calls `m_deviceContext.RequestCaptureNextFrame()`. |
@@ -80,13 +80,15 @@ When you see `[Error] Error: Cannot load X` in stdout, **scroll up** --
8080
short error line. The short line is kept so legacy log scrapers still match.
8181

8282
### 3. JS stack on every `console.error`
83-
`AppContext.cpp`'s `Console::Initialize` callback calls
83+
The Embedding layer's `Console::Initialize` callback calls
8484
`Babylon::Polyfills::Console::CaptureCurrentJsStack(env)` on every
85-
`LogLevel::Error` message and appends the captured stack to the
86-
`DumpFailure` banner body. The capture is best-effort -- if the JS engine
87-
can't produce a stack (no JS context active, etc.) the helper returns an
88-
empty string and the banner just shows the message.
89-
Do not add per-callsite stack capture -- it's automatic.
85+
`LogLevel::Error` message and appends the captured stack to the message
86+
body that the host's `RuntimeOptions::log` callback receives (which the
87+
Playground hosts then route into the `DumpFailure` banner). The capture is
88+
best-effort — if the JS engine can't produce a stack (no JS context active,
89+
unsupported engine, etc.) the helper returns an empty string and the host
90+
just gets the bare message.
91+
Do not add per-callsite stack capture — it's automatic.
9092

9193
### 4. Colored finish line
9294
`Playground: Finished in <time>. (exit <code>)` is printed once at every exit

Apps/Playground/Android/BabylonNative/CMakeLists.txt

Lines changed: 21 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -8,45 +8,30 @@ project(BabylonNative)
88
get_filename_component(PLAYGROUND_DIR "${CMAKE_CURRENT_LIST_DIR}/../.." ABSOLUTE)
99
get_filename_component(REPO_ROOT_DIR "${PLAYGROUND_DIR}/../.." ABSOLUTE)
1010

11+
# Build the cross-platform Embedding facade *and* the Android JNI
12+
# interop layer (`libBabylonNativeEmbedding.so`). The Playground
13+
# Android app loads that library directly via `BabylonNative.java`
14+
# (in the `com.babylonjs.embedding` package).
15+
set(BABYLON_NATIVE_EMBEDDING ON CACHE BOOL "" FORCE)
16+
set(BABYLON_NATIVE_EMBEDDING_ANDROID ON CACHE BOOL "" FORCE)
17+
1118
add_subdirectory(${REPO_ROOT_DIR} "${CMAKE_CURRENT_BINARY_DIR}/BabylonNative")
1219

13-
add_library(BabylonNativeJNI SHARED
14-
src/main/cpp/BabylonNativeJNI.cpp
15-
${PLAYGROUND_DIR}/Shared/AppContext.cpp
16-
${PLAYGROUND_DIR}/Shared/Diagnostics.cpp)
20+
# Append a Playground-specific JNI helper to the generic
21+
# BabylonNativeEmbedding target so the bootstrap script list can stay
22+
# in Apps/Playground/Shared/PlaygroundScripts.{h,cpp} (shared with the
23+
# Win32 / iOS / macOS hosts) instead of being duplicated on the Java
24+
# side. Compiling these into the same .so keeps the Embedding layer
25+
# as a single in-process instance — handles produced by one entry point
26+
# can be safely consumed by another.
27+
target_sources(BabylonNativeEmbedding
28+
PRIVATE src/main/cpp/PlaygroundJNI.cpp
29+
${PLAYGROUND_DIR}/Shared/Diagnostics.cpp
30+
${PLAYGROUND_DIR}/Shared/PlaygroundScripts.cpp)
1731

18-
target_include_directories(BabylonNativeJNI
32+
target_include_directories(BabylonNativeEmbedding
1933
PRIVATE ${PLAYGROUND_DIR})
2034

21-
target_link_libraries(BabylonNativeJNI
22-
PRIVATE GLESv3
23-
PRIVATE android
35+
target_link_libraries(BabylonNativeEmbedding
2436
PRIVATE bx
25-
PRIVATE EGL
26-
PRIVATE log
27-
PRIVATE -lz
28-
PRIVATE AbortController
29-
PRIVATE AndroidExtensions
30-
PRIVATE AppRuntime
31-
PRIVATE Blob
32-
PRIVATE Canvas
33-
PRIVATE Console
34-
PRIVATE Fetch
35-
PRIVATE File
36-
PRIVATE GraphicsDevice
37-
PRIVATE NativeCamera
38-
PRIVATE NativeCapture
39-
PRIVATE NativeEncoding
40-
PRIVATE NativeEngine
41-
PRIVATE NativeInput
42-
PRIVATE NativeOptimizations
43-
PRIVATE NativeTracing
44-
PRIVATE NativeXr
45-
PRIVATE Performance
46-
PRIVATE ScriptLoader
47-
PRIVATE ShaderCache
48-
PRIVATE TestUtils
49-
PRIVATE TextDecoder
50-
PRIVATE TextEncoder
51-
PRIVATE Window
52-
PRIVATE XMLHttpRequest)
37+
PRIVATE Foundation) # for <Babylon/PerfTrace.h> in PlaygroundScripts.cpp
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
-keep class com.babylonjs.embedding.BabylonNative { *; }
2+
-keep class com.babylonjs.embedding.BabylonNative$RuntimeOptions { *; }

Apps/Playground/Android/BabylonNative/src/main/cpp/BabylonNativeJNI.cpp

Lines changed: 0 additions & 194 deletions
This file was deleted.
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
// Playground-specific JNI helper. Built into the same .so as the generic
2+
// Embedding/Android JNI (added via target_sources in the Playground
3+
// CMakeLists), so there's a single copy of Babylon::Embedding across
4+
// the process — avoids cross-library handle-passing UB.
5+
//
6+
// Sole purpose: surface Apps/Playground/Shared/PlaygroundScripts.{h,cpp}
7+
// to Java so the bootstrap script list stays in one place, shared with
8+
// the other Playground hosts.
9+
10+
#include <Babylon/Embedding/Runtime.h>
11+
#include <Babylon/Embedding/Android/RuntimeHandle.h>
12+
13+
#include <Shared/PlaygroundScripts.h>
14+
15+
#include <jni.h>
16+
17+
extern "C"
18+
{
19+
20+
JNIEXPORT void JNICALL
21+
Java_com_android_babylonnative_playground_PlaygroundActivity_loadBootstrapScripts(
22+
JNIEnv*, jclass, jlong runtimeHandle)
23+
{
24+
auto* runtime = Babylon::Embedding::Android::RuntimeFromHandle(runtimeHandle);
25+
if (runtime == nullptr)
26+
{
27+
return;
28+
}
29+
30+
// Idempotent process-wide setup (PerfTrace level, etc.).
31+
Playground::Initialize();
32+
33+
// Queue bootstrap scripts; they run after the first View attach
34+
// completes engine init on the JS thread.
35+
Playground::LoadBootstrapScripts(*runtime);
36+
}
37+
38+
} // extern "C"
39+

0 commit comments

Comments
 (0)