Fix LiteRT-LM streaming and GPU packaging

[leehack]

Summary

• Fix LiteRT-LM response limits being passed to the upstream forced benchmark decode setting; request limits now use the per-generation maxOutputTokens option, so EOS completes immediately.
• Flush the first LiteRT-LM chat token immediately and avoid rebuilding the runtime client when only the response limit changes.
• Update the native runtime to v0.14.0-native.2, which exports Android GPU plugin symbols globally and uses the checksum-pinned official Apple runtime.
• Enable LiteRT-LM GPU/Auto on iOS, restore macOS app runtime embedding, and reduce Apple packaging to the consolidated runtime libraries.
• Update runtime pin tooling, companion SwiftPM targets, README, support matrix, changelogs, and backend-selection docs.

Production-readiness scope

• User-facing scope: Native LiteRT-LM responses can use allowances such as 1024 tokens without being forced to decode that many tokens; Android GPU no longer stalls from unresolved plugin symbols; iOS and macOS can select Metal.
• Supported platforms/paths: Flutter chat app and core native LiteRT-LM paths on Android arm64, iOS arm64, and macOS arm64; Apple SwiftPM also covers macOS x64 artifacts.
• Unsupported or intentionally unavailable paths: iOS x86_64 simulator remains unavailable because no runtime artifact is published. NPU remains Android-only and deployment/model-specific. Web behavior is unchanged.
• Out of scope / follow-ups: Cold model initialization remains runtime/model dependent (about 31 s on the tested Pixel and 16 s on the tested iPad); this PR removes the post-EOS/forced-decode wait, not model loading cost. No merge-blocking follow-up.

Completeness checklist

• Declared scope is fully implemented, or explicitly reduced above.
• Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable.
• Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant.
• Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
• Security/privacy review completed: no secrets, bearer tokens, signed URLs, or raw secret-bearing paths leak through logs, cache keys, metadata, errors, or snapshots.
• Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above.

PR type guidance

• Bugfix PR: The upstream engine setting was a forced benchmark decode count, not a response cap. Regression tests verify per-request output limits, client reuse, immediate stream flushing, consolidated platform assets, and incomplete-cache rejection.

Test Plan

• Changed Dart files pass dart format --output=none --set-exit-if-changed.
• Focused dart analyze for all changed core/hook/test paths: no issues.
• Focused VM regression suite: 145 tests passed.
• Chat app Flutter tests: 4 passed.
• LiteRT-LM companion Flutter tests: 3 passed.
• ./tool/docs/build_site.sh and ./tool/docs/validate_links.sh.
• dart run tool/testing/verify_release_docs_versions.dart.
• Full repository dart analyze: not used because independently managed examples do not have all package dependencies resolved from the root; all changed paths were analyzed directly.
• Chrome tests: N/A; native-only runtime and packaging change.

Matrix Evidence

Matrix row	Scope covered	Platform / model / backend	Result	Evidence / notes
Android release/device smoke	Native packaging, GPU load, 1024-token allowance, EOS completion	Pixel 9 Pro / Gemma 4 E2B / GPU	PASS	APK has DF_1_GLOBAL; warmed run completed in 337 ms with EOS and no forced-token wait.
iOS release/device smoke	Consolidated runtime, Metal symbols, generation	iPad / Gemma 4 E2B / GPU	PASS	Signed 64.1 MB app installed/launched; warm first token about 0.7 s; GPU smoke completed in 592 ms.
macOS release/runtime smoke	Runtime embedding, signing, Metal, 1024-token allowance	Apple Silicon Mac / Gemma 4 E2B / GPU	PASS	Signed app contains only two consolidated LiteRT-LM dylibs; warm generation completed in 124 ms.
Native runtime release	Android visibility and Apple official artifacts	litert-lm-native@v0.14.0-native.2	PASS	All 9 release build/package jobs passed; consumer pins and checksums updated.
Docs	Public runtime/platform contract	Docusaurus	PASS	Static build and strict link validation passed.

Review Notes

• Independent review status: PR-readiness code, optimization, regression, and docs pass completed locally.
• CI status / head SHA: all review comments resolved; all 13 GitHub checks passed for dd5991ade.

[github-actions]

Chat app preview removed for leehack/llamadart-chat-pr-297.

[Copilot]

Pull request overview

This PR updates the pinned native LiteRT-LM runtime to v0.14.0-native.2 and aligns the Dart runtime, build hook, Flutter SwiftPM companion package, macOS app embedding script, tests, and docs with the new packaging and streaming semantics (notably avoiding misuse of LiteRT-LM “decode token” benchmark controls as response limits).

Changes:

• Bump native LiteRT-LM pin to v0.14.0-native.2 and refresh checksums / bundle requirements across hook, runtime, docs, and companion SwiftPM package.
• Fix native LiteRT-LM generation so requested response limits apply per request (streaming promptly) rather than being treated as a forced benchmark decode-step count.
• Restore/adjust macOS packaging flow (prepare script + Xcode build phase) and expand iOS/macOS GPU selection/documentation.

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
website/docs/platforms/support-matrix.md	Updates LiteRT-LM runtime pin and platform/backend coverage notes (including iOS GPU).
website/docs/guides/backend-selection.md	Aligns backend capability table with iOS/macOS LiteRT-LM CPU/GPU support.
website/docs/changelog/recent-releases.md	Adds Unreleased notes for LiteRT-LM streaming fix and runtime pin bump.
tool/native/sync_native_release_pins.py	Extends pin-sync script to also update the macOS prepare script’s pinned LiteRT-LM version.
tool/macos_litert_lm_prepare_app.sh	Simplifies required runtime library expectations and updates pinned cache-path candidates to v0.14.0-native.2.
test/unit/tooling/sync_native_release_pins_script_test.dart	Adds coverage ensuring the pin-sync script updates the macOS prepare script.
test/unit/tooling/macos_litert_lm_prepare_app_script_test.dart	Adds regression test asserting the chat app’s macOS build phase ordering and updates library expectations.
test/unit/hook/build_hook_litert_lm_integration_test.dart	Updates iOS/macOS bundle expectations and tightens iOS code-asset filtering behavior.
test/unit/backends/litert_lm/litert_lm_service_test.dart	Updates expectations for output token defaults and adds test ensuring response limits change without recreating the client.
test/unit/backends/litert_lm/litert_lm_runtime_test.dart	Updates macOS required library/framework/native-SPM file expectations for the new runtime packaging.
test/unit/backends/litert_lm/litert_lm_platform_test.dart	Updates platform backend availability/defaults to include iOS GPU support.
README.md	Updates the documented default native LiteRT-LM pin to v0.14.0-native.2.
packages/llamadart_litert_lm_flutter/test/llamadart_litert_lm_flutter_test.dart	Updates tests for the SwiftPM manifest and macOS import expectations.
packages/llamadart_litert_lm_flutter/README.md	Updates companion-package documentation to the new pin and macOS SwiftPM status.
packages/llamadart_litert_lm_flutter/darwin/llamadart_litert_lm_flutter/Sources/llamadart_litert_lm_flutter/LlamadartLiteRtLmPlugin.swift	Imports LiteRT-LM on macOS (under conditional compilation) to match restored macOS SwiftPM artifacts.
packages/llamadart_litert_lm_flutter/darwin/llamadart_litert_lm_flutter/Package.swift	Bumps LiteRT-LM tag/checksums; replaces iOS Gemma provider target with consolidated upstream artifacts; enables macOS artifact set.
packages/llamadart_litert_lm_flutter/CHANGELOG.md	Adds Unreleased entry documenting the updated pin and consolidated artifacts/macOS enablement.
lib/src/backends/litert_lm/litert_lm_service.dart	Stops treating response length as a client-recreation parameter; relies on per-request max output tokens.
lib/src/backends/litert_lm/litert_lm_runtime.dart	Bumps runtime version; simplifies macOS required library/framework sets; fixes generation semantics by avoiding benchmark decode-token forcing and using per-request output limits.
lib/src/backends/litert_lm/litert_lm_runtime_stub.dart	Updates stub docs to clarify outputTokens meaning.
lib/src/backends/litert_lm/litert_lm_platform.dart	Extends iOS to report GPU backend availability/defaults.
hook/build.dart	Bumps pinned LiteRT-LM version + checksums; narrows iOS dynamic library emission to only required files.
example/chat_app/test/chat_generation_service_test.dart	Asserts LiteRT-LM chat defaults flush the first token immediately via stream batching threshold.
example/chat_app/macos/Runner.xcodeproj/project.pbxproj	Adds “Prepare LiteRT-LM Runtime” build phase after Flutter assembly to embed/sign runtime into the macOS app bundle.
example/chat_app/lib/services/chat_generation_service.dart	Forces LiteRT-LM chat streaming to flush the first token immediately (streamBatchTokenThreshold: 1).
CHANGELOG.md	Adds Unreleased entries documenting LiteRT-LM runtime pin bump, iOS Metal enablement, macOS packaging restoration, and streaming fix.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated no new comments.