Skip to content

Commit c3a80de

Browse files
committed
feat(cli): expose simulator screenshot orientation metadata
1 parent 5db6f1c commit c3a80de

14 files changed

Lines changed: 222 additions & 20 deletions

File tree

.agents/tritonkit-skills/internal/tritonkit-ops-governance/SKILL.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,10 +24,10 @@ metadata:
2424
- 设备控制参考 Baguette 时,先区分 embedded TritonKit runtime 与 macOS host-side adapter:embedded runtime 只能承诺公开 UIKit API 可验证的 in-app 控制;SimulatorKit / HID / Home / App Switcher 等设备级动作必须等 host-side adapter,当前要返回明确 unsupported。
2525
- Wails 绑定先测绑定对象和 DTO;有真实 UI 后再补桌面窗口验收。
2626
- 当前前端为空白 Wails 静态入口;任何恢复 UI 的工作必须先新建或更新 `space` 与 BDD 场景。
27-
- Package Manager 集成时,embedded TritonKit runtime 只在 `DEBUG` 编译配置下生效;Release API 保持可编译但 runtime 必须 no-op,不按端类型或 UIKit 可导入性决定是否启用。
28-
- 业务 App 侧 iOS 接入示例必须把所有 TritonKit 符号放进独立 Debug bootstrap 文件,并用文件级 `#if DEBUG` 包住 `import TritonKit``TritonKit.shared.start()` / `start { config in ... }` facade;AppDelegate、SceneDelegate 或 SwiftUI 入口只保留 `#if DEBUG` 调用点,不能只依赖库内部 Release no-op。只有需要自定义 delegate 或消息路由时才使用低层 `delegate` / `connect(host:port:)`
29-
- SwiftPM / Xcode package product dependency 没有 CocoaPods-style Debug-only 配置开关;对外接入指南必须明确:默认走源码级 `#if DEBUG` bootstrap + Release no-op runtime,若生产 Release target 必须完全不链接 TritonKit,则使用独立 Debug-only app target / scheme。
30-
- Package Manager 分发同时覆盖 SwiftPM 与 CocoaPods;CocoaPods 规格必须保留 `TritonKitShared` / `TritonKit` 两个 Swift module,避免 `TritonKit` 中的 `import TritonKitShared` 在 pod 集成时失效
27+
- Package Manager 集成时,embedded TritonKit runtime 由 package 内部 Debug compile flag `TRITONKIT_RUNTIME_ENABLED` 控制;Debug package build 启用,Release package build API 保持可编译但 runtime 必须 no-op,不按端类型或 UIKit 可导入性决定是否启用。
28+
- 业务 App 侧 iOS 接入示例必须把所有 TritonKit 符号放进独立 Debug bootstrap 文件,并用文件级 `#if DEBUG` 包住 `import TritonKit``TritonKit.shared.start()` / `start { config in ... }` facade;AppDelegate、SceneDelegate 或 SwiftUI 入口只保留 `#if DEBUG` 调用点,不能只依赖 package 内部 Release no-op。只有需要自定义 delegate 或消息路由时才使用低层 `delegate` / `connect(host:port:)`
29+
- SwiftPM 支持 configuration-scoped build settings / compile conditions,但 SwiftPM / Xcode package product dependency 没有 CocoaPods-style Debug-only product dependency 开关;对外接入指南必须明确:默认走 package Debug compile flag + 源码级 `#if DEBUG` bootstrap + Release no-op runtime,若生产 Release target 必须完全不链接 TritonKit,则使用独立 Debug-only app target / scheme。
30+
- Package Manager 分发同时覆盖 SwiftPM 与 CocoaPods;用户接入面只显式选择 / 添加 `TritonKit`,不得要求业务 App 手写 `TritonKitShared`;内部仍保留 `TritonKitShared` / `TritonKit` 两个 Swift module 边界,CocoaPods 通过 `TritonKit.podspec` 传递解析 `TritonKitShared`,CI 需校验两个 podspec 可 lint
3131
- SwiftPM 根 `Package.swift` 只描述业务 App 可依赖的 embedded SDK product,不声明 `triton` CLI executable、不声明 Hummingbird / ArgumentParser 等 CLI-only package dependencies;macOS CLI 统一由 `CLI/Package.swift` 构建,命令为 `swift build --package-path CLI --scratch-path .build/cli -c release --product triton`,避免 iOS App 解析 SwiftPM 时拉入 CLI 依赖。
3232
- Swift 源文件超过 1500 行即进入治理范围;新增或扩展 CLI 能力时,默认按 `*Commands.swift``*Runtime.swift` / `*Service.swift``*Models.swift` 拆分。`*Models.swift` 只放 Codable/Encodable/Argument enum 等 wire contract,`*Commands.swift` 只放 ArgumentParser 参数和 glue,执行逻辑进入 runtime/service 文件。
3333
- 新增配置项时同步覆盖默认值、环境变量覆盖和非法值。

AGENTS.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -58,10 +58,10 @@
5858
17. 当用户明确要求“由 subagent 去做、主控 agent 负责监督”时,主控 agent 必须承担需求边界、任务拆分、集成、验收、文档与最终完成判断,不得在“代码已改完”但截图、实机验证、文档写回等验收环节仍未完成时提前停止。
5959
18. GitHub CI / Release 产物最终必须包含 macOS arm64 / x86_64 `triton` CLI 包、checksum manifest 和对外项目级 skill 包,至少覆盖 `.agents/tritonkit-skills/public/tritonkit-dev-feedback``.agents/tritonkit-skills/public/tritonkit-real-project-regression``.agents/tritonkit-skills/public/tritonkit-emulator-cli-takeover`,确保使用者能同时拿到命令行工具、开发阶段反馈工作流、真实项目回归流程和本机模拟器 CLI 接管流程。
6060
19. `triton` CLI 必须支持 Homebrew 二进制安装与更新;tag release 后先以 arm64 CLI 包、skill 包和 checksum manifest 创建 GitHub Release 并更新 tap formula,x86_64 包由 Intel runner 后补上传并再次刷新 checksum / tap,避免 x86 runner 阻塞 Apple Silicon 发布。
61-
20. 作为 Package Manager 依赖提供给业务 App 时,embedded TritonKit runtime 只在 `DEBUG` 编译配置下生效;Release 下必须保持可编译但不连接、不采集、不上传、不响应控制,不按 iOS/macOS 或 UIKit 可导入性作为启停边界。
62-
21. 业务 App 侧 iOS 接入文件必须使用独立 Debug bootstrap 文件,并用文件级 `#if DEBUG` 包住 `import TritonKit``TritonKit.shared.start()` / `start { config in ... }` facade;AppDelegate、SceneDelegate 或 SwiftUI 入口只保留 `#if DEBUG` 调用点,不能只依赖库内部 Release no-op。只有需要自定义 delegate 或消息路由时才使用低层 `delegate` / `connect(host:port:)`
63-
22. SwiftPM / Xcode package product dependency 没有 CocoaPods-style Debug-only 配置开关;对外接入指南必须明确:默认走源码级 `#if DEBUG` bootstrap + Release no-op runtime,若生产 Release target 必须完全不链接 TritonKit,则使用独立 Debug-only app target / scheme。
64-
23. Package Manager 分发入口同时覆盖 SwiftPM 与 CocoaPods;CocoaPods 必须保持 `TritonKitShared` `TritonKit` 两个 module 边界,CI 需校验 podspec 可 lint。
61+
20. 作为 Package Manager 依赖提供给业务 App 时,embedded TritonKit runtime 由 package 内部 Debug compile flag `TRITONKIT_RUNTIME_ENABLED` 控制;Debug package build 启用,Release package build 必须保持可编译但不连接、不采集、不上传、不响应控制,不按 iOS/macOS 或 UIKit 可导入性作为启停边界。
62+
21. 业务 App 侧 iOS 接入文件必须使用独立 Debug bootstrap 文件,并用文件级 `#if DEBUG` 包住 `import TritonKit``TritonKit.shared.start()` / `start { config in ... }` facade;AppDelegate、SceneDelegate 或 SwiftUI 入口只保留 `#if DEBUG` 调用点,不能只依赖 package 内部 Release no-op。只有需要自定义 delegate 或消息路由时才使用低层 `delegate` / `connect(host:port:)`
63+
22. SwiftPM 支持 configuration-scoped build settings / compile conditions,但 SwiftPM / Xcode package product dependency 没有 CocoaPods-style Debug-only product dependency 开关;对外接入指南必须明确:默认走 package Debug compile flag + 源码级 `#if DEBUG` bootstrap + Release no-op runtime,若生产 Release target 必须完全不链接 TritonKit,则使用独立 Debug-only app target / scheme。
64+
23. Package Manager 分发入口同时覆盖 SwiftPM 与 CocoaPods;用户接入面只显式选择 / 添加 `TritonKit`,不得要求业务 App 手写 `TritonKitShared`;内部仍保留 `TritonKitShared` `TritonKit` 两个 module 边界,CocoaPods 通过 `TritonKit.podspec` 传递解析 `TritonKitShared`CI 需校验两个 podspec 可 lint。
6565
24. SwiftPM 根 `Package.swift` 只描述业务 App 可依赖的 embedded SDK product,不声明 `triton` CLI executable、不声明 Hummingbird / ArgumentParser 等 CLI-only package dependencies;macOS CLI 统一由 `CLI/Package.swift` 构建,命令为 `swift build --package-path CLI --scratch-path .build/cli -c release --product triton`,避免 iOS App 解析 SwiftPM 时拉入 CLI 依赖。
6666
25. Swift 源文件超过 1500 行即进入治理范围;新增或扩展 CLI 能力时,默认按 `*Commands.swift``*Runtime.swift` / `*Service.swift``*Models.swift` 拆分,入口文件只保留 root command 注册与少量共享 glue,禁止继续把新子命令和 wire model 堆回巨型文件。
6767

CLI/Tests/TritonKitCLITests/SchemaFactSourceTests.swift

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3810,6 +3810,12 @@ struct SchemaFactSourceTests {
38103810
expectContract(sim, selector: "host.simulator-list", fields: [
38113811
"ok", "simulators",
38123812
])
3813+
expectContract(sim, selector: "host.simulator-screenshot", fields: [
3814+
"ok", "action", "runtimeScope", "target", "tool", "exitCode", "riskLevel",
3815+
"sourceCommand", "stdoutTruncated", "stderrTruncated", "stderr", "artifact",
3816+
"pixelWidth", "pixelHeight", "display", "display.rawLine", "display.displayID",
3817+
"display.screenID", "display.name", "orientationPolicy", "orientationNote", "note",
3818+
])
38133819
expectContract(sim, selector: "host.simulator-action", fields: [
38143820
"ok", "action", "runtimeScope", "target", "tool", "exitCode", "riskLevel",
38153821
"sourceCommand", "stdoutTruncated", "stderrTruncated", "artifacts", "note",
@@ -4056,6 +4062,7 @@ private func outputContractKindTaxonomy() -> Set<String> {
40564062
"runtime-ledger",
40574063
"runtime-ledger-entry",
40584064
"runtime-manifest",
4065+
"simulator-screenshot",
40594066
"runtime-snapshot",
40604067
"runtime-state",
40614068
"screenshot-metadata",

CLI/Tests/TritonKitCLITests/SimulatorAdvancedControlsTests.swift

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,7 @@ struct SimulatorAdvancedControlsTests {
154154
func simSchemaExposesAdvancedCommands() throws {
155155
let sim = try #require(commandSchemas().first { $0.name == "sim" })
156156
let usageForms = sim.usageForms.map(\.form)
157+
let optionNames = sim.options.map(\.name)
157158

158159
#expect(usageForms.contains(where: { $0.hasPrefix("status-bar") }))
159160
#expect(usageForms.contains(where: { $0.hasPrefix("privacy") }))
@@ -172,6 +173,7 @@ struct SimulatorAdvancedControlsTests {
172173
#expect(usageForms.contains(where: { $0.hasPrefix("erase ") }))
173174
#expect(usageForms.contains(where: { $0.hasPrefix("upgrade ") }))
174175
#expect(usageForms.contains(where: { $0.hasPrefix("personalization ") }))
176+
#expect(optionNames.contains("--display"))
175177
#expect(sim.providedCapabilities.contains("host-simulator"))
176178
#expect(sim.providedCapabilities.contains("sim-video"))
177179
#expect(sim.providedCapabilities.contains("sim-logs"))
@@ -183,6 +185,40 @@ struct SimulatorAdvancedControlsTests {
183185
#expect(sim.providedCapabilities.contains("sim-push"))
184186
}
185187

188+
@Test("sim screenshot metadata parser preserves CoreSimulator display details")
189+
func simScreenshotMetadataParserPreservesCoreSimulatorDisplayDetails() throws {
190+
let stderr = """
191+
Detected file type from extension: PNG
192+
Note: No display specified. Defaulting to display: 70A4519E-10D6-4D54-A93A-381327FA385A (screenID: 1, name: LCD)
193+
Wrote screenshot to: /tmp/jobmd-ipad-mini-current.png
194+
"""
195+
196+
let metadata = parseSimctlScreenshotDisplayMetadata(stderr: stderr)
197+
198+
#expect(metadata.rawLine?.contains("Defaulting to display") == true)
199+
#expect(metadata.displayID == "70A4519E-10D6-4D54-A93A-381327FA385A")
200+
#expect(metadata.screenID == "1")
201+
#expect(metadata.name == "LCD")
202+
}
203+
204+
@Test("sim screenshot schema exposes raw framebuffer orientation metadata")
205+
func simScreenshotSchemaExposesRawFramebufferOrientationMetadata() throws {
206+
let sim = try #require(commandSchemas().first { $0.name == "sim" })
207+
let contract = try #require(sim.outputContracts.first { $0.selector == "host.simulator-screenshot" })
208+
let fields = Set(contract.fields.map(\.name))
209+
210+
#expect(contract.model == "HostSimulatorScreenshotOutput")
211+
#expect(fields.contains("pixelWidth"))
212+
#expect(fields.contains("pixelHeight"))
213+
#expect(fields.contains("display"))
214+
#expect(fields.contains("display.displayID"))
215+
#expect(fields.contains("display.screenID"))
216+
#expect(fields.contains("display.name"))
217+
#expect(fields.contains("orientationPolicy"))
218+
#expect(fields.contains("orientationNote"))
219+
#expect(sim.successShape?.contains("orientationPolicy") == true)
220+
}
221+
186222
@Test("schema exposes xctrace and coverage artifact commands")
187223
func schemaExposesXctraceAndCoverageCommands() throws {
188224
let xcode = try #require(commandSchemas().first { $0.name == "xcode" })

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -392,6 +392,8 @@ triton app prefs set DEBUG-mock true --device booted --bundle-id com.example.app
392392
triton app prefs dump --device booted --bundle-id com.example.app --json
393393
```
394394

395+
`sim screenshot` captures the CoreSimulator framebuffer. Its JSON output includes `pixelWidth`, `pixelHeight`, `display.*`, `orientationPolicy=raw-framebuffer`, and `orientationNote`; use `--display internal|external|<screen-id>|<display-uuid>` when the default display selected by `simctl` is not the one you want. Triton does not rotate iPad framebuffer screenshots yet, so downstream evidence viewers should treat the orientation metadata as authoritative.
396+
395397
Destructive commands require `--confirm` by default, and `runtime delete` supports `--dry-run` first so agents can inspect the selected runtimes before deleting anything.
396398

397399
`app go <url>` is the short iOS deep-link smoke entry: it opens the URL, waits for embedded runtime readiness, returns an app/route/AX snapshot summary, and defaults to JSON output. Use `--device <selector>` only when the current/default target is ambiguous.

Sources/TritonKitCLI/CLIHostCommands.swift

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -295,17 +295,16 @@ struct SimScreenshot: AsyncParsableCommand {
295295

296296
@Option(help: "Simulator UDID or booted") var simulator: String = "booted"
297297
@Option(help: "Output PNG path") var output: String
298+
@Option(help: "CoreSimulator display selector, for example internal, external, screen id, or display UUID") var display: String?
298299
@Flag(help: "Alias for --format json") var json = false
299300
@Option(help: "Output format: text or json") var format: ClientOutputFormat = .json
300301

301302
func run() async throws {
302-
try runSimpleHostCommand(
303-
action: "sim.screenshot",
304-
target: "sim:\(simulator)",
305-
command: TKSimctlCommand.screenshot(udid: simulator, output: output),
306-
outputFormat: effectiveFormat(format, json: json),
307-
artifacts: [output],
308-
note: "Host-side simulator screenshot was written."
303+
try runHostSimulatorScreenshotCommand(
304+
simulator: simulator,
305+
command: TKSimctlCommand.screenshot(udid: simulator, output: output, display: display),
306+
outputPath: output,
307+
outputFormat: effectiveFormat(format, json: json)
309308
)
310309
}
311310
}

Sources/TritonKitCLI/CLIHostModels.swift

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,34 @@ struct HostToolProbeOutput: Encodable {
2222
let sourceCommand: String
2323
}
2424

25+
struct HostSimulatorScreenshotDisplayMetadata: Encodable, Equatable {
26+
let rawLine: String?
27+
let displayID: String?
28+
let screenID: String?
29+
let name: String?
30+
}
31+
32+
struct HostSimulatorScreenshotOutput: Encodable {
33+
let ok: Bool
34+
let action: String
35+
let runtimeScope: String
36+
let target: String
37+
let tool: String
38+
let exitCode: Int32
39+
let riskLevel: String
40+
let sourceCommand: String
41+
let stdoutTruncated: Bool
42+
let stderrTruncated: Bool
43+
let stderr: String?
44+
let artifact: String
45+
let pixelWidth: Int?
46+
let pixelHeight: Int?
47+
let display: HostSimulatorScreenshotDisplayMetadata
48+
let orientationPolicy: String
49+
let orientationNote: String
50+
let note: String
51+
}
52+
2553
struct HostDeviceTarget: Encodable, Equatable {
2654
let platform: String
2755
let id: String

Sources/TritonKitCLI/CLIHostRuntime.swift

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import ArgumentParser
22
import Darwin
33
import Foundation
4+
import ImageIO
45
import TritonKitShared
56

67
func runSimpleHostCommand(
@@ -89,6 +90,87 @@ func runHostCommandCapturingStdoutArtifact(
8990
}
9091
}
9192

93+
func runHostSimulatorScreenshotCommand(
94+
simulator: String,
95+
command: TKHostCommand,
96+
outputPath: String,
97+
outputFormat: ClientOutputFormat
98+
) throws {
99+
do {
100+
let result = try runHostCommand(command)
101+
let stderr = result.stderr.trimmingCharacters(in: .whitespacesAndNewlines)
102+
let size = imagePixelSize(path: outputPath)
103+
let output = HostSimulatorScreenshotOutput(
104+
ok: true,
105+
action: "sim.screenshot",
106+
runtimeScope: "host-simulator",
107+
target: "sim:\(simulator)",
108+
tool: command.executable,
109+
exitCode: result.exitCode,
110+
riskLevel: command.riskLevel.rawValue,
111+
sourceCommand: result.sourceCommand,
112+
stdoutTruncated: result.stdoutTruncated,
113+
stderrTruncated: result.stderrTruncated,
114+
stderr: stderr.isEmpty ? nil : stderr,
115+
artifact: outputPath,
116+
pixelWidth: size?.width,
117+
pixelHeight: size?.height,
118+
display: parseSimctlScreenshotDisplayMetadata(stderr: result.stderr),
119+
orientationPolicy: "raw-framebuffer",
120+
orientationNote: "simctl io screenshot writes the simulator display framebuffer as provided by CoreSimulator. Triton reports this as raw framebuffer orientation and does not rotate iPad screenshots yet.",
121+
note: "Host-side simulator screenshot was written with raw framebuffer orientation metadata."
122+
)
123+
switch outputFormat {
124+
case .json:
125+
print(try encodeJSON(output))
126+
case .text:
127+
print(outputPath)
128+
print(output.orientationNote)
129+
}
130+
} catch {
131+
try failHostCommand(error, outputFormat: outputFormat)
132+
}
133+
}
134+
135+
func parseSimctlScreenshotDisplayMetadata(stderr: String) -> HostSimulatorScreenshotDisplayMetadata {
136+
let line = stderr
137+
.split(whereSeparator: \.isNewline)
138+
.map(String.init)
139+
.first { $0.contains("Defaulting to display:") || $0.contains("display:") }
140+
guard let line else {
141+
return HostSimulatorScreenshotDisplayMetadata(rawLine: nil, displayID: nil, screenID: nil, name: nil)
142+
}
143+
144+
let displayID = value(after: "display:", before: "(", in: line)
145+
let screenID = value(after: "screenID:", before: ",", in: line)
146+
let name = value(after: "name:", before: ")", in: line)
147+
return HostSimulatorScreenshotDisplayMetadata(
148+
rawLine: line,
149+
displayID: displayID,
150+
screenID: screenID,
151+
name: name
152+
)
153+
}
154+
155+
func imagePixelSize(path: String) -> (width: Int, height: Int)? {
156+
guard let source = CGImageSourceCreateWithURL(URL(fileURLWithPath: path) as CFURL, nil),
157+
let properties = CGImageSourceCopyPropertiesAtIndex(source, 0, nil) as? [CFString: Any],
158+
let width = properties[kCGImagePropertyPixelWidth] as? Int,
159+
let height = properties[kCGImagePropertyPixelHeight] as? Int
160+
else {
161+
return nil
162+
}
163+
return (width, height)
164+
}
165+
166+
private func value(after marker: String, before terminator: String, in line: String) -> String? {
167+
guard let markerRange = line.range(of: marker) else { return nil }
168+
let tail = line[markerRange.upperBound...]
169+
let end = tail.range(of: terminator)?.lowerBound ?? tail.endIndex
170+
let value = tail[..<end].trimmingCharacters(in: .whitespacesAndNewlines)
171+
return value.isEmpty ? nil : value
172+
}
173+
92174
private struct HostPipeDrainResult {
93175
let data: Data
94176
let bytes: Int

0 commit comments

Comments
 (0)