Skip to content

Commit c36e05a

Browse files
authored
✨ Harden Swift SDK package (#252)
## Why The Swift SDK was doing the right high-level job, but its package shape made native app integration heavier than it needed to be. The core `Vizzly` product pulled in XCTest helpers, and Swift comparison options were sent at the top level even though the TDD server reads per-screenshot comparison settings from `properties`. We also did not have the same real E2E confidence for Swift that the other SDKs have. A green unit test could still miss server discovery, actual PNG upload, local baseline creation, repeated screenshot matching, or the cloud CI path. ## What changed - Split XCTest convenience APIs into a new `VizzlyXCTest` product while keeping the core `Vizzly` client lightweight. - Updated Swift screenshot payload construction so `threshold`, `minClusterSize`, and `fullPage` flow through `properties`, matching the server contract. - Added focused Swift tests for payload shaping and default server-config behavior. - Added `VizzlyE2ETests`, which upload real PNG bytes through `VizzlyClient` against a real local TDD server. - Added `npm run test:swift:e2e`, which builds the CLI, starts an isolated TDD run in a temp directory, and runs the Swift E2E suite. - Added the Swift SDK to `.github/workflows/sdk-e2e.yml` with both TDD-mode and cloud-mode runs. - Wired cloud-mode Swift E2E through the repo secret `VIZZLY_SWIFT_CLIENT_TOKEN`, matching the per-SDK token pattern used by the other SDKs. - Simplified the synchronous upload path with a small request-state helper for timeout/cancel handling. - Updated Swift SDK docs and examples for the SPM product split, E2E command, direct-client usage, and current integration guidance. ## Validation - `swift test` - `npm run test:swift:e2e` - `swift build --sdk /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator26.5.sdk --triple arm64-apple-ios13.0-simulator` - `swift package describe --type json` - `git diff --check` - `ruby -e 'require "yaml"; YAML.load_file(".github/workflows/sdk-e2e.yml"); puts "ok"'` - Confirmed `VIZZLY_SWIFT_CLIENT_TOKEN` exists in repo Actions secrets.
1 parent c9e4974 commit c36e05a

12 files changed

Lines changed: 532 additions & 132 deletions

File tree

.github/workflows/sdk-e2e.yml

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -376,11 +376,51 @@ jobs:
376376
VIZZLY_COMMIT_MESSAGE: ${{ github.event.pull_request.head.commit.message || github.event.head_commit.message }}
377377
VIZZLY_COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.event.head_commit.id }}
378378

379+
# Swift SDK
380+
swift:
381+
name: Swift SDK
382+
runs-on: macos-latest
383+
timeout-minutes: 10
384+
385+
steps:
386+
- uses: actions/checkout@v4
387+
388+
- name: Use Node.js 22
389+
uses: actions/setup-node@v4
390+
with:
391+
node-version: 22
392+
cache: 'npm'
393+
394+
- name: Install CLI dependencies
395+
run: npm ci
396+
397+
- name: Build CLI
398+
run: npm run build
399+
400+
- name: Build Swift package
401+
working-directory: ./clients/swift
402+
run: swift build
403+
404+
- name: Run E2E tests (TDD mode)
405+
working-directory: ./clients/swift
406+
run: ../../bin/vizzly.js tdd run "VIZZLY_E2E=1 swift test --filter VizzlyE2ETests"
407+
env:
408+
CI: true
409+
410+
- name: Run E2E tests (Cloud mode)
411+
working-directory: ./clients/swift
412+
run: ../../bin/vizzly.js run "VIZZLY_E2E=1 swift test --filter VizzlyE2ETests"
413+
env:
414+
CI: true
415+
VIZZLY_TOKEN: ${{ secrets.VIZZLY_SWIFT_CLIENT_TOKEN }}
416+
VIZZLY_COMMIT_MESSAGE: ${{ github.event.pull_request.head.commit.message || github.event.head_commit.message }}
417+
VIZZLY_COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.event.head_commit.id }}
418+
379419
# Status check for branch protection
380420
check:
381421
name: E2E Status
382422
runs-on: ubuntu-latest
383-
needs: [core-js, vitest, storybook, static-site, ember, ruby]
423+
needs: [core-js, vitest, storybook, static-site, ember, ruby, swift]
384424
if: always()
385425
steps:
386426
- name: Check all SDK E2E tests passed
@@ -409,4 +449,8 @@ jobs:
409449
echo "Ruby SDK E2E tests failed"
410450
exit 1
411451
fi
452+
if [[ "${{ needs.swift.result }}" == "failure" ]]; then
453+
echo "Swift SDK E2E tests failed"
454+
exit 1
455+
fi
412456
echo "All SDK E2E tests passed"

clients/swift/Example/ExampleUITests.swift

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import XCTest
22
import Vizzly
3+
import VizzlyXCTest
34

45
/// Example UI tests demonstrating Vizzly integration
56
///
@@ -133,15 +134,13 @@ final class ExampleUITests: XCTestCase {
133134
func testResponsiveLayout() throws {
134135
// Test different device orientations
135136
XCUIDevice.shared.orientation = .portrait
136-
sleep(1) // Wait for orientation change
137137

138138
app.vizzlyScreenshot(
139139
name: "home-portrait",
140140
properties: ["orientation": "portrait"]
141141
)
142142

143143
XCUIDevice.shared.orientation = .landscapeLeft
144-
sleep(1)
145144

146145
app.vizzlyScreenshot(
147146
name: "home-landscape",
@@ -187,7 +186,7 @@ final class ExampleUITests: XCTestCase {
187186
// MARK: - Custom Threshold Example
188187

189188
func testWithCustomThreshold() throws {
190-
// Allow up to 5% pixel difference
189+
// Allow a higher comparison threshold for animated content
191190
app.vizzlyScreenshot(
192191
name: "animation-test",
193192
threshold: 5,

clients/swift/INTEGRATION.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,10 @@ In Xcode:
2121
1. **File → Add Package Dependencies**
2222
2. Enter URL: `https://github.com/vizzly-testing/cli`
2323
3. Select version/branch
24-
4. **Important**: Add the package to your **UI Test target** (not the main app target)
24+
4. Add the `VizzlyXCTest` product to your **UI Test target**
25+
26+
Use the core `Vizzly` product directly only when you need to send PNG data from
27+
app or test-support code without the XCTest convenience extensions.
2528

2629
#### Option B: Local Package
2730

@@ -78,6 +81,7 @@ Create or update your UI test file:
7881
```swift
7982
import XCTest
8083
import Vizzly
84+
import VizzlyXCTest
8185

8286
final class MyAppUITests: XCTestCase {
8387

@@ -265,13 +269,13 @@ For views with animations or timing-sensitive content:
265269
func testAnimatedView() {
266270
app.launch()
267271

268-
// Wait for animation to complete
269-
sleep(1) // Or use expectations
272+
let finishedState = app.otherElements["AnimatedBannerReady"]
273+
XCTAssertTrue(finishedState.waitForExistence(timeout: 5))
270274

271275
// Use threshold for slight variations
272276
app.vizzlyScreenshot(
273277
name: "animated-banner",
274-
threshold: 5 // Allow 5% difference
278+
threshold: 5
275279
)
276280
}
277281
```

clients/swift/Package.swift

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,22 @@ let package = Package(
1313
.library(
1414
name: "Vizzly",
1515
targets: ["Vizzly"]),
16+
.library(
17+
name: "VizzlyXCTest",
18+
targets: ["VizzlyXCTest"]),
1619
],
1720
targets: [
1821
.target(
1922
name: "Vizzly",
2023
dependencies: []),
24+
.target(
25+
name: "VizzlyXCTest",
26+
dependencies: ["Vizzly"]),
2127
.testTarget(
2228
name: "VizzlyTests",
29+
dependencies: ["Vizzly", "VizzlyXCTest"]),
30+
.testTarget(
31+
name: "VizzlyE2ETests",
2332
dependencies: ["Vizzly"]),
2433
]
2534
)

clients/swift/QUICKSTART.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ npm install -g @vizzly-testing/cli
1313
1. Open your iOS project in Xcode
1414
2. **File → Add Package Dependencies**
1515
3. Paste: `https://github.com/vizzly-testing/cli`
16-
4. Add to your **UI Test target** (not main app)
16+
4. Add the `VizzlyXCTest` product to your **UI Test target**
1717

1818
## 3. Start TDD Server
1919

@@ -28,6 +28,7 @@ vizzly tdd start
2828
```swift
2929
import XCTest
3030
import Vizzly
31+
import VizzlyXCTest
3132

3233
class MyAppUITests: XCTestCase {
3334
let app = XCUIApplication()
@@ -96,7 +97,7 @@ button.vizzlyScreenshot(name: "submit-button")
9697
### Custom Threshold
9798

9899
```swift
99-
// Allow 5% pixel difference (useful for animations)
100+
// Allow a higher comparison threshold for animated content
100101
app.vizzlyScreenshot(
101102
name: "animated-view",
102103
threshold: 5

clients/swift/README.md

Lines changed: 45 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ Unlike tools that render components in isolation, Vizzly captures screenshots di
77
## Features
88

99
- **Zero Configuration** - Auto-discovers Vizzly TDD server
10-
- **Native XCTest Integration** - Simple extensions for `XCUIApplication` and `XCUIElement`
10+
- **Native XCTest Integration** - Simple extensions for `XCUIApplication` and `XCUIElement` via the `VizzlyXCTest` helper product
1111
- **iOS & macOS Support** - Works on both platforms
1212
- **Automatic Metadata** - Captures device, screen size, and platform info
1313
- **TDD Mode** - Local visual testing with instant feedback
@@ -22,21 +22,29 @@ Add Vizzly to your test target using Xcode:
2222

2323
1. File → Add Package Dependencies
2424
2. Enter repository URL: `https://github.com/vizzly-testing/cli`
25-
3. Select version and add to your UI test target
25+
3. Select version and add the `VizzlyXCTest` product to your UI test target
26+
27+
The core `Vizzly` product has no XCTest dependency and can also be used from
28+
native app or test-support code when you want to send PNG data directly.
2629

2730
Or add to your `Package.swift`:
2831

2932
```swift
3033
dependencies: [
3134
.package(url: "https://github.com/vizzly-testing/cli", from: "1.0.0")
35+
],
36+
targets: [
37+
.testTarget(
38+
name: "MyAppUITests",
39+
dependencies: [
40+
.product(name: "VizzlyXCTest", package: "cli")
41+
]
42+
)
3243
]
3344
```
3445

35-
### CocoaPods
36-
37-
```ruby
38-
pod 'Vizzly', :git => 'https://github.com/vizzly-testing/cli', :branch => 'main'
39-
```
46+
Vizzly does not currently ship a CocoaPods podspec. Use Swift Package Manager
47+
for native app integration.
4048

4149
## Quick Start
4250

@@ -54,6 +62,7 @@ This starts a local server at `http://localhost:47392` that receives screenshots
5462
```swift
5563
import XCTest
5664
import Vizzly
65+
import VizzlyXCTest
5766

5867
class MyUITests: XCTestCase {
5968
let app = XCUIApplication()
@@ -128,14 +137,17 @@ func testNavigationBar() {
128137

129138
```swift
130139
func testAnimatedContent() {
131-
// Allow up to 5% pixel difference (useful for animations)
140+
// Allow a higher comparison threshold for animated content
132141
app.vizzlyScreenshot(
133142
name: "animated-banner",
134143
threshold: 5
135144
)
136145
}
137146
```
138147

148+
If `threshold` or `minClusterSize` is omitted, the server's configured
149+
comparison settings are used.
150+
139151
### Multiple Device Orientations
140152

141153
```swift
@@ -187,7 +199,8 @@ extension XCUIApplication {
187199
func vizzlyScreenshot(
188200
name: String,
189201
properties: [String: Any]? = nil,
190-
threshold: Int = 0,
202+
threshold: Double? = nil,
203+
minClusterSize: Int? = nil,
191204
fullPage: Bool = false
192205
) -> [String: Any]?
193206
}
@@ -200,7 +213,8 @@ extension XCUIElement {
200213
func vizzlyScreenshot(
201214
name: String,
202215
properties: [String: Any]? = nil,
203-
threshold: Int = 0
216+
threshold: Double? = nil,
217+
minClusterSize: Int? = nil
204218
) -> [String: Any]?
205219
}
206220
```
@@ -213,15 +227,17 @@ extension XCTestCase {
213227
name: String,
214228
app: XCUIApplication,
215229
properties: [String: Any]? = nil,
216-
threshold: Int = 0,
230+
threshold: Double? = nil,
231+
minClusterSize: Int? = nil,
217232
fullPage: Bool = false
218233
) -> [String: Any]?
219234

220235
func vizzlyScreenshot(
221236
name: String,
222237
element: XCUIElement,
223238
properties: [String: Any]? = nil,
224-
threshold: Int = 0
239+
threshold: Double? = nil,
240+
minClusterSize: Int? = nil
225241
) -> [String: Any]?
226242
}
227243
```
@@ -236,7 +252,8 @@ class VizzlyClient {
236252
name: String,
237253
image: Data,
238254
properties: [String: Any]? = nil,
239-
threshold: Int = 0,
255+
threshold: Double? = nil,
256+
minClusterSize: Int? = nil,
240257
fullPage: Bool = false
241258
) -> [String: Any]?
242259

@@ -254,8 +271,9 @@ class VizzlyClient {
254271
The SDK automatically discovers a running Vizzly TDD server using this priority order:
255272

256273
1. **VIZZLY_SERVER_URL environment variable** - Explicitly set server URL
257-
2. **Global server file** - `~/.vizzly/server.json` written by CLI
258-
3. **Default port health check** - Tests `http://localhost:47392/health`
274+
2. **Project server file** - `.vizzly/server.json` in the current directory
275+
3. **Global server file** - `~/.vizzly/server.json` written by CLI
276+
4. **Default port health check** - Tests `http://localhost:47392/health`
259277

260278
When you run `vizzly tdd start`, the CLI automatically writes server info to `~/.vizzly/server.json` in your home directory, enabling zero-config discovery from iOS tests.
261279

@@ -440,6 +458,18 @@ Check out the `Example/` directory for:
440458
- Custom properties and thresholds
441459
- Direct client usage
442460

461+
## SDK E2E Tests
462+
463+
The Swift SDK has an end-to-end test path that runs against a real local
464+
Vizzly TDD server and uploads real PNG bytes through `VizzlyClient`:
465+
466+
```bash
467+
npm run test:swift:e2e
468+
```
469+
470+
This command builds the CLI, starts an isolated TDD run in a temp directory,
471+
and executes the `VizzlyE2ETests` SwiftPM suite.
472+
443473
## Contributing
444474

445475
Bug reports and pull requests are welcome at https://github.com/vizzly-testing/cli

0 commit comments

Comments
 (0)