Skip to content

Commit ceea27e

Browse files
muzahidul-opticlaude
muzahidul-opti
and
claude
authored
[FSSDK-12240] fix flaky tests with proper synchronization (#622)
* docs: add comprehensive project documentation in CLAUDE.md Added CLAUDE.md file to provide context and guidelines for development: - Project overview and structure - Getting started guide with platform support and installation - Coding standards and common patterns (protocol-oriented design, thread safety) - Complete development workflow from branch creation to PR - Testing guide with xcodebuild commands and best practices - Key API usage examples - Helpful commands for file searching, testing, and git operations The documentation is organized logically from introduction to reference material for better developer onboarding and contribution experience. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * docs: remove redundant instructions from CLAUDE.md Consolidated duplicate sections to improve readability: - Moved all testing commands to dedicated 'Testing' section - Kept linting info only in 'Coding Standards' - Consolidated git commands in 'Helpful Commands' - Added cross-references in 'Making Changes' workflow - Removed repeated commands from multiple locations This reduces redundancy and makes the guide easier to maintain. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix: resolve flaky tests and improve test reliability Fixed multiple test failures and flaky tests across the test suite: 1. **CMAB Test Fixes** (OptimizelyUserContextTests_Decide_CMAB.swift): - Fixed testDecideAsync_cmabCacheOptions failing with "Expected 3 calls" - Root cause: CMAB cache persisting between tests causing cache hits - Solution: Reset cache in setUp/tearDown, use unique userId - Added thread-safe state tracking in MockCmabService - Added clarifying comments for variation IDs - Removed commented-out code 2. **Event Dispatcher Retry Tests** (EventDispatcherRetryTests.swift): - Fixed testRetry_AllAttemptsExhausted timing issues - Fixed testRetry_NetworkReachabilityDown synchronization - Replaced unreliable asyncAfter with notify.wait() for proper completion - Added setUp queue clearing to prevent test pollution 3. **JSON Encoding Determinism** (Utils.swift, OTUtils.swift): - Fixed testEncodeJSON failures on Xcode 16.x - Added .sortedKeys to JSONEncoder for deterministic key ordering - Updated expected JSON strings in DecisionServiceTests_Experiments - Updated expected JSON strings in OptimizelyUserContextTests_Decide_Reasons 4. **Event Dispatcher Batch Tests** (EventDispatcherTests_Batch.swift): - Fixed ArrayEventForDispatch extension edge cases - Improved batch processing test reliability 5. **CI Configuration** (swift.yml, unit_tests.yml): - Updated test runner configuration for better reliability All tests now pass reliably both individually and in the full suite. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix workflow branch --------- Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent 57e6bbf commit ceea27e

10 files changed

Lines changed: 492 additions & 94 deletions

‎.github/workflows/swift.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,11 @@ env:
2222
jobs:
2323

2424
lint_markdown_files:
25-
uses: optimizely/swift-sdk/.github/workflows/lint_markdown.yml@fix-release-process
25+
uses: optimizely/swift-sdk/.github/workflows/lint_markdown.yml@master
2626

2727
integration_tests:
2828
if: "${{ github.event.inputs.PREP == '' && github.event.inputs.RELEASE == '' }}"
29-
uses: optimizely/swift-sdk/.github/workflows/integration_tests.yml@fix-release-process
29+
uses: optimizely/swift-sdk/.github/workflows/integration_tests.yml@master
3030
secrets:
3131
CI_USER_TOKEN: ${{ secrets.CI_USER_TOKEN }}
3232

@@ -46,7 +46,7 @@ jobs:
4646
4747
unittests:
4848
if: "${{ github.event.inputs.PREP == '' && github.event.inputs.RELEASE == '' }}"
49-
uses: optimizely/swift-sdk/.github/workflows/unit_tests.yml@fix-release-process
49+
uses: optimizely/swift-sdk/.github/workflows/unit_tests.yml@master
5050
prepare_for_release:
5151
runs-on: macos-15
5252
if: "${{ github.event.inputs.PREP == 'true' && github.event_name == 'workflow_dispatch' }}"

‎CLAUDE.md‎

Lines changed: 335 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,335 @@
1+
# Optimizely Swift SDK - Claude Code Context
2+
3+
## Project Overview
4+
This is the Optimizely Swift SDK for Feature Experimentation and Full Stack. It provides A/B testing and feature management capabilities for iOS, tvOS, and watchOS platforms.
5+
6+
## Getting Started
7+
8+
### Platform Support
9+
- iOS 10.0+
10+
- tvOS 10.0+
11+
- watchOS 3.0+
12+
- Swift 5+
13+
14+
### Installation Methods
15+
- Swift Package Manager (preferred)
16+
- CocoaPods
17+
18+
### Dependencies
19+
- SwiftLint (development)
20+
21+
### Initial Setup
22+
```bash
23+
# Install dependencies
24+
pod install
25+
26+
# Build the SDK
27+
swift build
28+
29+
# Verify setup (see "Testing" section for detailed commands)
30+
swift test
31+
```
32+
33+
## Project Structure
34+
35+
### Source Code Organization
36+
37+
#### Core Modules
38+
- **Sources/Optimizely/**: Main SDK entry point and client implementation
39+
- `OptimizelyClient.swift`: Primary SDK interface
40+
- `OptimizelyConfig.swift`: Configuration management
41+
- `VuidManager.swift`: Visitor unique ID management
42+
43+
- **Sources/Optimizely+Decide/**: Decision-making and user context
44+
- `OptimizelyUserContext.swift`: User context for decision-making
45+
- `OptimizelyDecision.swift`: Decision results
46+
- `OptimizelyDecideOption.swift`: Decision options and flags
47+
48+
- **Sources/Data Model/**: Data structures for experiments, features, and events
49+
- Core entities: Experiment, FeatureFlag, Variation, Event, Audience
50+
- CMAB (Contextual Multi-Armed Bandit) models
51+
- Holdout configurations
52+
53+
- **Sources/Implementation/**: Core business logic
54+
- `DefaultDecisionService.swift`: Decision-making engine
55+
- `DefaultBucketer.swift`: User bucketing logic
56+
- Event handling and batch processing
57+
58+
- **Sources/CMAB/**: Contextual Multi-Armed Bandit implementation
59+
- `CmabClient.swift`: Client for CMAB predictions
60+
- `CmabConfig.swift`: Configuration for CMAB
61+
- `CmabService.swift`: Service layer for CMAB operations
62+
63+
- **Sources/ODP/**: Optimizely Data Platform integration
64+
- Event and segment management
65+
- API managers for ODP communication
66+
67+
- **Sources/Customization/**: Extensibility points
68+
- Protocol definitions for custom handlers
69+
- Default implementations (logger, event dispatcher, datafile handler)
70+
71+
- **Sources/Utils/**: Shared utilities
72+
- Atomic properties and thread-safe collections
73+
- Hashing (MurmurHash3)
74+
- Network reachability
75+
76+
#### Test Organization
77+
- **Tests/OptimizelyTests-Common/**: Common utility and core functionality tests
78+
- **Tests/OptimizelyTests-APIs/**: Public API tests
79+
- **Tests/OptimizelyTests-Batch/**: Event batching and dispatching tests
80+
- **Tests/OptimizelyTests-DataModel/**: Data model tests
81+
- **Tests/TestData/**: JSON fixture files for test data
82+
- Test naming convention: `{FeatureName}Tests.swift`
83+
- Test data fixtures: Predefined JSON files with sample configurations
84+
85+
## Coding Standards
86+
87+
### Style Guide
88+
We follow the [Ray Wenderlich Swift Style Guide](https://github.com/raywenderlich/swift-style-guide) for readability and consistency.
89+
90+
### Linting
91+
SwiftLint is enforced. Before committing:
92+
```bash
93+
swiftlint
94+
```
95+
Fix all warnings and errors. Configuration in `.swiftlint.yml`.
96+
97+
### Common Patterns
98+
99+
#### Protocol-Oriented Design
100+
The SDK uses protocols for extensibility:
101+
- `OPTLogger`: Custom logging
102+
- `OPTEventDispatcher`: Custom event dispatching
103+
- `OPTDatafileHandler`: Custom datafile management
104+
- `OPTUserProfileService`: Custom user profile persistence
105+
106+
#### Thread Safety
107+
- Use `AtomicProperty`, `AtomicArray`, `AtomicDictionary` for thread-safe state
108+
- All atomic utilities are located in `Sources/Utils/`
109+
- Ensure event dispatchers and managers are thread-safe
110+
111+
#### Error Handling
112+
- Use `OptimizelyError` enum for SDK-specific errors
113+
- Use `OptimizelyResult<T>` for result types
114+
- Handle errors gracefully with meaningful messages
115+
116+
#### Logging
117+
- Use `ThreadSafeLogger` or custom logger implementing `OPTLogger`
118+
- Log levels: debug, info, warning, error
119+
- Use appropriate log levels for different message types
120+
121+
## Testing
122+
123+
### Running Tests
124+
125+
#### Using Swift Package Manager
126+
```bash
127+
# Run all tests
128+
swift test
129+
130+
# Run with verbose output
131+
swift test --verbose
132+
```
133+
134+
#### Using Xcode
135+
```bash
136+
# Run all tests for iOS
137+
xcodebuild test \
138+
-workspace OptimizelySwiftSDK.xcworkspace \
139+
-scheme OptimizelySwiftSDK-iOS \
140+
-destination 'platform=iOS Simulator,name=iPhone 16'
141+
142+
# Run a specific test target
143+
xcodebuild test \
144+
-workspace OptimizelySwiftSDK.xcworkspace \
145+
-scheme OptimizelySwiftSDK-iOS \
146+
-destination 'platform=iOS Simulator,name=iPhone 16' \
147+
-only-testing:TestTarget/TestClass
148+
149+
# Run a specific test method
150+
xcodebuild test \
151+
-workspace OptimizelySwiftSDK.xcworkspace \
152+
-scheme OptimizelySwiftSDK-iOS \
153+
-destination 'platform=iOS Simulator,name=iPhone 16' \
154+
-only-testing:TestTarget/TestClass/testMethodName
155+
```
156+
157+
### Test Targets
158+
- `OptimizelyTests-Common-iOS`: Common utilities and core functionality
159+
- `OptimizelyTests-APIs-iOS`: Public API tests
160+
- `OptimizelyTests-Batch-iOS`: Event batching and dispatching
161+
- `OptimizelyTests-DataModel-iOS`: Data models
162+
- `OptimizelyTests-Legacy-iOS`: Legacy compatibility
163+
- `OptimizelyTests-MultiClients-iOS`: Multi-client scenarios
164+
- `OptimizelyTests-Others-iOS`: Miscellaneous tests
165+
- `OptimizelyTests-iOS`: Main test suite
166+
167+
Similar test targets exist for tvOS and other platforms.
168+
169+
### Testing Best Practices
170+
- All code must have test coverage
171+
- Use XCTest framework
172+
- Use `.sortedKeys` for JSONEncoder in tests to ensure deterministic JSON output
173+
- Override network calls in test mocks to avoid timeouts
174+
- Use JSON fixtures from `Tests/TestData/` for consistent test data
175+
- Each test should use unique file names for persistent storage
176+
177+
## Development Workflow
178+
179+
### Branch Strategy
180+
- Main branch: `master`
181+
- Create feature branches: `YOUR_NAME/branch_name`
182+
- Don't commit on master branch, create new branch before committing any changes
183+
184+
### Making Changes
185+
186+
1. **Create a branch** (see "Helpful Commands > Git Commands")
187+
2. **Make your changes** following coding standards
188+
3. **Write or update tests** (see "Testing" section)
189+
4. **Run linting** (see "Coding Standards > Linting")
190+
5. **Run tests** to verify changes (see "Testing" section)
191+
192+
### Pull Request Process
193+
194+
When creating a pull request, follow this checklist:
195+
196+
1. **Ensure all tests pass** (see "Testing" section)
197+
2. **Run SwiftLint and fix all issues** (see "Coding Standards > Linting")
198+
3. **Verify no merge conflicts with `master`**
199+
```bash
200+
git fetch origin
201+
git merge origin/master
202+
```
203+
204+
4. **Follow the PR template** (located at `pull_request_template.md`)
205+
206+
Your PR description MUST include:
207+
208+
**## Summary**
209+
- Bullet points describing "what" changed (each logical change)
210+
- Context explaining "why" the changes were made
211+
212+
**## Test plan**
213+
- Describe how the changes were tested
214+
- List specific test cases added or modified
215+
- Include manual testing steps if applicable
216+
217+
**## Issues**
218+
- Reference related issues: "THING-1234" or "Fixes #123"
219+
- If no issue exists, explain why the change is needed
220+
221+
Example:
222+
```markdown
223+
## Summary
224+
- Fixed flaky tests by replacing asyncAfter with DispatchGroup.wait()
225+
- Updated MockCmabService to override both sync and async methods
226+
227+
Recent async retry refactoring introduced timing issues in tests. Tests
228+
were using unreliable asyncAfter delays instead of proper synchronization.
229+
230+
## Test plan
231+
- Ran EventDispatcherRetryTests suite 20 times, all passed
232+
- Verified on GitHub CI across multiple Xcode versions
233+
- Added capturedOptions array to MockCmabService for thread-safe tracking
234+
235+
## Issues
236+
- Fixes #456
237+
```
238+
239+
5. **Get review from maintainer**
240+
- Request review from code owners
241+
- Address all feedback and comments
242+
243+
6. **Don't update SDK version**
244+
- Version updates are handled by maintainers during release process
245+
246+
## Key APIs & Usage
247+
248+
### Initialization
249+
Initialize the SDK with an SDK key and start fetching the datafile:
250+
```swift
251+
let optimizely = OptimizelyClient(sdkKey: "YOUR_SDK_KEY")
252+
optimizely.start { result in
253+
switch result {
254+
case .success:
255+
// SDK ready
256+
case .failure(let error):
257+
// Handle error
258+
}
259+
}
260+
```
261+
262+
### Decision Making
263+
Create a user context and make feature flag decisions:
264+
```swift
265+
let user = optimizely.createUserContext(userId: "user123")
266+
let decision = user.decide(key: "feature_key")
267+
if decision.enabled {
268+
// Feature is enabled
269+
}
270+
```
271+
272+
### Event Tracking
273+
Track custom events for analytics:
274+
```swift
275+
try optimizely.track(eventKey: "purchase", userId: "user123")
276+
```
277+
278+
## Helpful Commands
279+
280+
### Finding Files
281+
```bash
282+
# Find implementation files by pattern
283+
find Sources -name "*ClassName*.swift"
284+
285+
# Find test files by pattern
286+
find Tests -name "*TestName*.swift"
287+
288+
# List all files in a specific module
289+
find Sources/ModuleName -name "*.swift"
290+
```
291+
292+
### Searching Code
293+
```bash
294+
# Find protocol definitions
295+
grep -r "^protocol" Sources/ --include="*.swift"
296+
297+
# Search for specific functions or classes
298+
grep -r "class ClassName" Sources/ --include="*.swift"
299+
grep -r "func functionName" Sources/ --include="*.swift"
300+
301+
# Find TODO or FIXME comments
302+
grep -r "TODO\|FIXME" Sources/ --include="*.swift"
303+
```
304+
305+
### Git Commands
306+
```bash
307+
# Create a new branch
308+
git checkout -b YOUR_NAME/feature-name
309+
310+
# View recent commits
311+
git log --oneline -10
312+
313+
# Check what changed in a specific commit
314+
git show <commit-hash>
315+
316+
# View file changes
317+
git diff <file-path>
318+
319+
# Fetch and merge latest from master
320+
git fetch origin
321+
git merge origin/master
322+
```
323+
324+
### Xcode
325+
```bash
326+
# List available simulators
327+
xcrun simctl list devices available
328+
329+
# List schemes and build targets
330+
xcodebuild -workspace OptimizelySwiftSDK.xcworkspace -list
331+
332+
# Show build settings for a scheme
333+
xcodebuild -workspace OptimizelySwiftSDK.xcworkspace \
334+
-scheme OptimizelySwiftSDK-iOS -showBuildSettings
335+
```

‎Sources/Extensions/ArrayEventForDispatch+Extension.swift‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,11 @@ extension Array where Element == EventForDispatch {
113113
enrichDecisions: true,
114114
region: base.region)
115115

116-
guard let data = try? JSONEncoder().encode(batchEvent) else {
116+
let encoder = JSONEncoder()
117+
if #available(iOS 11.0, tvOS 11.0, watchOS 4.0, macOS 10.13, *) {
118+
encoder.outputFormatting = .sortedKeys
119+
}
120+
guard let data = try? encoder.encode(batchEvent) else {
117121
return nil
118122
}
119123

‎Sources/Utils/Utils.swift‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,13 @@ class Utils {
7979
#endif
8080
}
8181

82-
private static let jsonEncoder = JSONEncoder()
82+
private static let jsonEncoder: JSONEncoder = {
83+
let encoder = JSONEncoder()
84+
if #available(iOS 11.0, tvOS 11.0, watchOS 4.0, macOS 10.13, *) {
85+
encoder.outputFormatting = .sortedKeys
86+
}
87+
return encoder
88+
}()
8389

8490
// @objc NSNumber can be casted either Bool, Int, or Double
8591
// more filtering required to avoid NSNumber(false, true) interpreted as Int(0, 1) instead of Bool

0 commit comments

Comments
 (0)