You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
if (System.getenv("RUN_DEVIATIONS") !=null) return@runTest
362
+
if (System.getenv("RUN_DEVIATIONS") ==null) return@runTest
336
363
337
364
// ... spec-correct setup and assertions ...
338
365
}
@@ -354,3 +381,39 @@ Append to `uts/src/test/kotlin/io/ably/lib/deviations.md`. Each entry needs:
354
381
2. What the spec says
355
382
3. What the SDK does
356
383
4. Which test is affected and how it was adapted
384
+
385
+
---
386
+
387
+
## Step 7 — Review generated output against the spec
388
+
389
+
Re-read the original spec file and the generated Kotlin test file side-by-side and check every item below. Fix anything that fails a check before declaring the task done.
390
+
391
+
### Coverage check — every test case is present
392
+
393
+
For each test case ID in the spec:
394
+
-[ ] A `@Test` method exists with that ID in its `@UTS` KDoc tag
395
+
-[ ] The method name contains the spec ID and a meaningful description
396
+
397
+
Flag any spec test case that has no corresponding `@Test` method as **missing** and implement it.
398
+
399
+
### Assertion completeness — every `ASSERT` / `AWAIT` is translated
400
+
401
+
For each `ASSERT`, `AWAIT`, or observable outcome stated in the spec pseudocode:
402
+
-[ ] There is a direct Kotlin assertion (`assertEquals`, `assertNotNull`, `assertNull`, `assertIs`, `assertTrue`, `assertFailsWith`, etc.) or an `awaitState` / `awaitChannelState` call covering it
403
+
-[ ] No spec assertion has been silently dropped or weakened (e.g. replaced with a comment)
404
+
405
+
### Setup fidelity — preconditions match the spec
406
+
407
+
For each test case, verify:
408
+
-[ ] Client options (`autoConnect`, `recover`, timeouts, etc.) match the spec's `ClientOptions`
409
+
-[ ] Mock responses (success / refused / timeout / DNS error / custom messages) match the spec's prescribed network behaviour
410
+
-[ ] Timer setup (`enableFakeTimers`, `fakeClock.advance(...)`) matches every `enable_fake_timers` / `ADVANCE_TIME` in the spec
411
+
-[ ] Channel operations (attach, detach, publish) are performed in the order the spec requires
412
+
413
+
### Deviation honesty
414
+
415
+
For any place where the generated test diverges from the spec pseudocode (adapted assertion, env-gated skip, or omitted step):
416
+
-[ ] A `// DEVIATION:` comment explains why
417
+
-[ ] The deviation is recorded in `uts/src/test/kotlin/io/ably/lib/deviations.md`
418
+
419
+
If you find gaps during this review, fix them and re-run Steps 5–6 before finishing.
0 commit comments