You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
### 9.5 — Ordering Key Does Not Guarantee Strict Ordering
41
+
### 9.5 — Ordering Key with Per-Key Serialization + Global Concurrency Limit
42
42
43
-
**Status**: NOT RESOLVED — needs design decision
44
-
**Priority**: HIGH — correctness issue
45
-
**Found during**: E2E testing (2026-03-01)
46
-
47
-
**Problem**: `orderingKey` maps to Trigger.dev's `concurrencyKey`, which creates a **copy of the queue per key**, each with the same `concurrencyLimit`. This means:
48
-
49
-
- If task has `concurrencyLimit: 1` → ordering works per key, BUT the limit is per-key, not global. All different keys run in parallel with no global cap (only bounded by environment concurrency limit).
50
-
- If task has `concurrencyLimit: 10` → 10 events with the SAME key can run in parallel, breaking ordering.
51
-
- There's no way to express "strict ordering per key + global concurrency limit N" with Trigger.dev's current queue model.
43
+
**Status**: DONE (commit `ad83f88d3`)
52
44
53
-
**Expected behavior** (like Kafka/SQS FIFO):
54
-
-`orderingKey` = strict sequential per key (always 1 at a time per key)
55
-
-`concurrencyLimit` = total parallel runs across all keys (separate concept)
56
-
57
-
```
58
-
concurrencyLimit: 3, ordering keys A/B/C:
45
+
**Solution**: Added `globalConcurrencyLimit` to the run engine (new Redis keys `gcl`/`gcc`). Modified 4 Lua scripts (dequeue, release, enqueue, enqueueWithTtl) to check global limit when set. PublishEventService overrides queue to `evt-order:{eventSlug}` with per-key limit=1 and global limit=N.
59
46
60
-
Slot 1: A1 → A2 → A3 (key A in order)
61
-
Slot 2: B1 → B2 (key B in order)
62
-
Slot 3: C1 → C2 (key C in order)
63
-
Max 3 running at once, each key strictly ordered.
47
+
SDK usage:
48
+
```typescript
49
+
event({
50
+
id: "order.created",
51
+
ordering: { concurrencyLimit: 5 }, // max 5 keys in parallel, strict per-key ordering
52
+
});
64
53
```
65
54
66
-
**Trigger.dev's actual behavior with concurrencyKey**:
67
-
- Creates 3 separate queues (A, B, C), EACH with concurrencyLimit 3
68
-
- So 9 runs could execute simultaneously (3 per key × 3 keys)
69
-
- Not true ordering
70
-
71
-
**Options to resolve**:
72
-
1. Build ordering on top of Trigger.dev's queue system with custom logic in PublishEventService
73
-
2. Contribute ordering support upstream to Trigger.dev's run engine
74
-
3. Document as limitation and recommend `concurrencyLimit: 1` for ordering use cases
75
-
4. Use a separate ordering mechanism (Redis-based FIFO per key) before triggering runs
76
-
77
-
**Test results that confirmed this**:
78
-
-`concurrencyLimit: 1` + same key → sequential (correct)
79
-
-`concurrencyLimit: 1` + different keys → parallel (capped by env limit ~8, not by concurrencyLimit)
80
-
-`concurrencyLimit: 2` + same key → 2 at a time (breaks ordering)
81
-
- 10 different keys + `concurrencyLimit: 1` → only ~8 ran in parallel (env limit, not queue limit)
55
+
**Needs E2E verification** with live hello-world project to confirm behavior.
0 commit comments