Skip to content

Commit a8a5ce3

Browse files
giovaborgognoclaude
andcommitted
chore: mark 9.5 ordering as done in pending items
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent ad83f88 commit a8a5ce3

File tree

1 file changed

+10
-36
lines changed

1 file changed

+10
-36
lines changed

.claude/projects/-Users-terac-repos-trigger-dev/memory/pubsub-pending.md

Lines changed: 10 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -38,47 +38,21 @@ Items identified during post-implementation audit. Ordered by priority.
3838
- SDK docs in `rules/` directory
3939
- Update `.claude/skills/trigger-dev-tasks/SKILL.md`
4040

41-
### 9.5 — Ordering Key Does Not Guarantee Strict Ordering
41+
### 9.5 — Ordering Key with Per-Key Serialization + Global Concurrency Limit
4242

43-
**Status**: NOT RESOLVED — needs design decision
44-
**Priority**: HIGH — correctness issue
45-
**Found during**: E2E testing (2026-03-01)
46-
47-
**Problem**: `orderingKey` maps to Trigger.dev's `concurrencyKey`, which creates a **copy of the queue per key**, each with the same `concurrencyLimit`. This means:
48-
49-
- If task has `concurrencyLimit: 1` → ordering works per key, BUT the limit is per-key, not global. All different keys run in parallel with no global cap (only bounded by environment concurrency limit).
50-
- If task has `concurrencyLimit: 10` → 10 events with the SAME key can run in parallel, breaking ordering.
51-
- There's no way to express "strict ordering per key + global concurrency limit N" with Trigger.dev's current queue model.
43+
**Status**: DONE (commit `ad83f88d3`)
5244

53-
**Expected behavior** (like Kafka/SQS FIFO):
54-
- `orderingKey` = strict sequential per key (always 1 at a time per key)
55-
- `concurrencyLimit` = total parallel runs across all keys (separate concept)
56-
57-
```
58-
concurrencyLimit: 3, ordering keys A/B/C:
45+
**Solution**: Added `globalConcurrencyLimit` to the run engine (new Redis keys `gcl`/`gcc`). Modified 4 Lua scripts (dequeue, release, enqueue, enqueueWithTtl) to check global limit when set. PublishEventService overrides queue to `evt-order:{eventSlug}` with per-key limit=1 and global limit=N.
5946

60-
Slot 1: A1 → A2 → A3 (key A in order)
61-
Slot 2: B1 → B2 (key B in order)
62-
Slot 3: C1 → C2 (key C in order)
63-
Max 3 running at once, each key strictly ordered.
47+
SDK usage:
48+
```typescript
49+
event({
50+
id: "order.created",
51+
ordering: { concurrencyLimit: 5 }, // max 5 keys in parallel, strict per-key ordering
52+
});
6453
```
6554

66-
**Trigger.dev's actual behavior with concurrencyKey**:
67-
- Creates 3 separate queues (A, B, C), EACH with concurrencyLimit 3
68-
- So 9 runs could execute simultaneously (3 per key × 3 keys)
69-
- Not true ordering
70-
71-
**Options to resolve**:
72-
1. Build ordering on top of Trigger.dev's queue system with custom logic in PublishEventService
73-
2. Contribute ordering support upstream to Trigger.dev's run engine
74-
3. Document as limitation and recommend `concurrencyLimit: 1` for ordering use cases
75-
4. Use a separate ordering mechanism (Redis-based FIFO per key) before triggering runs
76-
77-
**Test results that confirmed this**:
78-
- `concurrencyLimit: 1` + same key → sequential (correct)
79-
- `concurrencyLimit: 1` + different keys → parallel (capped by env limit ~8, not by concurrencyLimit)
80-
- `concurrencyLimit: 2` + same key → 2 at a time (breaks ordering)
81-
- 10 different keys + `concurrencyLimit: 1` → only ~8 ran in parallel (env limit, not queue limit)
55+
**Needs E2E verification** with live hello-world project to confirm behavior.
8256

8357
### 9.6 — Large Payloads >512KB Return 0 Runs (Silent Partial Failure)
8458

0 commit comments

Comments
 (0)