What
Investigate adding an optional historical-timestamp argument (or a sibling method) to TID::generate() so that backfilled records can carry rkeys that reflect the original publish time rather than the time of the backfill run.
This is paired with the WP-CLI backfill request (issue 88) — the CLI is what makes large-scale historical backfills practical, and once you're backfilling thousands of posts, having every rkey clustered around "now" stops being a curiosity and starts being a real ordering problem.
Why
Today every transformer's get_rkey() calls TID::generate(), which uses microtime(true). The record's createdAt field is correct (it already reads post_date_gmt in class-post.php:171, class-comment.php:85, etc.), so Bluesky and standard.site clients display the right date. But the rkey itself encodes "now" — so a 2019 post backfilled in 2026 lives in the repo at a 2026 TID.
For anything that reads the repo in TID order (firehose consumers, repo browsers, archival tooling, ozone, anything that assumes "later TID == later content") this misorders the whole historical catalog. The transformer already knows the original publish time; the TID just doesn't use it.
How (investigation)
A handful of things to work through before this is implementable:
-
API shape. Two reasonable options:
- Overload:
TID::generate( ?int $microseconds = null ): string — null preserves the current behavior, an explicit value mints a historical TID.
- Sibling method:
TID::generate_for_time( DateTimeInterface $when ): string — louder at call sites, doesn't need a flag to disable the monotonic floor.
I lean sibling, because the semantics of the two paths diverge enough (see point 2) that a single signature would lie about what it does.
-
Monotonic floor. The persisted atmosphere_tid_last_ts option is a forward floor for current-time TIDs (the CAS dance in class-tid.php exists specifically to keep two PHP-FPM workers from regressing it). A historical timestamp will always be below that floor, so the existing if ( $ts <= $floor ) { $ts = $floor + 1; } snaps it right back to "now," defeating the entire change. Historical writes have to skip the floor check entirely and — equally important — must not update the floor on the way out. The floor is for live publishing only.
-
Collision within a second. post_date_gmt is MySQL datetime, so resolution is seconds. Two posts published in the same second will hash to identical microseconds. That leaves only the 10-bit $clock_id (1024 values) to disambiguate, and within a single backfill run the $clock_id is fixed for the process — so two same-second posts in one batch produce identical TIDs. Options to fix:
- Spread same-second posts deterministically across microseconds by post ID (
$seconds * 1_000_000 + ( $post_id % 1_000_000 )).
- Maintain a per-second counter for the duration of a batch.
- Catch the AT Protocol write failure and retry with
microtime(true) — but that throws away the whole point for the retried record.
Probably worth seeding the microsecond portion from the post ID so the encoding is deterministic and re-running the backfill mints the same TID.
-
rkey uniqueness against existing records. AT Protocol enforces unique rkey per (repo, collection). If a backdated TID happens to collide with a record already in the PDS, applyWrites fails. Need either a pre-check, or a catch-and-retry path.
-
AT Protocol semantics. The spec describes TIDs as timestamp-based but doesn't require commit-order monotonicity at the record level — historical TIDs should be legal. Worth confirming in the spec and noting any client assumptions (firehose consumers especially) that might break.
-
Caller changes. Five TID::generate() call sites need to opt in:
includes/transformer/class-post.php:280includes/transformer/class-document.php:202includes/transformer/class-publication.php:94includes/transformer/class-comment.php:142includes/class-publisher.php:377 (the reply rkey — probably stays as "now," since replies aren't part of the backfill flow)
-
Tests. New cases for the historical-time path: deterministic encoding of a known instant, no floor regression after a historical call, collision behavior for same-second post pairs. Existing monotonicity tests stay scoped to the no-argument path.
What
Investigate adding an optional historical-timestamp argument (or a sibling method) to
TID::generate()so that backfilled records can carry rkeys that reflect the original publish time rather than the time of the backfill run.This is paired with the WP-CLI backfill request (issue 88) — the CLI is what makes large-scale historical backfills practical, and once you're backfilling thousands of posts, having every rkey clustered around "now" stops being a curiosity and starts being a real ordering problem.
Why
Today every transformer's
get_rkey()callsTID::generate(), which usesmicrotime(true). The record'screatedAtfield is correct (it already readspost_date_gmtinclass-post.php:171,class-comment.php:85, etc.), so Bluesky and standard.site clients display the right date. But the rkey itself encodes "now" — so a 2019 post backfilled in 2026 lives in the repo at a 2026 TID.For anything that reads the repo in TID order (firehose consumers, repo browsers, archival tooling, ozone, anything that assumes "later TID == later content") this misorders the whole historical catalog. The transformer already knows the original publish time; the TID just doesn't use it.
How (investigation)
A handful of things to work through before this is implementable:
API shape. Two reasonable options:
TID::generate( ?int $microseconds = null ): string— null preserves the current behavior, an explicit value mints a historical TID.TID::generate_for_time( DateTimeInterface $when ): string— louder at call sites, doesn't need a flag to disable the monotonic floor.I lean sibling, because the semantics of the two paths diverge enough (see point 2) that a single signature would lie about what it does.
Monotonic floor. The persisted
atmosphere_tid_last_tsoption is a forward floor for current-time TIDs (the CAS dance inclass-tid.phpexists specifically to keep two PHP-FPM workers from regressing it). A historical timestamp will always be below that floor, so the existingif ( $ts <= $floor ) { $ts = $floor + 1; }snaps it right back to "now," defeating the entire change. Historical writes have to skip the floor check entirely and — equally important — must not update the floor on the way out. The floor is for live publishing only.Collision within a second.
post_date_gmtis MySQL datetime, so resolution is seconds. Two posts published in the same second will hash to identical microseconds. That leaves only the 10-bit$clock_id(1024 values) to disambiguate, and within a single backfill run the$clock_idis fixed for the process — so two same-second posts in one batch produce identical TIDs. Options to fix:$seconds * 1_000_000 + ( $post_id % 1_000_000 )).microtime(true)— but that throws away the whole point for the retried record.Probably worth seeding the microsecond portion from the post ID so the encoding is deterministic and re-running the backfill mints the same TID.
rkey uniqueness against existing records. AT Protocol enforces unique rkey per
(repo, collection). If a backdated TID happens to collide with a record already in the PDS,applyWritesfails. Need either a pre-check, or a catch-and-retry path.AT Protocol semantics. The spec describes TIDs as timestamp-based but doesn't require commit-order monotonicity at the record level — historical TIDs should be legal. Worth confirming in the spec and noting any client assumptions (firehose consumers especially) that might break.
Caller changes. Five
TID::generate()call sites need to opt in:includes/transformer/class-post.php:280includes/transformer/class-document.php:202includes/transformer/class-publication.php:94includes/transformer/class-comment.php:142includes/class-publisher.php:377(the reply rkey — probably stays as "now," since replies aren't part of the backfill flow)Tests. New cases for the historical-time path: deterministic encoding of a known instant, no floor regression after a historical call, collision behavior for same-second post pairs. Existing monotonicity tests stay scoped to the no-argument path.