[mirror] Threadiverse tutorial and public audience interop fix#4
Open
yashwant86 wants to merge 43 commits intomm-base-710from
Open
[mirror] Threadiverse tutorial and public audience interop fix#4yashwant86 wants to merge 43 commits intomm-base-710from
yashwant86 wants to merge 43 commits intomm-base-710from
Conversation
Add docs/tutorial/threadiverse.md with the Introduction, Target audience, Goals, and Setting up the development environment sections, and register it in the tutorial nav. This first chapter walks the reader through `fedify init -w next -p npm -k in-memory -m in-process`, the Next.js 15.5.x pin workaround, and verifying the dev server and federation middleware with `fedify lookup`. Subsequent commits will add chapters in step with commits in the paired fedify-dev/threadiverse example repository. Assisted-By: Claude Code:claude-opus-4-7
Add the *Swapping ESLint for Biome* subsection to the Setup chapter. It walks the reader through deleting the two ESLint configs that `fedify init` leaves behind for Next.js, turning on Biome's linter, rewriting the `lint` and `format` scripts to call Biome, and cleaning up the unused `getLogger` call that `noUnusedVariables` flags in *federation/index.ts*. Matches commit 839def1 in fedify-dev/threadiverse. Assisted-By: Claude Code:claude-opus-4-7
Add the *Layout and navigation* chapter. It walks the reader through replacing `create-next-app`'s demo landing page with a minimal root layout (top nav with brand + Home/New community links), a one-paragraph home page, and an ~80-line plain stylesheet for typography, forms, cards, buttons, and the nested reply tree. The reader copies the stylesheet once and never touches CSS again, which keeps later chapters focused on federation rather than presentation. Matches commit 4d31dab in fedify-dev/threadiverse. Assisted-By: Claude Code:claude-opus-4-7
Add the opening of the *User accounts* chapter. It introduces Drizzle ORM and SQLite, walks through installing `drizzle-orm`, `better-sqlite3`, and the dev-time `drizzle-kit` + `@types/better-sqlite3`, and declares the first table (`users`) in *db/schema.ts* with `id`, `username`, `password_hash`, and `created_at` columns. Also covers opening the database with WAL + foreign-key pragmas, wiring up *drizzle.config.ts*, adding `db:push` and `db:studio` scripts, and gitignoring the SQLite files. Matches commit d3aa008 in fedify-dev/threadiverse. Assisted-By: Claude Code:claude-opus-4-7
Add the *Signup form and password hashing* subsection. It introduces Node's scrypt as the password hashing primitive, shows how to build a Next.js App Router *server action* (a server function called directly from an HTML form via the `action` attribute, with no client fetch or API route), and walks the reader through the full validation chain: client-side HTML pattern, server-side regex re-validation, Drizzle duplicate-username lookup, scrypt hash, and a redirect back to the login page with a success message. Includes a screenshot of the rendered form for visual confirmation. Matches commit b170aa5 in fedify-dev/threadiverse. Assisted-By: Claude Code:claude-opus-4-7
Add the *Login and sessions* subsection. It introduces the server-side `sessions` table (indexed by an opaque random token, with `onDelete: "cascade"` on the user reference), walks through writing `createSession`, `getCurrentUser`, and `destroySession` helpers in *lib/session.ts*, and discusses each cookie flag (httpOnly, sameSite, secure in production only, 30-day maxAge). It then builds the login page + login/logout server actions and turns the root layout into an async server component so every page can branch on whether a user is signed in, showing `@username / Log out` or `Log in / Sign up` in the nav accordingly. Includes a screenshot of the logged-in home page. Matches commit b5e8ba7 in fedify-dev/threadiverse. Assisted-By: Claude Code:claude-opus-4-7
Add the *Profile page* subsection. It introduces Next.js App Router dynamic segments (`[username]`), server-side `params` as a Promise, and `notFound()` for 404 handling, then builds a simple profile page that renders the user's handle, join date, and a placeholder for future threads. Also updates the nav-bar `@username` label into a link pointing at that page. Ends with a teaser: the same URL currently serves HTML to browsers and a placeholder `Person` JSON to ActivityPub clients, and the next commit will swap the placeholder for a dispatcher backed by the `users` table. Matches commit a0ed553 in fedify-dev/threadiverse. Assisted-By: Claude Code:claude-opus-4-7
Add the *Federating your user: the Person actor* chapter. It walks
the reader through:
- The `keys` table, keyed by actor identifier so the same table
serves both user and community keys when we add `Group` actors.
- A real `setActorDispatcher` that looks the identifier up in
`users` and returns a `Person` with `inbox`, shared-inbox
endpoints, `publicKey`, and `assertionMethods` populated from
`ctx.getActorKeyPairs`.
- `setKeyPairsDispatcher` that lazily generates and persists both
RSA and Ed25519 key pairs as JWK JSON.
- A minimal `setInboxListeners` call (URL templates only; handlers
come later) so `ctx.getInboxUri` resolves.
- Local verification with `fedify lookup` and a direct WebFinger
`curl`.
- Running `fedify tunnel`, why `x-forwarded-fetch` is needed when
the app is behind a tunnel, and how to rewrite *middleware.ts*
with `getXForwardedRequest`.
- A screenshot of ActivityPub.Academy finding the local actor via
WebFinger through the tunnel URL.
Matches commits ad9ae55 ("Actor dispatcher and key pairs") and
4d462c1 ("Use x-forwarded-fetch in middleware") in
fedify-dev/threadiverse.
Assisted-By: Claude Code:claude-opus-4-7
Add the *Communities as Group actors* chapter. It covers four
example-repo commits' worth of content in one coherent narrative:
- Declaring the `communities` table and lifting the identifier
uniqueness check into a shared `lib/identifiers.ts` so signup and
community creation both go through `isIdentifierTaken`.
- The community creation UI and the server action that inserts a
new row under the logged-in user.
- Teaching the existing profile page to fall through from `users`
to `communities`, with screenshots of the filled form and the
rendered community page.
- Extending the actor dispatcher (and key pairs dispatcher) so the
same `/users/{identifier}` URL returns a `Group` for community
slugs and a `Person` for usernames, plus a screenshot of
ActivityPub.Academy resolving `@<slug>@<host>` via WebFinger.
Matches commits 8200730 ("communities table"), 46c628b ("Community
creation form"), 32e75bf ("Community page"), and 6de227a ("Group
actor dispatcher") in fedify-dev/threadiverse.
Assisted-By: Claude Code:claude-opus-4-7
Add the *Subscribing to communities* chapter. It walks through:
- Declaring a polymorphic `follows` table that carries the
follower's inbox and optional shared inbox inline, plus a
unique `(follower_uri, followed_uri)` index.
- Wiring `followers` into the `Group` actor and implementing
`setFollowersDispatcher` so remote threadiverse servers can
paginate subscribers.
- Inbound `Follow` handling: validate, fetch the follower actor,
upsert an accepted row, and ship `Accept(Follow)` back.
- Inbound `Accept(Follow)` handling: unwrap the enclosed Follow
and flip the matching outbound row to accepted.
- Outbound follow UI: `/follow` form, `followCommunity` server
action using `ctx.lookupObject`, and a `currentOrigin` helper
that builds the correct origin from `x-forwarded-host`.
- Verification with a screenshot of ActivityPub.Academy listing
the local community with an "Unfollow" button and a follower
count of 1 after the full round trip.
Matches commits 8a8cb7d ("follows table and followers dispatcher")
and f4162ad ("Follow and Accept(Follow)") in
fedify-dev/threadiverse.
Assisted-By: Claude Code:claude-opus-4-7
Add the *Threads* chapter covering five example-repo commits:
- `threads` table keyed by ActivityPub URI with a UNIQUE constraint
so `onConflictDoNothing()` on inserts is idempotent.
- Per-community creation form scoped via `createThread.bind(null,
slug)`, plus community-page thread list + *Start a thread* CTA.
- Server action that inserts the thread row optimistically, then
federates a `Create(Page)` with the threadiverse-standard
audience addressing (`audience: community`, `to: community`,
`cc: [PUBLIC, community/followers]`).
- Community inbox `Create` handler: stores the thread idempotently,
then `sendActivity(..., "followers", Announce(Create))` with
`preferSharedInbox` so local community posts fan out to
every subscriber.
- Follower-side `Announce` handler that `routeActivity`s the
enclosed Create back through the same Create handler, verifying
the enclosed activity's origin in the process.
A TIP at the end explains that Mastodon (Academy) doesn't render
`Page` objects in its timeline even though Fedify's Announce lands;
Lemmy/Mbin/NodeBB, the intended consumers, do.
Matches commits e1feb72 ("threads table"), 191379e ("Thread
creation form"), 415bdce ("Publish Create(Page) to the community"),
1d623cb ("Community inbox Create handler + Announce"), and 9b5f149
("Announce handler routes to Create") in fedify-dev/threadiverse.
Assisted-By: Claude Code:claude-opus-4-7
Add the *Replies* chapter walking through three example-repo commits:
- Thread detail page at `/users/<slug>/threads/<id>` that
re-derives the canonical thread URI from `currentOrigin()` and
the route params.
- `replies` table parallel to `threads` (URI-keyed, carries
`thread_uri`, `parent_uri`, and `community_uri`) plus the
reply tree UI (recursive `<ul class="reply-tree">` with
per-node inline reply forms wrapped in `<details>`).
- Extension of the community inbox `Create` handler for `Note`
objects: resolve the parent via both `threads` and `replies`,
derive the top-level `thread_uri`, insert with
`onConflictDoNothing`, then Announce to followers through the
same local-community branch that handles threads.
Matches commits 8c46ff9 ("Thread page"), f7a5b9d ("replies table +
thread-page reply UI"), and a93ed2b ("Community Announce of
Create(Note)") in fedify-dev/threadiverse.
Assisted-By: Claude Code:claude-opus-4-7
Add the *Votes* chapter pairing with the two vote example-repo
commits: Like/Dislike UI plus outbound action, and the shared
`handleVote` inbox handler that upserts the vote row and
re-Announces to the community's followers.
The chapter walks through the `(voter_uri, target_uri)` unique
index and why an upsert on that pair replaces the previous vote
instead of stacking rows, the `Map<targetUri, VoteTally>` approach
that bundles all the tallies for a thread page into one `inArray`
query, and the common `VoteClass = kind === "Like" ? Like : Dislike`
construction pattern for activity types with identical field shape.
Ends with a TIP showing the single SQL expression
(`SUM(CASE kind WHEN 'Like' THEN 1 ELSE -1 END)`) that threadiverse
platforms use for ranking, and the rationale for keeping per-vote
rows instead of pre-aggregating counters.
Matches commits 7136e54 ("votes table + Like/Dislike UI") and
aa00671 ("Community Announce of Like / Dislike") in
fedify-dev/threadiverse.
Assisted-By: Claude Code:claude-opus-4-7
Close the local-user loop with the two remaining user-facing
features.
- app/page.tsx becomes the subscribed feed: anonymous visitors
see the welcome blurb, logged-in users see every accepted
subscription listed above a chronological thread list pulled
via inArray(threads.communityUri, subscribedUris).
- Unsubscribing is Ch. 14 played backwards. The outbound
unfollowCommunity action wraps the original Follow's actor +
object in an Undo(Follow), sends it to the followee's inbox,
and deletes the follow row. The community inbox handler flips
on Undo, unwraps the enclosed Follow, verifies
undo.actorId === follow.actorId so nobody can Undo someone
else's subscription, and deletes the corresponding row.
Also teach .hongdown.toml to preserve the product and tool names
that show up in this tutorial (Biome, Drizzle, Drizzle Kit, Drizzle
ORM, Next.js, Node.js, SQLite, TypeScript, Mbin, Piefed, NodeBB,
cloudflared, ngrok, Serveo, Cloudflare Tunnel, x-forwarded-fetch,
create-next-app, scrypt, JSX) and fix the subsection headings that
hongdown previously lower-cased for ActivityPub activity type
names; those (Follow, Accept(Follow), Create(Page),
Create(Note), Like, Dislike, Announce, Undo, Group)
are now wrapped in backticks so hongdown leaves them alone.
Matches commits 717cad6 ("Subscribed front page") and f83fecd
("Unsubscribe with Undo(Follow)") in fedify-dev/threadiverse.
Assisted-By: Claude Code:claude-opus-4-7
Close out *docs/tutorial/threadiverse.md* with the standard two
sections the microblog tutorial uses, adapted for threadiverse
scope:
- "Areas to improve" enumerates extensions the reader can
reasonably add next: link threads, Update/Delete/Tombstone,
local communities index, ranking via the vote-sum expression
already computable from the votes table, persistent KV/MQ
via @fedify/postgres or @fedify/sqlite, PostgreSQL for
application data, and Lemmy-specific Group actor fields
(attributedTo, moderators, featured) for full Lemmy
interop.
- "Next steps" links into docs/manual/deploy.md (specifically
its canonical-origin, reverse-proxy, persistent-KV/MQ,
actor-key-lifecycle, and running-the-process sections), the
@fedify/next README for deployment caveats, and the Fedify
community channels.
Matches commits 0ed0741 ("Advertise outbox URL on actors") and
2cad236 ("Front matter") in fedify-dev/threadiverse.
Assisted-By: Claude Code:claude-opus-4-7
Add a new chapter between *Unsubscribing with `Undo(Follow)`* and
*Areas to improve* that covers the three field additions needed for
Lemmy to accept the community as a first-class community: a
`moderators` collection (registered through `setCollectionDispatcher`
+ returning the community creator as the sole `Person`),
a `featured` URL (via `setFeaturedDispatcher` + an empty
`OrderedCollection`), and an `attributedTo` pointing at the
moderators collection URL (Lemmy's convention for locating mods).
A NOTE callout explains Lemmy's exponential-backoff federation
retries so the reader isn't surprised by a multi-minute Subscribe
Pending state, and a TIP covers the remaining Lemmy-specific
`postingRestrictedToMods` boolean that Fedify's `Group` vocab
doesn't expose directly but that Lemmy tolerates being absent.
The entry in *Areas to improve* that previously pointed at these
fields is removed, since they're now covered in-chapter.
Matches commit df0e2a5 ("Lemmy-compatible Group actor fields") in
fedify-dev/threadiverse.
Assisted-By: Claude Code:claude-opus-4-7
Extend the *Making the community actor Lemmy-compatible* chapter with the two more fixes needed for Lemmy's actual `Subscribe` button to reach the *Joined* state, not just *Subscribe Pending*: - Bundle Lemmy's JSON-LD context (`https://join-lemmy.org/context.json`) locally and register a `documentLoaderFactory` / `contextLoaderFactory` pair that short-circuits the URL to the bundled copy. Without this, Lemmy's activities fail to parse because Lemmy serves the context as `application/json` without a JSON-LD `Link` header, and Fedify's default loader rejects those responses. - Rewrite the `Accept(Follow)` we ship to satisfy Lemmy's strict `PersonInboxActivities` enum: a UUID-based path for the `id` instead of a URL-encoded fragment, and a brand-new minimal `Follow` (`id` + `actor` + `object`) as the nested `object` instead of echoing the entire incoming Follow. Ends with a verification SQL snippet, a screenshot of Lemmy showing "Joined" after the round-trip, and a WARNING callout about a known `Announce(Create(Page))` digest-verification failure specific to Cloudflare quick tunnels re-framing the HTTP/2 body in transit — `Follow` delivery works through that tunnel but the larger Announce body can trip Lemmy's SHA-256 digest check; named Cloudflare tunnels, `serveo`, `ngrok`, and normal reverse proxies don't hit this. Matches commit 4c01faa ("Preload Lemmy's JSON-LD context and use minimal Accept(Follow)") in fedify-dev/threadiverse. Assisted-By: Claude Code:claude-opus-4-7
Revise the WARNING callout in *Making the community actor Lemmy-
compatible* to reflect what actually works against lemmy.ml today:
- Inbound `Follow`, outbound `Follow`, `Accept(Follow)`, and
`Undo(Follow)` all round-trip cleanly over both ngrok and
named Cloudflare tunnels in both directions. The earlier
wording blamed Cloudflare quick tunnels for the digest
mismatch, which turned out to be a red herring once we had
proper logging.
- The remaining blocker is Lemmy returning
`{"error":"object_is_not_public"}` on our community's fan-out
`Announce(Create(Page))`. It's a nested-object audience shape
issue, not a tunnel issue — Mastodon, Mbin, and Peertube accept
the same activity — and the fix belongs in the example repo's
open items rather than in the tutorial.
Also refresh *docs/tutorial/threadiverse/lemmy-subscribed.png* to
the ngrok-hosted community screenshot, which is cleaner and matches
the reader's expected host format.
Assisted-By: Claude Code:claude-opus-4-7
JSON-LD compaction rewrites `https://www.w3.org/ns/activitystreams#Public` to the CURIE `as:Public` (or, once `@vocab` applies, the bare term `Public`) whenever the activity's `@context` defines the `as` prefix. Strictly speaking the compact form is equivalent, but several threadiverse implementations — Lemmy among them, see [LemmyNet/lemmy#6465] — match the `to`, `cc`, `bto`, `bcc`, and `audience` fields against the full URI by string equality without running JSON-LD expansion. Outgoing `Announce(Create(Page))` from a community with Lemmy subscribers therefore gets silently rejected with `{"error":"object_is_not_public"}`. Add a post-compaction normalization pass in `FederationImpl.sendActivity()` that walks the serialized JSON-LD and replaces `as:Public` / `Public` occurrences inside those five addressing fields with the expanded URI. Other fields (and the full URI elsewhere, e.g. inside tags) are left untouched. The extra pass only runs on the direct-send path; the fan-out queue path re-deserializes through `Activity.fromJsonLd()` before reaching the same code, so it benefits from the same normalization without double-processing. Also add a regression test that sends an activity addressed to `vocab.PUBLIC_COLLECTION`, captures the posted body, and asserts the `to` field is the full URI rather than a CURIE. [LemmyNet/lemmy#6465]: LemmyNet/lemmy#6465 Assisted-By: Claude Code:claude-opus-4-7
Replace the WARNING callout in *Making the community actor Lemmy- compatible* that previously labelled outbound `Announce(Create(Page))` as blocked by Lemmy's `object_is_not_public` check. Now that Fedify 2.2 serializes the public audience as the full `https://www.w3.org/ns/activitystreams#Public` URI (and Lemmy's upstream fix in [LemmyNet/lemmy#6466] is on the way), the community's fan-out lands in subscribers' feeds on Lemmy, and replies and votes ride the same path back. Rewritten as a NOTE with the historical context preserved so readers encountering older Fedify or Lemmy versions can still diagnose the original symptom. Also bump the `fedify --version` prerequisite from 2.1.0 to 2.2.0, since that's the release that ships the sender-side workaround this tutorial now relies on for the Lemmy walkthrough. [LemmyNet/lemmy#6466]: LemmyNet/lemmy#6466 Assisted-By: Claude Code:claude-opus-4-7
Two follow-ups on the workaround added in the previous commit, both
prompted by a closer read of how the normalization interacts with
application-defined JSON-LD contexts and with Object Integrity Proof
signing:
- The unconditional CURIE rewrite was unsafe in the presence of a
custom `@context` that redefines the `as:` prefix or the bare
`Public` term. Add a URDNA2015 canonicalization pass on both
the original and the rewritten document and only emit the
rewritten form when the two produce identical N-Quads. When
the canonical forms diverge, or when canonicalization itself
errors out, the original document is returned unchanged so the
application's semantics are preserved.
- The Ed25519 `eddsa-jcs-2022` Object Integrity Proof path in
`createProof` canonicalizes the compact JSON-LD byte-for-byte
with JCS, not URDNA2015. Applying the CURIE rewrite only after
signing would therefore have invalidated the proof for every
receiver, since `verifyProof` JCS-hashes the on-wire form. Move
the normalization into `createProof` itself, before the JCS pass,
so the proof is signed over the same bytes we emit.
Extract the helper into a dedicated *packages/fedify/src/compat/public-
audience.ts* module so both `FederationImpl.sendActivity` and
`createProof` can share it without `sig/` importing from `federation/`.
Add unit tests covering the CURIE rewrite, the no-op path, the
non-addressing fields, the semantic-equivalence bailout for a
redefined `as:` context, and a full `signObject`/`verifyProof`
round-trip with `tos: [PUBLIC_COLLECTION]` to pin down the
before-JCS ordering.
Assisted-By: Claude Code:claude-opus-4-7
Polish pass on the threadiverse tutorial based on review feedback:
- Tighten intra-sentence em dashes to have no surrounding
whitespace; keep the spaced form only when separating a list-item
label from its body. Many of these read better as comma,
semicolon, or parentheses and were converted that way.
- Switch bold emphasis in list-item labels and step titles to
italics, matching the project's house style.
- Use the full names *Drizzle ORM* (never bare *Drizzle*) and
*Linked Data Signatures* (never *LD Signatures*); drop the now
unused *Drizzle* entry from the hongdown proper-nouns list.
- Add markdown-it-abbr definitions for *CURIE* and *FEP* so the
acronyms get tooltip expansions alongside the existing *JSX*,
*TSX*, and *ORM* ones.
- Link every FEP-xxxx reference to its canonical `w3id.org/fep/…`
URL.
- Spell HTTP status codes as `<code> <name>` (`404 Not Found`
instead of a bare 404), and keep header names in backticks
wherever they appear.
- Replace every *Ch. N* cross-reference with an italicised
chapter title linked to its VitePress anchor, so readers
actually have somewhere to click; chapter numbers don't show up
in the rendered TOC.
- Rework the highlight ranges on every `typescript{…}` / `tsx{…}`
code fence so the highlighted lines match what the surrounding
prose calls attention to. Several were silently off-by-one or
outright referenced lines that existed in an earlier draft of
the listing.
Assisted-By: Claude Code:claude-opus-4-7
- Iterate over own keys only (Object.keys) instead of for...in in
hasPublicCurieInAddressing and rewritePublicAudience, so enumerable
inherited properties on the incoming (potentially adversarial)
JSON-LD record are not copied into the normalized output. Addresses
fedify-dev#710 (comment)
and
fedify-dev#710 (comment).
- Add a URDNA2015 fast-path for JSON-LD documents whose @context is
composed entirely of string IRIs. In that case no inline context
entry can redefine the `as:` prefix or the bare `Public` term, so
the rewrite is provably semantics-preserving and the
canonicalization equivalence check can be skipped. Only documents
that embed an inline @context object continue to pay the
canonicalization cost. Addresses
fedify-dev#710 (comment).
- Accept either the on-wire form or the normalized form in
verifyProof(). createProof() signs the bytes produced *after*
normalization, but signObject()'s return value still serializes
back to the `as:Public` CURIE form by default, so a caller doing
an in-memory sign -> reserialize -> verify round-trip would have
seen a spurious signature mismatch. verifyProof() now tries the
input as-is first (preserving verification of signatures produced
by other implementations that signed the CURIE form) and falls
back to the normalized form when the original fails, restoring
the signObject()/verifyProof() API contract. Addresses
fedify-dev#710 (comment).
Added regression tests: the string-only-context fast path uses a
throwing contextLoader to prove URDNA2015 is not invoked; a
prototype-pollution test confirms inherited keys are not rewritten;
and the signObject() test now also verifies a proof directly from
`signed.toJsonLd({ format: "compact" })` output (which still contains
`as:Public`), exercising the verifyProof() fallback.
Assisted-By: Claude Code:claude-opus-4-7
Three follow-ups on the previous review round, all local to the
public-audience helper:
- Skip `@context` in the recursive addressing-field search and
rewrite. Addressing fields such as `to`, `cc`, `audience` never
live inside a term definition, so traversing into the context
object was wasted work on documents with large inline contexts
and risked rewriting a context term's value that happened to
equal the CURIE. Addresses
fedify-dev#710 (comment)
and
fedify-dev#710 (comment).
- Have `rewritePublicAudience()` return the same reference when a
subtree has no changes, so unchanged arrays and records are no
longer cloned on every outbound send. Addresses
fedify-dev#710 (comment).
- Tighten the URDNA2015 fast path: a string `@context` entry was
previously trusted as semantics-preserving even when the URL
pointed at a remote context that could in principle redefine
`as:` or `Public`. The fast path now requires every `@context`
entry to be drawn from Fedify's preloaded-contexts set and the
ActivityStreams URL to be present; other shapes (inline objects,
unknown external URLs, empty arrays) fall back to the
canonicalization equivalence check. Since every preloaded
context other than ActivityStreams leaves `as:` and `Public`
alone, combining any subset of them with the ActivityStreams URL
cannot change public-audience semantics. Addresses
fedify-dev#710 (comment).
Tests exercise the tightened fast path (a throwing contextLoader
proves `jsonld.canonize` is not invoked for known-safe contexts),
the slow-path fallback when an unknown URL sneaks in (a mock loader
records calls and we assert at least one), and the `@context`-skip
invariant (a `customTerm: "as:Public"` inside an inline context is
left untouched).
Assisted-By: Claude Code:claude-opus-4-7
When `verifyProof()` found a cached verification key whose algorithm was not Ed25519, it retried `verifyProof()` with a substitute `keyCache` whose `get()` returned `null`. Per the `KeyCache` contract in `packages/fedify/src/sig/key.ts`, `null` means "the key was fetched previously and found to be unavailable" and is treated as a cached-negative result; `undefined` means "no entry cached at all" and forces `fetchKey()` to hit the network. The retry here wants the latter, so returning `null` silently prevented the refetch and made the Ed25519-mismatch recovery path ineffective. The sibling retry a few lines later (for the proofValue-mismatch case) already uses `undefined`; this change brings the Ed25519 path in line so both retries behave consistently. Addresses fedify-dev#710 (comment). Assisted-By: Claude Code:claude-opus-4-7
The previous fast path only inspected the top-level @context. A JSON-LD document can carry a nested @context inside any subtree to redefine term mappings in a local scope, which means even a safe top-level @context (an allowlisted ActivityStreams document) could sit above an `object` whose inline @context remaps the `as:` prefix to a different namespace. Since `rewritePublicAudience()` descends into those subtrees and would happily rewrite `as:Public` there, the fast path could silently change the target IRI of a nested addressing field. Extend `hasKnownSafeContext()` to walk every non-@context child of the top-level object and require that no descendant carries its own @context. Whenever a nested @context is detected the helper defers to the URDNA2015 canonicalisation check, which catches an actual semantics mismatch and returns the document unchanged. Activities without nested contexts continue to take the fast path. Regression test feeds an activity whose top-level @context is the standard ActivityStreams URL but whose embedded `object` inlines an @context that remaps `as:` to a different namespace, and asserts that both the top-level and the nested `to` keep their original CURIE values (i.e., the rewrite bails out rather than corrupt the nested addressing). Addresses fedify-dev#710 (comment). Assisted-By: Claude Code:claude-opus-4-7
`verifyProof()` documents that it ignores any proof already present on the input JSON-LD, but `verifyProofInternal()` only deleted the compact `proof` key before JCS-hashing the message. A caller handing the function a JSON-LD document in expanded form, which keys the property as `https://w3id.org/security#proof` instead, would leave those bytes in the message digest and fail to verify what would otherwise be a valid signature. The sibling `hasProofLike()` already recognises both the compact and the expanded shapes, so deleting both forms here brings `verifyProofInternal()` in line with that convention. Added a regression test that feeds the FEP-8b32 test vector with a stale expanded proof stapled under `https://w3id.org/security#proof` and asserts verification still returns the expected key. Addresses fedify-dev#710 (comment). Assisted-By: Claude Code:claude-opus-4-7
The unknown-URL fallback test used a dynamic `await import()` of a relative path to `packages/vocab-runtime/src/contexts/activitystreams.json` to hand a stand-in context back to the mock document loader. Deno and Node.js resolve that path just fine at runtime, but the Cloudflare Workers test harness bundles every test module with esbuild up front and could not resolve a relative specifier pointing outside the `packages/fedify` tree, which broke the `test:cfworkers` CI job with `Build failed with 1 error: Could not resolve "../../../vocab-runtime/src/contexts/activitystreams.json"`. Replace the dynamic import with a static reference to `preloadedContexts[AS_CONTEXT]`, which is re-exported from `@fedify/vocab-runtime` and already used elsewhere in the fedify package. The behaviour of the test is unchanged: the loader still resolves every URL to the ActivityStreams context so the two canonical forms match and the rewrite goes through, and `loaderCalls` still proves the slow path was taken for an unknown-URL @context. Assisted-By: Claude Code:claude-opus-4-7
Two security follow-ups on `normalizePublicAudience()`, both now
relevant because `verifyProof()` started calling the helper on
untrusted inbound JSON-LD:
- The URDNA2015 equivalence check previously fell back to
`getDocumentLoader()` whenever the caller did not pass an explicit
`contextLoader`. That default loader fetches remote `@context`
URLs, so an attacker could craft an activity whose top-level
`@context` points at an arbitrary domain and force outbound HTTP
requests every time `verifyProof()` ran the normalization helper.
Require an explicit `contextLoader` for the canonicalization
fallback instead; documents whose `@context` is not drawn from
Fedify's preloaded-contexts set are now returned unchanged when no
loader was supplied. Addresses
fedify-dev#710 (comment).
- The recursive walkers (`hasPublicCurieInAddressing`,
`rewritePublicAudience`, and the new `hasNestedContext` helper)
had no depth limit, so a sufficiently deep JSON-LD payload could
have exhausted the V8 call stack. Cap the walk at a
MAX_TRAVERSAL_DEPTH of 64 levels (effectively unlimited for
legitimate ActivityPub activities, which are typically two or
three deep) and return the conservative answer at the guard:
`false` from the has-CURIE walker (skip the rewrite), the subtree
itself from the rewriter (leave it alone), and `true` from
`hasNestedContext` (force the canonicalization path, which with
the first fix above also skips the rewrite when untrusted).
Addresses
fedify-dev#710 (comment).
Updated tests cover: the no-loader path leaves a non-standard @context
unchanged; a deeply nested 256-level document returns without
overflowing the stack; and the vocab round-trip now passes an explicit
`getDocumentLoader()` because Fedify's default compact form embeds an
inline namespace-prefix object and takes the canonicalization path.
`verifyProofInternal()` dropped the proof key before JCS-hashing by
shallow-copying the input via `{ ...jsonLd }`. If the caller handed
the function an array instead of an object, that spread produced an
object with stringified numeric keys (`{ "0": elem, "1": elem, ... }`),
which JCS would then canonicalize into a shape that never matches what
the signer actually hashed. An array is also not a valid top-level
FEP-8b32 signed document: proofs live on single objects.
Widen the up-front validation to reject arrays (as well as the
already-rejected non-objects and `null`) and return `null` so the
caller treats the input as unsigned. A regression test hands
`verifyProof()` an array wrapping the same document the next-line test
verifies successfully and asserts it gets `null`. Addresses
fedify-dev#710 (comment).
The previous hardening commit moved the SSRF gate into `normalizePublicAudience()` itself: when no `contextLoader` was supplied, non-standard `@context` shapes skipped the URDNA2015 equivalence check and the document was returned unchanged. That was safe for the inbound verification path, but it silently broke `createProof()` for callers that did not pass an explicit loader. The Fedify/vocab default compact form embeds an inline namespace-prefix object at the end of `@context`, so most signed activities fail the "known-safe context" shortcut and fall to the canonicalization path. After the previous commit, a caller that invoked `signObject()` without `contextLoader` would therefore sign the bytes of the `as:Public` CURIE form (because normalization was suppressed), but `middleware.ts` sends activities through `normalizePublicAudience()` again at wire time with its own `contextLoader`, which does rewrite the CURIE. The signed bytes and the on-wire bytes then diverge and every `eddsa-jcs-2022` proof produced along that path fails verification. Move the gate to the one place that actually handles adversarial input. `normalizePublicAudience()` again falls back to `getDocumentLoader()` when `contextLoader` is omitted, which is fine for signing paths that run on local, trusted JSON-LD (and restores the consistent signer/wire bytes Fedify's fan-out relies on). `verifyProofInternal()` now wraps its `normalizePublicAudience()` call in an explicit `options.contextLoader != null` check; without a loader the fallback candidate is simply not tried, so an attacker cannot steer the default loader into fetching an arbitrary `@context` URL during verification. Callers who want the fallback during verification already supply a `contextLoader` (all internal call sites in `middleware.ts` do), so no functional change there. Addresses fedify-dev#710 (comment), fedify-dev#710 (comment), and fedify-dev#710 (comment). The earlier SSRF concern at fedify-dev#710 (comment) remains addressed by the new gate in `verifyProofInternal()`. Regression test in proof.test.ts feeds verifyProof an activity whose `@context` mentions an attacker-controlled URL and confirms the call returns null without attempting a fetch. The no-loader helper test that asserted the now-removed skip behavior is gone; the unknown-URL loader-counting test still exercises the slow canonicalization path for callers that do pass a loader.
The previous hardening commit described the default fallback loader
as "only resolves URLs in the preloaded-contexts set", but the loader
it actually used was `@fedify/vocab-runtime`'s `getDocumentLoader()`,
which happily issues network requests for any non-preloaded URL after
its `validatePublicUrl()` check. The docstring was therefore wrong
and, worse, the verification-side gate that had been added to close
off the SSRF vector caused a new false-negative regression: verifying
a Fedify 2.2-signed proof against `signed.toJsonLd({ format: "compact" })`
output without passing `options.contextLoader` used to work (the
canonicalization fallback produced the normalized candidate) but now
returned null because normalization was suppressed entirely.
Fix both by giving `normalizePublicAudience()` its own narrow default
loader that resolves only URLs in the preloaded-contexts set and
rejects every other URL without issuing a network request.
Canonicalization errors against this restricted loader fall through
to the existing "return the document unchanged" branch, so
adversarial `@context` URLs cannot be weaponized into outbound HTTP
requests during verification. For URLs Fedify already preloads
(ActivityStreams, security, data-integrity, multikey, and so on) the
canonicalization path still works, so the round-trip case from
discussion_r3122865191 verifies again even when the caller omits
`contextLoader`.
Remove the now-redundant `options.contextLoader != null` gate in
`verifyProofInternal()`: the helper is already safe by construction,
so the gate was only blocking the normalized candidate from being
tried at all. Updated the adjacent test and comment accordingly:
the test still exercises a non-preloaded attacker `@context` and
still returns null, but the reasoning shifts from "normalization was
skipped" to "canonicalization rejected the fetch without touching
the network".
Addresses
fedify-dev#710 (comment)
and
fedify-dev#710 (comment).
Two small cleanups on the same function, both from the same
review:
- The `{ ...jsonLd } as Record<string, unknown>` spread triggers
a strict-TypeScript warning because `jsonLd` is still typed as
`unknown` at that point, even though the enclosing guard has
already narrowed it to a non-null, non-array object. Move the
cast inside the spread so the spread operates on a typed record
from the start. Addresses
fedify-dev#710 (comment).
- `proofDigest` is constant across candidate messages, so the
combined digest buffer can be allocated once outside the loop
and only the message-digest tail rewritten per iteration.
SHA-256 always produces 32 bytes, pulled into a named constant.
The `proof.proofValue.slice()` call stays put with a clarifying
comment: removing it looks like a pure micro-optimization but
actually matters for typing, since `proof.proofValue`'s buffer
is `ArrayBufferLike` and `crypto.subtle.verify()` needs a
non-shared `ArrayBuffer`, which `.slice()` provides. Addresses
fedify-dev#710 (comment).
Fedify CLI 2.1.8 ships with `@fedify/next` 2.1.8, whose peer dependency on `next` was widened to `>=15.4.6 <17`, so `fedify init -w next` against the current `create-next-app` now installs Next.js 16 cleanly without the manual `package.json` edit the tutorial used to describe. Remove the WARNING callout and the pinning block, and update the prereq sentence in [*Installing the `fedify` command*] to require CLI 2.1.8 or higher (the version that gained Next.js 16 support) rather than the 2.2.0 it previously named. Also bump the Next.js version printed in the sample *npm run dev* console output from 15.5.15 to 16.2.4 so it matches what a reader actually sees. The `fedify-dev/threadiverse` example repository has been updated to the same dependency versions in a paired commit. [*Installing the `fedify` command*]: https://fedify.dev/tutorial/threadiverse#installing-the-fedify-command Resolves fedify-dev#713.
`@fedify/vocab-runtime` 2.1.10 preloads `https://join-lemmy.org/context.json` internally (closes fedify-dev#714), so the *Bundle Lemmy's JSON-LD context* sub-section in *Making the community actor Lemmy-compatible* is no longer relevant. Remove the sub-section, including the 40-line *federation/lemmy-context.json* + custom `documentLoaderFactory` block it walked readers through. Replace it with a brief NOTE callout that preserves the historical context (why the workaround existed, when it went away) for readers on older Fedify versions. Adjust the preceding "Two more changes" sentence to "One more change" since only the `Accept(Follow)` trim is left. Also bump the *Installing the `fedify` command* prereq note from 2.1.8 to 2.1.10, since 2.1.10 is the first release that ships the Lemmy-context preload the tutorial's Lemmy-interop chapter now relies on. Closes fedify-dev#714.
Two small hardening tweaks from the latest review pass, both
addressing surface that adversarial JSON-LD input could reach:
- `preloadedOnlyDocumentLoader` was checking `url in
preloadedContexts`, which walks up `Object.prototype`. An
attacker who puts a context URL like `"toString"` or
`"constructor"` in an inbound activity would have the `in` check
return true and then receive `Object.prototype.toString` (or
similar) as the document body, which makes `jsonld.canonize`
behave oddly on a freshly-arrived activity. Switch to
`Object.hasOwn()` so only the URLs actually registered in the
preloaded-contexts set resolve.
- `rewritePublicAudience()` cloned each recursively-rewritten
subtree into a plain `{}` literal and then did `normalized[key]
= rewritten`. `JSON.parse()` stores a literal `__proto__` key
as an own enumerable data property, so `Object.keys(record)`
surfaces it and the subsequent assignment goes through the
`__proto__` setter, which mutates the prototype chain and
(given that an inbound activity is the source) turns into
prototype pollution of the host process's
`Object.prototype`. Clone into `Object.create(null)` so the
target has no prototype and the `__proto__` assignment stays a
regular own property.
Extended the addressing-field test so it actually exercises all
five normalized fields (`to`, `cc`, `bto`, `bcc`, `audience`) with
both CURIE forms (`as:Public` and bare `Public`), and added a
new test that hand-crafts an inbound activity with `__proto__` as
a first-class JSON key and asserts `Object.prototype` is not
polluted after normalization.
Addresses
fedify-dev#710 (comment)
and
fedify-dev#710 (comment).
When a reader follows the *Swap ESLint for Biome* chapter and runs `npm run format`, Biome emits *two* errors, not one: the unused `getLogger` import the tutorial already called out, plus a CSS parse error on the Tailwind directives `create-next-app` leaves in *app/globals.css* (`@theme inline`, a dark-mode media query). The CSS error aborts formatting for that file and was confusing enough to get reported as a review comment. Rewrite the paragraph that described the unused-logger issue as a two-item list, noting that the CSS error is expected and will go away in the next chapter when we replace *app/globals.css* wholesale with a small set of vanilla rules. The logger fix keeps its existing code block and explanation; only the framing changes. Addresses fedify-dev#710 (comment).
Twelve code blocks in the tutorial used VitePress title annotations that contained Next.js dynamic-segment brackets, e.g. \`~~~~ tsx [app/users/[username]/page.tsx]\`. VitePress's title parser matches on the first closing bracket, so those annotations rendered as \`app/users/[username\` in the final HTML and left the closing \`]/page.tsx]\` dangling. Screenshots from sij411 in review surfaced the same symptom on two different sections; the same truncation affects every block whose filename contains a dynamic segment. VitePress offers no escape syntax for brackets inside a title, HTML entities and backslash escapes both leak through verbatim, and hongdown happily decodes either form on the next format pass, so there is no reliable way to keep the annotation as-is. The pragmatic fix is to drop the title annotation from the twelve affected blocks. The prose immediately above each block already names the file (e.g., "Create *app/users/\[username\]/page.tsx*:"), so the signpost readers need is preserved; only the redundant, truncated VitePress chip goes away. Addresses fedify-dev#710 (comment) and fedify-dev#710 (comment).
The *Subscribing to communities* chapter gradually introduces three identifiers in *federation/index.ts* without showing them in an import block: the \`follows\` table from \`@/db\`, and the \`Follow\` and \`Accept\` activity classes from \`@fedify/vocab\`. Readers who were copy-pasting the code snippets verbatim (or who scanned the snippets expecting them to be complete standalone examples) ran into TypeScript errors because the identifiers were undeclared. Add a short TIP callout immediately under the chapter's intro paragraph so the need to extend the existing import blocks is flagged up front, before any of the new names show up in a snippet. Addresses fedify-dev#710 (comment).
With the VitePress title annotations stripped from code blocks whose
filename contains a Next.js dynamic segment, the three snippets in
the *Votes (`Like` and `Dislike`)* chapter lost their only cue about
which file the reader should be editing. Add that back in the
surrounding prose:
- The vote-tally query now opens with "Inside *app/users/\[username\]/threads/\[id\]/page.tsx*, declare a small \`VoteTally\` type and ...".
- The `<VoteButtons>` component snippet opens with "Still in *app/users/\[username\]/threads/\[id\]/page.tsx*, define a small ...".
- The *Outbound: `castVote` action* section gains a one-line intro telling the reader to create *app/users/\[username\]/threads/\[id\]/vote-actions.ts*.
While I was there I also hoisted the \`VoteTally\` type into the first
snippet itself. The snippet was using \`new Map<string, VoteTally>()\`
without defining or importing the type anywhere in the tutorial, so
readers copying the snippet into their own file got an undeclared
identifier error. The new type alias at the top of the snippet
matches the \`{ likes, dislikes, myVote }\` shape the code already
constructs further down.
Addresses
fedify-dev#710 (comment),
fedify-dev#710 (comment),
fedify-dev#710 (comment),
and
fedify-dev#710 (comment).
Seven places in the tutorial wrote paired alternatives with a spaced
forward slash (\`Like\` / \`Dislike\`, \`Update(Page)\` / \`Update(Note)\`,
Active / Hot, and so on). The Fedify house style for this
construction is a tight slash — \`Like\`/\`Dislike\`, \`Update(Page)\`/
\`Update(Note)\` — regardless of whether the flanking tokens are
backticked, italicised, or plain text. The spaced form reads as an
editorial pause and takes up more room than the alternation
warrants.
Apply a single sweep across *docs/tutorial/threadiverse.md*:
- *Log in*/*Sign up* in the layout nav description
- \`Like\`/\`Dislike\` in two sub-section titles and prose
- \`Update(Page)\`/\`Update(Note)\`, \`Delete(Page)\`/\`Delete(Note)\`,
Lemmy *Active*/*Hot*, and \`SqliteKvStore\`/\`SqliteMessageQueue\`
in the closing *Areas to improve* list
The tutorial's Target audience section mentions \`ORMs\` alongside other abbreviated acronyms, but markdown-it-abbr matches literal terms and does not stem across morphology, so the existing \`*[ORM]: Object–Relational Mapping\` definition leaves the plural unannotated. Add a sibling \`*[ORMs]: Object–Relational Mappings\` definition so the plural also renders as an \`<abbr>\` with a hover expansion, matching the reader experience of the singular.
⚡ Risk Assessment —
|
| Files | Summary |
|---|---|
Public Audience Normalization Implementationpackages/fedify/src/compat/public-audience.tspackages/fedify/src/compat/public-audience.test.ts |
Adds normalizePublicAudience() helper to rewrite compact as:Public/Public CURIEs to expanded URIs. Includes fast path for known-safe contexts and URDNA2015 equivalence check for unknown contexts. Protects against SSRF by using preloaded-only document loader by default. Comprehensive test coverage for edge cases including prototype pollution, deep nesting, and nested context redefinition. |
Proof Verification with Public Audience Supportpackages/fedify/src/sig/proof.tspackages/fedify/src/sig/proof.test.ts |
Integrates normalizePublicAudience() into proof verification to handle both compact and expanded public addressing forms. Strips both compact and expanded proof forms before JCS hashing. Tries on-wire form first, then normalized form as fallback. Adds regression tests ensuring round-trip signing/verification works with public activities. |
Activity Sending with Public Audience Normalizationpackages/fedify/src/federation/middleware.tspackages/fedify/src/federation/middleware.test.ts |
Applies normalizePublicAudience() to activities before sending to ensure public addressing is expanded to full URI form. Adds test verifying activities with PUBLIC_COLLECTION are sent with expanded URI, not CURIE. |
Documentation Updatesdocs/.vitepress/config.mtsCHANGES.md |
Adds threadiverse community platform tutorial link to navigation. Documents threadiverse interoperability improvements and new tutorial in changelog. |
Sequence Diagram
sequenceDiagram
participant Caller
participant sendActivity
participant normalizePublicAudience
participant verifyProof
participant crypto
Caller->>sendActivity: activity with PUBLIC_COLLECTION
sendActivity->>sendActivity: serialize to JSON-LD (compact form)
sendActivity->>normalizePublicAudience: jsonLd with as:Public CURIE
normalizePublicAudience->>normalizePublicAudience: check if CURIE present
alt has CURIE
normalizePublicAudience->>normalizePublicAudience: rewrite to expanded URI
alt known-safe context
normalizePublicAudience-->>sendActivity: normalized (fast path)
else unknown context
normalizePublicAudience->>normalizePublicAudience: canonicalize both forms
normalizePublicAudience-->>sendActivity: normalized or original
end
else no CURIE
normalizePublicAudience-->>sendActivity: unchanged
end
sendActivity->>sendActivity: send to inboxes
Caller->>verifyProof: received activity + proof
verifyProof->>normalizePublicAudience: try normalize
verifyProof->>crypto: verify with on-wire form
alt verification fails
verifyProof->>crypto: verify with normalized form
end
crypto-->>verifyProof: verified or null
verifyProof-->>Caller: key or null
Dig Deeper With Commands
/review <file-path> <function-optional>/chat <file-path> "<question>"/roast <file-path>
Runs only when explicitly triggered.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mirror of upstream fedify-dev#710 for benchmark. Do not merge.
Summary by MergeMonkey