Fixes to atproto-loader

Author: essential-randomness

.changeset/atproto-loader-blobs-and-pds.md

@@ -0,0 +1,15 @@
+---
+"@fujocoded/astro-atproto-loader": patch
+---
+
+Fixes `0.2.0`, which was broken: `defineAtProtoCollection` and `defineAtProtoLiveCollection` didn't return the shape Astro expected, so callers had to wrap them in `defineCollection` / `defineLiveCollection` themselves. They now return the real Astro collection shape and work as documented.
+
+**Breaking:** `fetchRecord({ atUri })` now resolves to `{ value, repo }` instead of just the record value. Existing callers need to read `.value`. The new `repo` field carries the fetched record's `{ did, pds }` so it can be passed directly to `toHostedBlob` without re-resolving identity. This is shipped as a patch (and not a minor bump) because `0.2.0` was only on npm for ~8 hours and is being deprecated alongside this release, so realistically nobody is depending on the old `fetchRecord` shape.
+
+Also in this release:
+
+- New `toHostedBlob({ repo, blob })` for building `com.atproto.sync.getBlob` URLs, plus the `isAtBlob` guard and the `AtBlob` type.
+- `AtProtoRecordRepo` now includes `pds` alongside `did`, so `args.repo` works directly with `toHostedBlob`.
+- `getPds(repo)` is exported and shares the identity cache with `getClient`.
+- `BlobRef` and `CID` instances are flattened to `{ $link }` before being stored, so records with blobs don't break Astro's devalue store.
+- Added the `04-single-entry` example.

astro-atproto-loader/README.md

@@ -57,6 +57,9 @@ In this package, you'll find:
   `defineLiveCollection()`
 - `defineAtProtoCollection()`, which reads public AtProto records at build
   time. Use it where you'd otherwise call Astro's `defineCollection()`
+- `toHostedBlob()` and `isAtBlob()`, helpers for turning a record's blob
+  ref (a profile avatar, a sprite sheet, a stream thumbnail) into a URL you
+  can drop into `<img src>`. Use them inside `transform`
 
 > [!WARNING]
 >
@@ -81,6 +84,9 @@ In this package, you'll find:
 - **Hydrate linked records** like `strongRef`s and `subject` URIs from inside
   your `transform`, so a post's quoted record or a label's subject is already
   resolved by the time your page renders
+- **Display blob-backed media** like profile avatars, sprite sheets, or
+  stream thumbnails by handing the blob ref to `toHostedBlob()` inside
+  `transform`
 
 > [!TIP]
 >
@@ -101,9 +107,27 @@ Before you start, you'll need:
 1. Run the following command:
 
 ```bash
-npm add @fujocoded/astro-atproto-loader
+npm add @fujocoded/astro-atproto-loader@latest
 ```
 
+> [!CAUTION]
+>
+> **`0.2.0` is broken. Upgrade to `0.2.1` or later.**
+>
+> If you're stuck on `0.2.0`, the correct code requires wrapping the call in "defineCollection":
+>
+> ```ts
+> import { defineCollection, z } from "astro:content";
+> import { defineAtProtoCollection } from "@fujocoded/astro-atproto-loader";
+>
+> const sprites = defineCollection(
+>   defineAtProtoCollection({
+>     source: { repo: "bmann.ca", collection: "actor.rpg.sprite" },
+>     outputSchema: z.any(),
+>   }),
+> );
+> ```
+
 2. Define a static collection (for build time magic)...
 
 ```ts
@@ -160,6 +184,9 @@ You can start with any of these:
 - [`03-grouped-reposts`](./__examples__/03-grouped-reposts/) for reading from
   multiple repos at once with `sources: [...]` and merging records with
   `groupBy`
+- [`04-single-entry`](./__examples__/04-single-entry/) for fetching one
+  record by `rkey` with `getEntry()` and `getLiveEntry()`, and turning the
+  record's blob into a URL with `toHostedBlob()`
 
 The first two examples show off two patterns:
 
@@ -240,7 +267,7 @@ defineAtProtoLiveCollection({
     // value is already the parsed lexicon type
     const quoted =
       value.embed?.$type === "app.bsky.embed.record"
-        ? await fetchRecord({ atUri: value.embed.record.uri })
+        ? (await fetchRecord({ atUri: value.embed.record.uri }))?.value
         : null;
     return { id: uri, data: { text: value.text, quoted } };
   },
@@ -256,16 +283,22 @@ Every `filter` and `transform` callback receives `fetchRecord({ atUri, parse?
 than one callback asks for the _same_ URI in the same cycle (for example a
 `subject` URI shared across many records), they share a single network call.
 
+A successful call resolves to `{ value, repo }`. `value` is the record body
+(or whatever your `parse` callback returned). `repo` is the fetched record's
+owning DID and PDS, already resolved. So you can hand it straight to
+`toHostedBlob({ repo, blob })` for any blob inside that hydrated record
+without re-resolving identity.
+
 ```ts
 import { $parse, lexicons } from "@atproto/lex";
 
 transform: async ({ value, uri, fetchRecord }) => {
-  const subject = await fetchRecord({
+  const result = await fetchRecord({
     atUri: value.subject.uri,
     parse: (v) => $parse(lexicons, "app.bsky.actor.profile", v),
   });
-  if (!subject) return null; // record was missing, unparseable, or unreachable
-  return { id: uri, data: { label: value.val, subject } };
+  if (!result) return null; // record was missing, unparseable, or unreachable
+  return { id: uri, data: { label: value.val, subject: result.value } };
 };
 ```
 
@@ -274,6 +307,55 @@ PDS that can't be reached, a 404, a record whose value isn't an object, or a
 `parse` callback that threw. Each of these logs a distinct warning to your
 console, so when something is missing you can tell which thing went wrong.
 
+## Blob helpers: turning record blobs (and images) into URLs
+
+When looking to display images or other files associated with records,
+you won't (unfortunately) find a simple address you can drop into
+`<img src>`: AtProto records don't store this content themselves, but instead
+only hold a "pointer" (called a _blob ref_) to the actual file on a user PDS.
+
+To show that profile avatar, sprite sheet, or video thumbnail on
+your page, we must turn the pointer into a real URL the browser
+can load. That's what `toHostedBlob()` is for.
+
+`toHostedBlob()` needs 2 things:
+
+- the `blob`ref itself
+- the `repo` that owns the file, that is the DID + PDS url wher the record
+  is hosted
+
+For records you own, `repo` will likely out of `args.repo`. While for records
+you've hydrated via `fetchRecord`, use the `repo` field on the result.
+
+```ts
+import {
+  defineAtProtoLiveCollection,
+  isAtBlob,
+  toHostedBlob,
+} from "@fujocoded/astro-atproto-loader";
+
+defineAtProtoLiveCollection({
+  source: { repo: "boba-tan.bsky.social", collection: "actor.rpg.sprite" },
+  outputSchema: z.object({
+    spriteSheet: z.object({
+      url: z.url(),
+      mimeType: z.string(),
+      size: z.number(),
+    }),
+  }),
+  transform: ({ repo, rkey, value }) => {
+    const v = value as { spriteSheet: unknown };
+    if (!isAtBlob(v.spriteSheet)) return undefined;
+    return {
+      id: rkey,
+      data: { spriteSheet: toHostedBlob({ repo, blob: v.spriteSheet }) },
+    };
+  },
+});
+```
+
+You can find a working example at [`__examples__/04-single-entry`](./__examples__/04-single-entry/).
+
 ## Multi-source reads and `onSourceError`
 
 When you're reading from `sources: [...]`, `onSourceError` decides what

astro-atproto-loader/__examples__/03-grouped-reposts/src/live.config.ts

@@ -140,7 +140,7 @@ const sharedReposts = defineAtProtoLiveCollection({
     // Hydrate the post, the author's profile, and every reposter's
     // profile in parallel. fetchRecord coalesces same-URI requests within
     // one load, so duplicate lookups across groups hit the network once.
-    const [post, authorProfile] = await Promise.all([
+    const [postResult, authorProfileResult] = await Promise.all([
       fetchRecord({
         atUri: key,
         parse: (value: unknown) => PostSchema.parse(value),
@@ -150,18 +150,21 @@ const sharedReposts = defineAtProtoLiveCollection({
         parse: (value: unknown) => ProfileSchema.parse(value),
       }),
     ]);
-    if (!post) return null;
+    if (!postResult) return null;
+    const post = postResult.value;
+    const authorProfile = authorProfileResult?.value;
 
     // Hydrate each reposter's profile. `record.repo.did` is the resolved
     // DID; `record.repo.handle` is set when the source config gave us a
     // handle (true for all three of ours), so we use it for `@handle`-style
     // display and fall back to the DID otherwise.
     const repostedBy = await Promise.all(
       records.map(async (record) => {
-        const profile = await fetchRecord({
+        const profileResult = await fetchRecord({
           atUri: getProfileAtUri({ did: record.repo.did }),
           parse: (value: unknown) => ProfileSchema.parse(value),
         });
+        const profile = profileResult?.value;
         const avatarCid = profile?.avatar?.cid;
         const displayHandle = record.repo.handle ?? record.repo.did;
         return {

astro-atproto-loader/__examples__/04-single-entry/.gitignore

@@ -0,0 +1,24 @@
+# build output
+dist/
+# generated types
+.astro/
+
+# dependencies
+node_modules/
+
+# logs
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+
+
+# environment variables
+.env
+.env.production
+
+# macOS-specific files
+.DS_Store
+
+# jetbrains setting folder
+.idea/

astro-atproto-loader/__examples__/04-single-entry/README.md

@@ -0,0 +1,34 @@
+# Single entry example
+
+This example shows how to pull a single AtProto record by `rkey` instead of
+loading the whole collection, using `@fujocoded/astro-atproto-loader` with both
+`defineAtProtoCollection()` (static) and `defineAtProtoLiveCollection()`
+(live).
+
+It reads one `actor.rpg.sprite` record from `bmann.ca` at the `rkey` `self`,
+turns the sprite sheet's blob ref into a hosted URL, and renders it on a page
+with `<img src>`.
+
+The example uses the loader two ways:
+
+- The static page (`/`) calls `getEntry("sprites-static", "self")` against the
+  collection from `content.config.ts`. The record is fetched at build time
+- The live page (`/live`) calls `getLiveEntry("sprites-live", "self")` against
+  the collection from `live.config.ts`. The record is fetched on each request
+
+Both configs share the same `transform`. It uses `isAtBlob()` to drop the
+record if the `spriteSheet` field isn't a blob ref, then `toHostedBlob()` to
+turn the blob ref into `{ url, mimeType, size }` ready for `<img src>`.
+
+## Run it
+
+```bash
+npm install
+npm run dev
+```
+
+Then open `http://127.0.0.1:4321` for the static page and
+`http://127.0.0.1:4321/live` for the live page.
+
+If you want to inspect the built static output instead, run `npm run build`
+and then `npm run preview`.

astro-atproto-loader/__examples__/04-single-entry/astro.config.mjs

@@ -0,0 +1,10 @@
+// @ts-check
+import { defineConfig } from "astro/config";
+import node from "@astrojs/node";
+
+export default defineConfig({
+  output: "server",
+  adapter: node({
+    mode: "standalone",
+  }),
+});