diff --git a/.agents/skills/building-with-hypercerts-lexicons/SKILL.md b/.agents/skills/building-with-hypercerts-lexicons/SKILL.md index a2cf7dd4..88ad1dda 100644 --- a/.agents/skills/building-with-hypercerts-lexicons/SKILL.md +++ b/.agents/skills/building-with-hypercerts-lexicons/SKILL.md @@ -190,6 +190,41 @@ if (result.success) { } ``` +## Permission Sets (OAuth scopes) + +To **write** Hypercerts, Hyperboards, or Certified records to a user's repo, +your app needs the user's authorization. AT Protocol grants this via OAuth +scopes: each collection write needs a `repo:?action=…` scope. Rather +than list every collection by hand, request a **permission set** — a published +bundle — with a single `include:` scope: + +| Permission set NSID | Grants write (create/update/delete) on | +| --------------------------- | ------------------------------------------ | +| `org.hypercerts.authWrite` | all `org.hypercerts.*` record collections | +| `org.hyperboards.authWrite` | all `org.hyperboards.*` record collections | +| `app.certified.authWrite` | all `app.certified.*` record collections | + +In your OAuth client's authorization request, ask for the set(s) you need: + +```text +scope: include:org.hypercerts.authWrite +``` + +The user's PDS resolves the published set and expands it into the underlying +`repo:` scopes on the consent screen, where the user sees the set's plain-language +description (e.g. "Manage your Hypercerts data"). Notes: + +- **One set per namespace, never combined.** A permission set may only reference + its own namespace authority (atproto permission spec), so `org.hypercerts.*`, + `org.hyperboards.*`, and `app.certified.*` each need their own set. An app + needing more than one requests **each** `include:` scope. +- **Writes only.** These cover create/update/delete. **Reading** records needs no + scope at all — atproto repo records are public. +- **`repo:` scopes carry no `aud`** — request the `include:` without an `?aud=`. +- These same published sets are also consumed by the Certified group service + (CGS) when it expands an API key's `include:` scope — see the CGS integration + guide. + ## Lexicon Overview ### Claims — the core impact record @@ -673,6 +708,8 @@ avoid typos. ## Further Reading - [SCHEMAS.md](https://github.com/hypercerts-org/hypercerts-lexicon/blob/main/SCHEMAS.md) — full property-level documentation for every lexicon +- [docs/design/permission-sets.md](https://github.com/hypercerts-org/hypercerts-lexicon/blob/main/docs/design/permission-sets.md) — the permission sets and how they are consumed (OAuth + CGS API keys) +- [ATProto permission spec](https://atproto.com/specs/permission#permission-sets) — OAuth scopes and permission sets - [CHANGELOG.md](https://github.com/hypercerts-org/hypercerts-lexicon/blob/main/CHANGELOG.md) — version history and migration guides - [ATProto Lexicon Guide](https://atproto.com/guides/lexicon) — AT Protocol lexicon fundamentals - [ATProto Lexicon Style Guide](https://atproto.com/guides/lexicon-style-guide) — schema design conventions diff --git a/.changeset/permission-sets-crud.md b/.changeset/permission-sets-crud.md new file mode 100644 index 00000000..82c89962 --- /dev/null +++ b/.changeset/permission-sets-crud.md @@ -0,0 +1,15 @@ +--- +"@hypercerts-org/lexicon": minor +--- + +Add three permission-set lexicons — `org.hypercerts.authWrite`, `org.hyperboards.authWrite`, and `app.certified.authWrite` — each granting create/update/delete over every record collection in its namespace. + +A [permission set](https://atproto.com/specs/permission#permission-sets) lets any AT Protocol app request a whole bundle of `repo:` scopes with a single `include:` OAuth scope, instead of enumerating each collection by hand. The user's PDS resolves and expands the set during the OAuth grant; the same published set can also be consumed by services (e.g. the Certified group service) when expanding API-key scopes. + +There are **three** sets rather than one because the spec requires it: a permission set "is limited to expressing permissions that reference resources under the same NSID namespace as the set itself" and "can not address 'sibling groups' or 'parents'". `org.hypercerts`, `org.hyperboards`, and `app.certified` are separate namespace authorities, so they cannot be combined in a single set — an app needing more than one requests each `include:` scope. + +Permission sets are published as-is (they are the source of truth for what gets published to AT Protocol) but have no TypeScript shape — `lex gen-api` cannot generate code for `permission-set` defs. They are therefore excluded from the codegen globs (`gen-api`/`gen-md`/`gen-ts`) and from `generated/exports.ts`, while still shipping as raw lexicon JSON. + +Collection lists are enumerated explicitly because the spec forbids wildcards inside a permission set; they must be kept in sync as record types are added. See `docs/design/permission-sets.md`. + +The generated `SCHEMAS.md` reference now renders permission-set entries (title, detail, and the resource/collections/actions each set grants) instead of leaving them as empty sections. diff --git a/AGENTS.md b/AGENTS.md index 07566ed2..6cc7a518 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -234,11 +234,32 @@ scripts/ Build and codegen scripts - Generate types in `generated/` (including vendored external lexicons) - Auto-generate `generated/exports.ts` with all exports -3. Update `ERD.puml` as appropriate: +3. Update the relevant **permission set** if appropriate: + - The `permission-set` lexicons (`lexicons/org/hypercerts/authWrite.json`, + `lexicons/org/hyperboards/authWrite.json`, and + `lexicons/app/certified/authWrite.json`) each enumerate **every + `type: "record"` collection in their namespace** — the list is both + the grant and its boundary, with no wildcards allowed inside a set. + - When you **add** a new `org.hypercerts.*`, `org.hyperboards.*`, or + `app.certified.*` record lexicon, add its NSID to the matching set's + `permissions[].collection` list, or it will not be grantable via the + published set. Likewise **remove** the NSID when you delete a record + lexicon. + - A set may only reference collections under **its own namespace + authority**, so each namespace's records go in its own set + (`org.hypercerts.*` → `org.hypercerts.authWrite`, `org.hyperboards.*` → + `org.hyperboards.authWrite`, `app.certified.*` → + `app.certified.authWrite`) — never another set. + - This does not apply to non-record defs (e.g. `query`, `object`, or + `permission-set` lexicons themselves), which are not repo collections. + - See [docs/design/permission-sets.md](docs/design/permission-sets.md) + for the full rationale. + +4. Update `ERD.puml` as appropriate: - **Include all fields except facet fields** (they're cosmetic and don't affect structure) -4. Update `README.md` and `SKILL.md` as appropriate: +5. Update `README.md` and `SKILL.md` as appropriate: - If `README.md` or `.agents/skills/building-with-hypercerts-lexicons/SKILL.md` already references that lexicon, **both files must be updated** @@ -263,14 +284,14 @@ scripts/ Build and codegen scripts valid export names, `$type` strings, and API call signatures. This will catch many (but not all) documentation errors. -5. Run `npm run gen-schemas-md` to regenerate `SCHEMAS.md` +6. Run `npm run gen-schemas-md` to regenerate `SCHEMAS.md` -6. Run `npm run format` to ensure everything is formatted correctly +7. Run `npm run format` to ensure everything is formatted correctly via Prettier -7. Run `npm run check` to validate, typecheck, and build +8. Run `npm run check` to validate, typecheck, and build -8. **REQUIRED: Create a changeset file** using the `writing-changesets` skill +9. **REQUIRED: Create a changeset file** using the `writing-changesets` skill - **This step is MANDATORY for ALL changes that affect users** - See the `writing-changesets` skill for details on changeset requirements - Use the skill - do NOT write changeset files manually diff --git a/SCHEMAS.md b/SCHEMAS.md index 1c16ca94..3249f53c 100644 --- a/SCHEMAS.md +++ b/SCHEMAS.md @@ -7,6 +7,20 @@ Hypercerts-specific lexicons for tracking impact work and claims. +### `org.hypercerts.authWrite` + +**Title:** Manage your Hypercerts data + +**Detail:** Create, edit, and delete your Hypercerts records (impact claims, evaluations, funding, and related data). + +**Resource:** `repo` + +**Collections:** `org.hypercerts.claim.activity`, `org.hypercerts.claim.contribution`, `org.hypercerts.claim.contributorInformation`, `org.hypercerts.claim.rights`, `org.hypercerts.collection`, `org.hypercerts.context.acknowledgement`, `org.hypercerts.context.attachment`, `org.hypercerts.context.evaluation`, `org.hypercerts.context.measurement`, `org.hypercerts.funding.receipt`, `org.hypercerts.workscope.tag` + +**Actions:** `create`, `update`, `delete` + +--- + ### `org.hypercerts.claim.activity` **Description:** A hypercert record tracking impact work. @@ -484,6 +498,20 @@ A labeled URL reference. --- +### `app.certified.authWrite` + +**Title:** Manage your Certified data + +**Detail:** Create, edit, and delete your Certified records (profile, badges, follows, wallet links, and related data). + +**Resource:** `repo` + +**Collections:** `app.certified.actor.organization`, `app.certified.actor.profile`, `app.certified.badge.award`, `app.certified.badge.definition`, `app.certified.badge.response`, `app.certified.graph.follow`, `app.certified.link.evm`, `app.certified.location`, `app.certified.signature.proof` + +**Actions:** `create`, `update`, `delete` + +--- + ### `app.certified.graph.follow` **Description:** Record declaring a social 'follow' relationship of another account. Duplicate follows will be ignored by the AppView. @@ -722,6 +750,20 @@ Specifies the sub-string range a facet feature applies to. Start index is inclus --- +### `org.hyperboards.authWrite` + +**Title:** Manage your Hyperboards data + +**Detail:** Create, edit, and delete your Hyperboards records (board configurations and display profiles). + +**Resource:** `repo` + +**Collections:** `org.hyperboards.board`, `org.hyperboards.displayProfile` + +**Actions:** `create`, `update`, `delete` + +--- + ### `org.hyperboards.board` **Description:** Configuration record for a hyperboard, wrapping an underlying activity or collection with visual presentation settings. Stored in the creator's PDS. diff --git a/docs/design/permission-sets.md b/docs/design/permission-sets.md new file mode 100644 index 00000000..71cc242c --- /dev/null +++ b/docs/design/permission-sets.md @@ -0,0 +1,408 @@ +# Design: Permission Sets for Hypercerts, Hyperboards & Certified data + +Status: **Implemented** — the sets described here are published as lexicon +files in this repository (`lexicons/org/hypercerts/authWrite.json`, +`lexicons/org/hyperboards/authWrite.json`, +`lexicons/app/certified/authWrite.json`). This document records the design and +the rationale behind it. + +This document designs **permission sets** for the `org.hypercerts.*`, +`org.hyperboards.*`, and `app.certified.*` record collections defined in this +repository — published, reusable bundles of AT Protocol OAuth scopes that any +application can request with a single `include:` scope, instead of +enumerating every underlying `repo:` scope by hand. + +## Background: what a permission set is + +AT Protocol clients act on a user's data with **OAuth scopes**. Writing records +of a given collection requires a `repo:?action=…` scope; calling a +service method requires an `rpc:?aud=…` scope. (See the +[AT Protocol permission spec](https://atproto.com/specs/permission).) + +A **permission set** ([spec](https://atproto.com/specs/permission#permission-sets)) +is a named, published Lexicon schema that bundles several such scopes under one +NSID. A client requests the whole bundle with one scope string — +`include:` — and the user's Authorization Server (their PDS) resolves the +set and expands it into the underlying scopes when granting access. Per the +spec, permission sets are _"published publicly and can be used by any client +developer"_, and _"Authorization Servers resolve, authenticate, and process +permission-sets dynamically."_ + +The motivation is plain: an application that does CRUD over a family of +collections — say, all the records that make up a hypercert — would otherwise +have to list every `repo:org.hypercerts.…?action=…` scope itself, keep that list +correct, and re-derive it independently from every other integrator. A published +set replaces that with one curated, versioned name. + +## Who uses these sets + +These are general AT Protocol artifacts, not specific to any one service. The +expected consumers, in rough order of generality: + +1. **Any atproto app working with Hypercerts / Hyperboards / Certified data + directly.** An OAuth client requests `include:org.hypercerts.authWrite` + (and/or the `org.hyperboards.*` or `app.certified.*` set) in its + authorization request; the user's PDS expands it and the app reads/writes + those collections in the user's own repo. No intermediary is involved. This + is the broadest case. +2. **The Certified group service (CGS), via OAuth.** CGS is used as a standard + OAuth resource (reached through AT Protocol service proxying). A client + calling CGS on a user's behalf requests the same sets; the user's PDS expands + them the same way. CGS is a likely-popular consumer, but it consumes the sets + like any other client — it does not own them. +3. **CGS API keys.** CGS also issues long-lived API keys whose scopes can be + specified as `include:`; CGS resolves the **same** published set and + expands it to concrete scopes on the key. (Detailed in CGS's own design doc; + summarised under _Consumption by CGS_ below.) + +The common thread: the **published Lexicon record is the single source of +truth.** Whatever resolves a set — a PDS during an OAuth grant, or CGS at +key-creation — reads the same published definition. + +## The sets + +We define three **record-collection write** sets — one per data namespace in +this repo. Each grants the three `repo:` write actions — create, update, delete — +over all of that namespace's record collections. + +| Set NSID | Grants write (create/update/delete) on | +| --------------------------- | ------------------------------------------ | +| `org.hypercerts.authWrite` | all `org.hypercerts.*` record collections | +| `org.hyperboards.authWrite` | all `org.hyperboards.*` record collections | +| `app.certified.authWrite` | all `app.certified.*` record collections | + +All three are authored and published from **this repository**, which is the +namespace authority for all of them — see _Namespace authority_ below. + +> **Write only — no read action.** The leaf is `authWrite` because a `repo:` +> permission has only three actions — `create`, `update`, `delete` (there is no +> `read` action in the scope grammar). Reading records needs **no permission at +> all**: atproto repo records are public and world-readable via +> `com.atproto.repo.getRecord` / `listRecords`, with no scope required. So these +> sets cover the only repo operations that _are_ permissioned — the writes — and +> "read" is implicitly available to everyone. The user-facing `title` / `detail` +> (the authorization-dialog consent copy) say "Manage your … data" / "Create, +> edit, and delete …" accordingly. +> +> **Naming — and why the NSID is a single segment under the namespace root.** +> Two constraints, one of them hard: +> +> 1. **The set NSID's authority must parent the collections it grants** (the +> namespace-authority rule below). `NSID.authority` is the reversed domain of +> _all but the last segment_, so a set named `org.hypercerts.permissions.crud` +> has authority `permissions.hypercerts.org`, which does **not** parent +> `org.hypercerts.claim.*` — `IncludeScope.toScopes` would silently return an +> empty list. The set NSID must therefore be a **single leaf directly under the +> namespace root** (`org.hypercerts.`), whose authority is `hypercerts.org`. +> 2. **Convention follows real precedent.** Bluesky publishes its own permission +> sets exactly this way — single segment under the namespace, `auth`-prefixed +> capability leaf: `app.bsky.authViewAll`, `app.bsky.authCreatePosts`, +> `app.bsky.authDeleteContent`, `app.bsky.authFullApp`. The spec's illustrative +> examples (`com.example.authFull`, `com.example.authBasicFeatures`) match. So +> `authWrite` mirrors that style: `auth`-prefixed, names the capability. + +### `org.hypercerts.authWrite` + +Enumerates every `type: "record"` collection under `org.hypercerts.*`: + +```jsonc +{ + "lexicon": 1, + "id": "org.hypercerts.authWrite", + "defs": { + "main": { + "type": "permission-set", + "title": "Manage your Hypercerts data", + "detail": "Create, edit, and delete your Hypercerts records (impact claims, evaluations, funding, and related data).", + "permissions": [ + { + "type": "permission", + "resource": "repo", + "collection": [ + "org.hypercerts.claim.activity", + "org.hypercerts.claim.contribution", + "org.hypercerts.claim.contributorInformation", + "org.hypercerts.claim.rights", + "org.hypercerts.collection", + "org.hypercerts.context.acknowledgement", + "org.hypercerts.context.attachment", + "org.hypercerts.context.evaluation", + "org.hypercerts.context.measurement", + "org.hypercerts.funding.receipt", + "org.hypercerts.workscope.tag", + ], + "action": ["create", "update", "delete"], + }, + ], + }, + }, +} +``` + +### `org.hyperboards.authWrite` + +Enumerates every `type: "record"` collection under `org.hyperboards.*`: + +```jsonc +{ + "lexicon": 1, + "id": "org.hyperboards.authWrite", + "defs": { + "main": { + "type": "permission-set", + "title": "Manage your Hyperboards data", + "detail": "Create, edit, and delete your Hyperboards records (board configurations and display profiles).", + "permissions": [ + { + "type": "permission", + "resource": "repo", + "collection": [ + "org.hyperboards.board", + "org.hyperboards.displayProfile", + ], + "action": ["create", "update", "delete"], + }, + ], + }, + }, +} +``` + +### `app.certified.authWrite` + +Enumerates every `type: "record"` collection under `app.certified.*`: + +```jsonc +{ + "lexicon": 1, + "id": "app.certified.authWrite", + "defs": { + "main": { + "type": "permission-set", + "title": "Manage your Certified data", + "detail": "Create, edit, and delete your Certified records (profile, badges, follows, wallet links, and related data).", + "permissions": [ + { + "type": "permission", + "resource": "repo", + "collection": [ + "app.certified.actor.organization", + "app.certified.actor.profile", + "app.certified.badge.award", + "app.certified.badge.definition", + "app.certified.badge.response", + "app.certified.graph.follow", + "app.certified.link.evm", + "app.certified.location", + "app.certified.signature.proof", + ], + "action": ["create", "update", "delete"], + }, + ], + }, + }, +} +``` + +> The collection lists are derived from this repo's `lexicons/org/hypercerts/`, +> `lexicons/org/hyperboards/`, and `lexicons/app/certified/` (`type: "record"` +> defs at time of writing). They must be **kept in sync** as record types are +> added — see _Maintenance_. + +## Design rules these sets follow + +### Namespace authority + +A permission set may only contain permissions under its **own** namespace +authority. Spec, verbatim: _"Permission sets are limited to expressing +permissions that reference resources under the same NSID namespace as the set +itself"_ — it may address its own NSID group and children, never siblings or +parents. The rule applies uniformly to `repo:` permissions (by collection NSID) +and `rpc:` permissions (by lxm NSID). + +Consequences here: + +- A set under `org.hypercerts.*` may grant `repo:org.hypercerts.*` collections, + and **only** those — it cannot reference `org.hyperboards.*` or + `app.certified.*` collections, or any third party's, or any service's `rpc:` + methods. Likewise each of the `org.hyperboards.*` and `app.certified.*` sets + is confined to its own namespace. +- This is why there are **three** sets rather than one combined set: the three + namespaces have distinct authorities (`hypercerts.org`, `hyperboards.org`, + `certified.app`) and cannot legally be mixed in a single set. An app that + needs more than one requests each `include:` scope. +- The authority confinement is also what makes a published set **safe for any + client to resolve**: an `include:` can only ever widen access to the set's own + namespace, never smuggle in foreign permissions. + +### Collections are enumerated, never wildcarded + +Inside a permission set, a `repo:` permission names collections by **exact +NSID**. Spec, verbatim: _"Wildcards are not supported in permissions within a +permission set."_ So a CRUD set cannot say "all `org.hypercerts.*` collections" — +it lists each one. The enumerated list **is** the grant, and is the security +boundary: a newly-added collection is not covered until it is added to the set +and the set is re-published. (Contrast a standalone scope outside a set, where +`repo:*` is permitted — but that is "all collections", not "a namespace", and is +not what these sets are for.) + +### `repo:` permissions carry no `aud` + +A `repo:` scope targets the **user's own repo**, so it has no audience. All +three sets here are `repo:`-only, so an `include:` for them is requested +**without** any `?aud=` parameter. Each set expands to a **single combined +`repo:` scope** that +lists every collection — `IncludeScope.toScopes` coalesces the permission's +collection array into one scope string +(`repo:?collection=&collection=&…&action=create&action=update&action=delete`), +rather than emitting one scope per collection. + +(For completeness: an `rpc:` permission _does_ take an `aud` naming the target +service, and a set can let the `include:`'s `?aud=` flow into `rpc:` permissions +marked `inheritAud: true`. Neither CRUD set uses `rpc:`, so this does not apply — +it is noted only because a future Hypercerts/Certified _service-method_ set +would.) + +## Publication and resolution + +All three sets are published like any other Lexicon schema in this repo (see +[`PUBLISHING.md`](../PUBLISHING.md)) and resolved at runtime via the standard +**Lexicon resolution system**, to which the permission spec defers: +_"Permission sets are Lexicon schemas and are published and fetched using the +Lexicon resolution system, which includes cryptographic authentication."_ + +The resolution chain a consumer follows (per +[`/specs/lexicon`](https://atproto.com/specs/lexicon), core protocol): + +1. **NSID → authority DID via DNS.** Reverse the NSID's authority portion (all + but the final name), prepend `_lexicon.`, query that domain for a `TXT` record + `did=`. +2. **DID → PDS endpoint** via standard DID resolution. +3. **Fetch the schema record**: `com.atproto.repo.getRecord` with + `repo=`, `collection=com.atproto.lexicon.schema`, + `rkey=` (the rkey **is** the NSID). +4. Validate the record's `main` def is `type: "permission-set"`, then expand. + +For a direct OAuth client this is done by the **user's PDS** during the grant. +The publication of the `_lexicon` DNS records and schema records for these +namespaces is a maintenance responsibility of this repo's operators. + +## Consumption by CGS + +CGS is expected to be a prominent consumer, in two ways. Neither changes the +sets — they are recorded here so the sets are designed with these uses in mind; +full detail lives in the CGS design doc +([`api-key-permission-sets.md`](https://github.com/hypercerts-org/certified-group-service/blob/main/docs/design/api-key-permission-sets.md)). + +- **Via OAuth + service proxying.** A client calls CGS on a user's behalf; the + user's PDS has already expanded the requested `include:` set into the grant. + CGS sees ordinary `repo:` scopes. Identical to any other OAuth resource. +- **Via CGS API keys.** CGS lets an owner specify an API key's scopes as + `include:`. CGS resolves the published set (same chain as above) and + expands it to concrete scopes stored on the key, **once at key-creation time** + (a snapshot — a later change to the set does not retroactively alter issued + keys until they are re-issued). CGS then gates its own write methods + (`app.certified.group.repo.*`, which proxy to the group's PDS) by matching the + request's collection against those `repo:` scopes. + +The point for _this_ repo: the sets must be **self-contained and +deployment-agnostic** — no CGS-specific assumptions baked into the definitions. +All three CRUD sets satisfy this trivially (pure `repo:` collection grants). + +## Maintenance + +The enumerated `collection` lists are the one ongoing obligation. Whenever a +`type: "record"` lexicon is added to or removed from `org.hypercerts.*`, +`org.hyperboards.*`, or `app.certified.*`, the corresponding set must be updated +and re-published, or the new collection will be silently uncovered by the set. +The `AGENTS.md` "Adding / modifying a lexicon" checklist calls this out. Worth a +check in the release process too (and ideally an automated test asserting each +set lists exactly the `type: "record"` defs in its namespace). + +### Growing a set silently widens existing OAuth grants + +Because we will keep adding lexicons, the key question is what happens to a grant +a user already accepted when the set is later re-published with **more** +collections. + +**The consent caveat, stated plainly:** on the OAuth path this widening is +**silent** — a user who granted `include:org.hypercerts.authWrite` when it +covered 11 collections gets write access to the 12th on their next token refresh, +with **no new consent prompt**. This is by design in AT Protocol: the user +consents to _the set_, an intentionally-mutable named bundle, not to a frozen +snapshot of its members. The refresh path (`rotateToken`) re-expands and re-issues +the widened scope without gating on consent (verified — there is no consent or +`authorizedScopes` re-check in that path). If that is not acceptable for a given +capability, do **not** add it to an existing set: publish it under a **new** set +NSID so clients must request it explicitly and users must re-consent. + +Two things bound how alarming this is in practice: + +1. **Newly-added permissions are highly correlated with what the user already + granted.** The [authority rule](#namespace-authority) forces every collection + in a set to live under the set's own namespace — so anything we can add to + `org.hypercerts.authWrite` is _another `org.hypercerts.*` write_, in the same + trust domain the user already opted into. The set is, in effect, a standing + consent to "write my Hypercerts data", and new Hypercerts record types are the + natural extension of that. A set can **never** silently pull in a _cross_- + namespace permission (e.g. `app.certified.*`) — that is structurally + impossible and always needs a separate `include:` and explicit re-consent. +2. It only ever _adds_; it can never silently _narrow_ or redirect an existing + grant. + +With that caveat understood, the mechanics differ for the two consumption paths: + +- **OAuth grants are dynamic — no re-auth, no `.vN` NSID.** The user's + Authorization Server (their PDS) persists the _raw_ `include:` scope in + the grant, not a frozen expansion of it, and **re-resolves the published set on + each token refresh**. So adding a collection to a published set widens every + existing session the next time it refreshes, without the user re-authorizing. + The permission spec makes this explicit: _"The permissions associated with an + Access Token should remain fixed, but when a client refreshes their tokens + (obtaining a new access token), the computed permissions for the session may be + updated to reflect changes to sets"_, so _"client software can then be updated + to take advantage of those new lexicons, without requiring users to + re-authenticate their sessions."_ In the reference PDS + ([`@atproto/oauth-provider`](https://github.com/bluesky-social/atproto/tree/main/packages/oauth/oauth-provider)) + this is the `rotateToken` path re-expanding `parameters.scope` via + `buildTokenScope`. Propagation is bounded by the Auth Server's resolved-set + cache (spec-recommended stale lifetime and firm upper bound: **24 h**) plus the + access-token lifetime (~15–30 min); bumping the client-metadata + `software_version` is the spec's cache-purge lever if a faster rollout is + needed. This holds only because every collection we add stays under the set's + **own namespace authority** (the [authority rule](#namespace-authority) both + permits it and is why no cross-namespace addition can ever go into one of these + sets — that still needs a separate `include:` and an explicit scope upgrade). + +- **CGS API keys are a frozen snapshot** (see [Consumption by CGS](#consumption-by-cgs)): + expansion happens once at key-creation time, so a re-published set does **not** + retroactively widen already-issued keys. Growing access there means re-issuing + the key — which is the intended workflow, since an API key's scope should be + auditable and stable for its lifetime. + +So a new NSID (`*.authWrite2` / `.v2`) is **not** needed to _add_ collections as +a mechanical matter — the platform is built to expand sets in place. Reach for a +new set NSID (or an explicit separate `include:`) only when the addition should +**not** be granted silently — i.e. when re-consent is the point — or to _remove +or attenuate_ existing permissions, which the spec advises doing "very sparingly" +because it silently narrows live sessions. For our sets, whose members are all +same-namespace CRUD, in-place growth is the expected and appropriate path. + +This holds for **ePDS**, our own Authorization Server, too. ePDS wraps a stock +[`@atproto/pds`](https://www.npmjs.com/package/@atproto/pds) / +[`@atproto/oauth-provider`](https://www.npmjs.com/package/@atproto/oauth-provider) +with a Better Auth login layer; it customises the authentication and consent +flow but does **not** fork or override token issuance or refresh. It persists the +raw `include:` scope (unexpanded) and delegates refresh to the provider's +stock `rotateToken`, which re-runs `buildTokenScope` on every rotation — so ePDS +re-expands the current published set exactly like the reference PDS. Any other +non-reference or forked Authorization Server should still be checked against this +before relying on the dynamic re-expansion. + +## References + +- AT Protocol permission spec — permission sets: + https://atproto.com/specs/permission#permission-sets +- AT Protocol lexicon resolution: https://atproto.com/specs/lexicon +- CGS consumption design: + https://github.com/hypercerts-org/certified-group-service/blob/main/docs/design/api-key-permission-sets.md diff --git a/lexicons/app/certified/authWrite.json b/lexicons/app/certified/authWrite.json new file mode 100644 index 00000000..7db71506 --- /dev/null +++ b/lexicons/app/certified/authWrite.json @@ -0,0 +1,29 @@ +{ + "lexicon": 1, + "id": "app.certified.authWrite", + "defs": { + "main": { + "type": "permission-set", + "title": "Manage your Certified data", + "detail": "Create, edit, and delete your Certified records (profile, badges, follows, wallet links, and related data).", + "permissions": [ + { + "type": "permission", + "resource": "repo", + "collection": [ + "app.certified.actor.organization", + "app.certified.actor.profile", + "app.certified.badge.award", + "app.certified.badge.definition", + "app.certified.badge.response", + "app.certified.graph.follow", + "app.certified.link.evm", + "app.certified.location", + "app.certified.signature.proof" + ], + "action": ["create", "update", "delete"] + } + ] + } + } +} diff --git a/lexicons/org/hyperboards/authWrite.json b/lexicons/org/hyperboards/authWrite.json new file mode 100644 index 00000000..6f7a3018 --- /dev/null +++ b/lexicons/org/hyperboards/authWrite.json @@ -0,0 +1,22 @@ +{ + "lexicon": 1, + "id": "org.hyperboards.authWrite", + "defs": { + "main": { + "type": "permission-set", + "title": "Manage your Hyperboards data", + "detail": "Create, edit, and delete your Hyperboards records (board configurations and display profiles).", + "permissions": [ + { + "type": "permission", + "resource": "repo", + "collection": [ + "org.hyperboards.board", + "org.hyperboards.displayProfile" + ], + "action": ["create", "update", "delete"] + } + ] + } + } +} diff --git a/lexicons/org/hypercerts/authWrite.json b/lexicons/org/hypercerts/authWrite.json new file mode 100644 index 00000000..2d5806a2 --- /dev/null +++ b/lexicons/org/hypercerts/authWrite.json @@ -0,0 +1,31 @@ +{ + "lexicon": 1, + "id": "org.hypercerts.authWrite", + "defs": { + "main": { + "type": "permission-set", + "title": "Manage your Hypercerts data", + "detail": "Create, edit, and delete your Hypercerts records (impact claims, evaluations, funding, and related data).", + "permissions": [ + { + "type": "permission", + "resource": "repo", + "collection": [ + "org.hypercerts.claim.activity", + "org.hypercerts.claim.contribution", + "org.hypercerts.claim.contributorInformation", + "org.hypercerts.claim.rights", + "org.hypercerts.collection", + "org.hypercerts.context.acknowledgement", + "org.hypercerts.context.attachment", + "org.hypercerts.context.evaluation", + "org.hypercerts.context.measurement", + "org.hypercerts.funding.receipt", + "org.hypercerts.workscope.tag" + ], + "action": ["create", "update", "delete"] + } + ] + } + } +} diff --git a/package.json b/package.json index 95bf35db..e8e9bf7f 100644 --- a/package.json +++ b/package.json @@ -42,11 +42,12 @@ "check": "npm run gen-api && npm run lint && npm run typecheck && npm run build && npm run test", "build": "rollup -c && npm run build:types", "build:types": "tsc --project tsconfig.build.json", - "gen-api": "find ./lexicons -name '*.json' | xargs lex gen-api --yes ./generated && npm run gen-index", - "gen-md": "find ./lexicons -name '*.json' | xargs lex gen-md --yes ./lexicons.md", + "//permissions": "permission-set lexicons are published as-is but have no TS shape; lex gen-* cannot codegen them. scripts/run-lex-codegen.js runs a lex subcommand over all lexicons EXCEPT permission-sets (excluded by content — the same check generate-exports.js uses). It is a single Node process that spawns lex with the file list as argv and forwards lex's exit code, so a producer failure propagates without `set -o pipefail` (not portable across npm's sh) and shell word-splitting is avoided. (argv is still bounded by the OS execve arg limit, ~2 MB on Linux — far above this list.)", + "gen-api": "node ./scripts/run-lex-codegen.js gen-api --yes ./generated && npm run gen-index", + "gen-md": "node ./scripts/run-lex-codegen.js gen-md --yes ./lexicons.md", "gen-schemas-md": "node ./scripts/generate-schemas.js", "//gen-ts": "UNUSED - use gen-api instead", - "gen-ts": "find ./lexicons -name '*.json' | xargs lex gen-ts-obj > generated/DO-NOT-USE-lexicons.ts", + "gen-ts": "node ./scripts/run-lex-codegen.js gen-ts-obj > generated/DO-NOT-USE-lexicons.ts", "gen-index": "node ./scripts/generate-exports.js", "lex": "lex", "test": "vitest run", diff --git a/scripts/generate-exports.js b/scripts/generate-exports.js index 34498011..332c0fd3 100644 --- a/scripts/generate-exports.js +++ b/scripts/generate-exports.js @@ -14,7 +14,7 @@ */ import { readdirSync, readFileSync, writeFileSync } from "node:fs"; -import { join, dirname, relative, parse } from "node:path"; +import { join, dirname, relative, parse, sep } from "node:path"; import { fileURLToPath } from "node:url"; import { execSync } from "node:child_process"; @@ -22,6 +22,22 @@ const __dirname = dirname(fileURLToPath(import.meta.url)); const projectRoot = join(__dirname, ".."); const lexiconsDir = join(projectRoot, "lexicons"); +/** + * Read and parse a lexicon JSON file, rethrowing parse failures with the + * offending file path so codegen errors are traceable (mirrors + * scripts/generate-schemas.js). + */ +function readLexicon(filePath) { + const content = readFileSync(join(lexiconsDir, filePath), "utf-8"); + + try { + return JSON.parse(content); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + throw new Error(`Failed to parse lexicon ${filePath}: ${message}`); + } +} + /** * Recursively find all JSON files in a directory */ @@ -34,7 +50,10 @@ function findJsonFiles(dir, baseDir = dir) { if (entry.isDirectory()) { results.push(...findJsonFiles(fullPath, baseDir)); } else if (entry.isFile() && entry.name.endsWith(".json")) { - const relativePath = relative(baseDir, fullPath); + // Normalize to POSIX separators: downstream helpers (pathToImportName, + // pathToNamespace, etc.) treat lexicon paths as POSIX and split on "/". + // On Windows, path.relative yields "\"-separated paths that would break them. + const relativePath = relative(baseDir, fullPath).split(sep).join("/"); results.push(relativePath); } } @@ -153,28 +172,31 @@ function pathToGeneratedType(filePath) { return `./types/${withoutExt}.js`; } -/** - * Read lexicon NSID from JSON file - */ -function readNsid(filePath) { - const fullPath = join(lexiconsDir, filePath); - const content = readFileSync(fullPath, "utf-8"); - const lexicon = JSON.parse(content); - return lexicon.lexicon === 1 ? lexicon.id : null; -} - /** * Generate generated/exports.ts content */ function generateIndex() { - const jsonFiles = findJsonFiles(lexiconsDir).sort(); - - const lexicons = jsonFiles.map((filePath) => ({ + // Parse each lexicon once, then derive everything from the parsed doc — both + // the permission-set check and the NSID — to avoid re-reading the same file. + const lexiconDocs = findJsonFiles(lexiconsDir) + .sort() + .map((filePath) => ({ + filePath, + doc: readLexicon(filePath), + })) + // Permission-set lexicons have no TS shape — lex gen-api cannot codegen them + // — so they are excluded from `generated/` here, by the SAME content check + // (`main.type === "permission-set"`) that scripts/run-lex-codegen.js + // uses to exclude them from the gen-* CLI invocations. Excluding by content + // (not by path) keeps the two in lock-step regardless of where set files live. + .filter(({ doc }) => doc?.defs?.main?.type !== "permission-set"); + + const lexicons = lexiconDocs.map(({ filePath, doc }) => ({ path: filePath, importName: pathToImportName(filePath), namespace: pathToNamespace(filePath), generatedTypePath: pathToGeneratedType(filePath), - nsid: readNsid(filePath), + nsid: doc.lexicon === 1 ? doc.id : null, })); // Filter out lexicons without NSIDs (should be rare) diff --git a/scripts/generate-schemas.js b/scripts/generate-schemas.js index db292fd8..0addb7e3 100644 --- a/scripts/generate-schemas.js +++ b/scripts/generate-schemas.js @@ -272,10 +272,48 @@ function generateDescription(description) { return description ? [`**Description:** ${description}`, ""] : []; } +function generatePermissionSetSection(mainDef) { + const output = []; + + if (mainDef.title) output.push(`**Title:** ${mainDef.title}`, ""); + if (mainDef.detail) output.push(`**Detail:** ${mainDef.detail}`, ""); + + const permissions = mainDef.permissions || []; + for (const permission of permissions) { + output.push(`**Resource:** \`${permission.resource}\``, ""); + + // repo permissions carry collection/action; rpc permissions carry + // lxm/aud/inheritAud. Render whichever fields are present so an rpc + // permission isn't reduced to a bare "**Resource:** `rpc`". + const collections = (permission.collection || []) + .map((c) => `\`${c}\``) + .join(", "); + const actions = (permission.action || []).map((a) => `\`${a}\``).join(", "); + if (collections) output.push(`**Collections:** ${collections}`, ""); + if (actions) output.push(`**Actions:** ${actions}`, ""); + + const lxm = (permission.lxm || []).map((l) => `\`${l}\``).join(", "); + if (lxm) output.push(`**LXM:** ${lxm}`, ""); + if (permission.aud) output.push(`**Audience:** \`${permission.aud}\``, ""); + if (permission.inheritAud !== undefined) { + output.push(`**Inherit audience:** \`${permission.inheritAud}\``, ""); + } + } + + return output; +} + function generateMainSection(mainDef, lexicon) { const output = []; - if (!mainDef) return output; + if (!mainDef) return { output, hasProperties: false, hasPropertyRows: false }; + + // Permission sets have no properties — render their title/detail/permissions + // so the section isn't blank in the reference. + if (mainDef.type === "permission-set") { + const permOutput = generatePermissionSetSection(mainDef); + return { output: permOutput, hasProperties: false, hasPropertyRows: false }; + } output.push(...generateDescription(mainDef.description)); diff --git a/scripts/run-lex-codegen.js b/scripts/run-lex-codegen.js new file mode 100644 index 00000000..fde5556f --- /dev/null +++ b/scripts/run-lex-codegen.js @@ -0,0 +1,90 @@ +#!/usr/bin/env node + +/** + * Run a `lex` codegen subcommand over every lexicon under `lexicons/` EXCEPT + * permission-set lexicons. + * + * `lex gen-api` / `gen-md` / `gen-ts` cannot codegen a `type: "permission-set"` + * def — it has no TypeScript shape and the CLI throws on it. Permission sets are + * published as-is, so they are excluded from codegen. The exclusion is by + * **content** (`defs.main.type === "permission-set"`), the same check + * `scripts/generate-exports.js` uses — so the two cannot drift as files move. + * + * Why a Node runner rather than a shell pipeline: + * - `lex … $(node …)` swallows a producer failure: command substitution + * discards the inner exit code, so lex runs on whatever (partial/empty) + * output it produced and may report success. + * - `node … | xargs -0 lex …` needs `set -o pipefail` to propagate a producer + * failure, and `pipefail` is NOT portable across the `sh` npm runs scripts + * with (rejected by the dash on GitHub's Ubuntu runners). + * This runner is a single process with one authoritative exit code: it forwards + * lex's exit code (and fails on an empty list or a spawn error). Passing the + * file list as argv also avoids shell word-splitting/quoting pitfalls. (It does + * not lift the OS `execve` argument-length limit — argv is still bounded — but + * that limit is ~2 MB on Linux, far above the few KB this list occupies.) + * + * Usage (in package.json): + * node ./scripts/run-lex-codegen.js gen-api --yes ./generated + * node ./scripts/run-lex-codegen.js gen-ts-obj > generated/DO-NOT-USE-lexicons.ts + */ + +import { readdirSync, readFileSync } from "node:fs"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { spawnSync } from "node:child_process"; + +const lexiconsDir = fileURLToPath(new URL("../lexicons", import.meta.url)); + +function findJsonFiles(dir) { + const out = []; + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const full = join(dir, entry.name); + if (entry.isDirectory()) out.push(...findJsonFiles(full)); + else if (entry.isFile() && entry.name.endsWith(".json")) out.push(full); + } + return out; +} + +function isPermissionSet(file) { + try { + return ( + JSON.parse(readFileSync(file, "utf-8"))?.defs?.main?.type === + "permission-set" + ); + } catch { + return false; // let lex surface a parse error rather than silently skip + } +} + +const lexArgs = process.argv.slice(2); +if (lexArgs.length === 0) { + process.stderr.write( + "run-lex-codegen: expected a lex subcommand, e.g. `gen-api --yes ./generated`\n", + ); + process.exit(2); +} + +const files = findJsonFiles(lexiconsDir) + .filter((f) => !isPermissionSet(f)) + .sort(); + +// Fail loudly rather than run lex on an empty list (which could "succeed" +// producing nothing). +if (files.length === 0) { + process.stderr.write("run-lex-codegen: no lexicons found\n"); + process.exit(1); +} + +// `shell: false` (the default) — args are passed as argv, so there is no shell +// word-splitting/quoting to get wrong, and lex's exit code is ours. (argv is +// still subject to the OS `execve` arg-length limit, but that is ~2 MB on Linux +// vs. the few KB this list occupies.) stdio inherited so `> file` redirection in +// the npm script still works. +const result = spawnSync("lex", [...lexArgs, ...files], { stdio: "inherit" }); +if (result.error) { + process.stderr.write( + `run-lex-codegen: failed to spawn lex: ${result.error.message}\n`, + ); + process.exit(1); +} +process.exit(result.status ?? 1);