Commit dbaea9c
authored
feat(admin): install sandboxed plugins from the experimental registry (#1011)
* fix(deps): catalog-pin zod so trusted plugins typecheck
Astro bundles its own Zod and re-exports it as 'astro/zod'. Trusted
plugins like @emdash-cms/plugin-forms import their route schemas via
'astro/zod', then pass those schemas to definePlugin() in core. With
emdash's 'zod: ^4.3.5' resolving independently of Astro's caret,
pnpm kept two Zod 4 patches in the tree (e.g. 4.3.6 alongside 4.4.1).
Zod 4 embeds its semver in the type system, so two patches of Zod 4
are not assignable to each other. The forms plugin's route schemas
(ZodObject<..., $strip>) were rejected by PluginRoute<TInput>['input']
(ZodType<unknown, unknown, $ZodTypeInternals<unknown, unknown>>) with
'Type "3" is not assignable to type "4"' on the internal version
field. The native definePlugin overload silently failed, TS fell
through to the StandardPluginDefinition overload, and reported a
misleading 'id does not exist' error -- masking 8 cascading errors.
Catalog-pinning Zod forces a single workspace-wide instance and
restores normal overload resolution. No code changes needed in core
or plugins/forms.
Also adds a pnpm-workspace.yaml comment explaining the gotcha so the
next person doesn't bump emdash's pin past Astro's range.
* feat(registry): experimental decentralized plugin registry
Adds opt-in support for installing sandboxed plugins from the
decentralized plugin registry described in RFC #694. Enabled via
`experimental.registry.aggregatorUrl` in the EmDash integration
options; when set, the admin UI replaces marketplace browse/install
with the registry path.
Server: new install handler (RFC verification chain), endpoint at
POST /_emdash/api/admin/plugins/registry/install, migration 038 adds
`source = 'registry'` plus `registry_publisher_did` /
`registry_slug` columns on `_plugin_state`, runtime sync split into
shared marketplace + registry tiers via a normalized opaque
`r_<hash>` plugin id.
Browser: aggregator XRPC calls go direct from the admin UI via
@emdash-cms/registry-client. Install POST runs through the server.
Includes a minimum-release-age policy with a per-publisher exclude
allowlist, enforced both client-side (UX) and server-side (gate).
Hardening (5 rounds of adversarial review): bundle id rewritten to
the derived pluginId before storage, aggregator identity
cross-checked, artifact and aggregator URLs validated for SSRF
(https-only in prod, IPv6 brackets handled), per-request and total
budgets on every outbound call, decompressed bundle capped at 256
KiB to match the RFC publish-time limit, migration 038 idempotent
on both SQLite and Postgres.
Known gaps tracked for follow-up: full MST signature verification
against the publisher's PDS, multibase multihash decoding (hex SHA-256
is accepted today), registry plugin update + uninstall handlers.
* fix(registry-lexicons): drop codegen from build script
The generated lexicon types are committed to git so consumers don't
need the codegen toolchain. Running lex-cli generate as part of the
default build pipeline broke Cloudflare Pages builds for sites that
pull registry-lexicons in transitively, because lex-cli imports
lex.config.ts directly and Node in the CF Pages build environment
can't load .ts natively.
Codegen moves to a separate `regen` script (`pnpm regen` runs
codegen + full build). Maintainers run it when they edit the
lexicons; consumers just consume the committed output.
* fix(registry): copilot review fixes
- Drift check normalizes capabilities (filter strings, dedupe, sort) on
both browser and server so reorderings or junk entries can't trigger
spurious rejection. Adds a shared normalizeCapabilities helper in
registry/config.ts and a mirror in admin/lib/api/registry.ts.
- RegistryPluginDetail no longer trusts the aggregator-supplied
ext?.capabilities as already-validated string[]; runs it through
normalizeCapabilities before display and before send.
- Fix stale '32 MiB' docstring on extractBundle (cap is actually
MAX_DECOMPRESSED_BUNDLE_BYTES = 256 KiB).
- Fix plugin-id.ts JSDoc: validatePluginIdentifier regex is
/^[a-z][a-z0-9_-]*$/ (allows hyphens); the prior 'cannot collide
with marketplace ids' claim was too strong and is now framed as
'syntactically distinct, plus an explicit pre-existing-row check
in the install handler.'
* fix(registry): address review findings + CI failures
CI fixes:
- Rename normalizeCapabilities -> canonicalCapabilitiesForDriftCheck
to avoid namespace clash with the existing capability normalizer
exported from @emdash-cms/plugin-types via core's index. The old
name shadowed plugin-types' helper at the top level of core's dist,
which made the definePlugin() overload set look ambiguous to TS in
plugins/forms and caused a typecheck cascade there.
- [...seen].toSorted() instead of [...seen].sort() to clear the
e18e/prefer-spread-syntax + unicorn/no-array-sort lint errors.
Review findings (ask-bonk[bot]):
- HIGH: drift check tripped on every install when the release record's
extension was empty. The browser now omits acknowledgedDeclaredAccess
when capabilities is empty, opting out of the server-side drift gate
for the (currently common) case where publishers haven't filled in
the extension block. The bundle's real capabilities are still bound
to the checksum-verified bytes.
- HIGH: DID-only publishers (no resolvable handle) could be linked from
the browse grid but never installed because the server rejects
handles without a '.'. Cards now render as non-interactive with a
'Publisher handle unresolved' badge; the detail page surfaces a
matching warning and disables Install.
- MEDIUM: registry-enabled sites were unconditionally routing existing
marketplace plugin detail URLs to RegistryPluginDetail, breaking deep
links. Detail-route selection now discriminates by param shape
(pluginId.includes('/')) rather than the manifest flag.
- MEDIUM: state-row write failure after storeBundleInR2 left orphan
bundles. Best-effort cleanup in the catch via deleteBundleFromR2.
- LOW: parseDurationSeconds runs on the user-supplied integration
option per install (not the already-normalized manifest shape). Wrap
in try/catch and surface as REGISTRY_POLICY_INVALID rather than
letting it bubble to a generic INSTALL_FAILED.
- LOW: validator-pattern doc drift in plugin-id.ts (already fixed in
the prior commit).
* fix(registry): move registry config types to their own module
The new RegistryConfig + ExperimentalConfig interfaces lived alongside
definePlugin's overloads in astro/integration/runtime.ts. tsdown +
rolldown's chunking decided to inline a bigger subset of plugin-related
types into the entry chunk as a result, which broke definePlugin()
overload resolution for trusted plugins building against core's dist
on CI (plugins/forms failed with 'id does not exist in type
StandardPluginDefinition').
Move both types to packages/core/src/registry/types.ts (still re-exported
from runtime.ts for backwards compatibility) so the chunking matches
main's layout and definePlugin's overloads resolve as before.
* fix(registry): wire up real-world install + display polish
Aggregator (apps/aggregator):
- Add CORS to /xrpc/* so the admin UI can call it from any origin
(preflight 204, response headers on every method). Aggregator is a
public read-only service; * is correct here.
Core (packages/core):
- Implement multibase-multihash checksum verification by re-encoding
our SHA-256 digest in the same 'b<base32>' shape the registry CLI
produces, rather than decoding the publisher's checksum. Same trust
contract, no base32 decoder needed. Bare hex SHA-256 still accepted
as a convenience fallback.
- Switch install handler to take 'did' (not handle) so packages whose
handle the aggregator couldn't resolve are still installable. The
browser resolves handle→DID via the aggregator before posting and
sends DID directly; the server skips resolvePackage and goes
straight to getPackage.
- Coerce 'experimental.registry' bare-string shorthand into the full
RegistryConfig object via 'coerceRegistryConfig'. 'registry:
"..."' is now equivalent to 'registry: { aggregatorUrl: "..." }'.
- Plumb 'experimental' through the integration's serializableConfig
so the manifest endpoint actually sees the user's registry block.
Previously it was being stripped, so the admin UI never branched to
the registry path.
- Split RegistryConfig + ExperimentalConfig types into their own
module (registry/types.ts) so they don't get bundled into the
astro/integration/runtime.ts dist chunk -- the wider inlining was
breaking definePlugin overload resolution for trusted plugins
building against core's dist.
Admin (packages/admin):
- New <PublisherHandle> component + usePublisherHandle hook with
tri-state result ('ok' / 'invalid' / 'missing'). Renders @handle,
'Unverified publisher' (red), or DID respectively. Uses
@atcute/identity-resolver's LocalActorResolver for bidirectional
handle verification, localStorage-cached for 24h.
- Detail page disables install on 'invalid' status (publisher claims
a handle that doesn't round-trip back to its DID -- impersonation
risk). Surfaces 'We couldn't verify this publisher's identity'
alert in plain language.
- Detail page reads installed state from fetchPlugins() and swaps
the Install button to 'Installed' (disabled) when the package
already has a 'source = "registry"' row matching its DID + slug.
React Query's existing ['plugins'] invalidation handles the
post-install UI update.
- Browse cards reuse <PublisherHandle> (variant='card') and link by
handle when available, DID otherwise. Detail page parses either
form from the URL.
- Browser sends 'did' (not handle) in the install POST.
Workspace:
- '@cloudflare/kumo' moved to the pnpm catalog and bumped to ^1.16.0
workspace-wide. Older 1.10.0 was missing Sidebar export and being
hoisted into the admin via packages/blocks's transitive dep.
- Add '@atcute/multibase' to core (for checksum encoding) and
'@atcute/identity-resolver' to admin (for DID->handle resolution).
- Update DEFAULT_AGGREGATOR_URL + DiscoveryClient doc example from
'experimental-registry.emdashcms.com' to 'registry.emdashcms.com'
(the actual production host).
* fix(registry): adversarial review round 6 findings
Addresses 7 findings from the round-6 adversarial review and
documents the eighth.
#1 (high) Capability consent bypass [registry.ts, RegistryPluginDetail.tsx]
The drift check was gated on the client sending acknowledgedDeclaredAccess.
If the publisher's release record had no extension, the admin saw an
empty permission dialog, omitted the acknowledgement, and the server
skipped the check entirely -- letting a bundle whose manifest declares
real capabilities slip through behind an empty consent UI. Server now
extracts capabilities from the bundle manifest after download and
refuses with DECLARED_ACCESS_REQUIRED if the bundle declares any
capabilities and no acknowledgement was sent. Client always sends the
list (empty when no extension) so the new server check is always armed.
#2 (high) Concurrent install bundle deletion [registry.ts]
Two parallel installs of the same (did, slug, version) both passed the
pre-existing-row check, both uploaded to the same deterministic R2
prefix, and one then won the state-row PK race. The loser's catch block
deleted the R2 bundle the winner had just written. On state-write
failure we now re-query the state row: if a winner exists, we lost the
race and must not touch the R2 bundle. Cleanup runs only when the
failure is a real DB error, not a lost concurrent install.
#3 (high) SSRF via DNS-resolving public hostnames [registry.ts, ssrf.ts moved]
Literal-IP blocklist alone left a DNS-rebinding gap: any public DNS
service resolving an attacker-chosen hostname to loopback / RFC1918 /
169.254.169.254 passed the URL check. The import pipeline already
shipped resolveAndValidateExternalUrl which does Cloudflare DoH
resolution and rejects on any forbidden resolved address; reuse it
for artifact downloads. Move src/import/ssrf.ts to src/security/ssrf.ts
to reflect that it's not import-specific. Leave a re-export shim at
the old path so 13 existing callers keep working unchanged. Add
#security/* path alias.
#5 (high) Aggregator-supplied handles treated as verified [PublisherHandle.tsx]
usePublisherHandle returned status: 'ok' with the aggregator-supplied
handle whenever one was present, skipping local DID->handle round-trip.
A compromised aggregator could label an attacker DID as e.g.
'stripe.com' and the UI would render it as verified. Always run
LocalActorResolver via resolveDidToHandle; use the aggregator handle
only for a cross-check. If the aggregator's claim differs from the
verified handle, mark the publisher invalid.
#6 (medium) Postgres migration 038 schema-qualification [038_registry_plugin_state.ts]
The columns probe queried information_schema.columns without filtering
by table_schema. A _plugin_state table in another schema (multi-tenant
Postgres, per-test schemas) could make the migration skip the column
adds. Filter by table_schema = current_schema().
#7 (medium) Install errors leak full artifact URLs [registry.ts]
fetchArtifact recorded each full URL in the joined error message that
bubbled up to the admin client. Artifacts hosted on storage backends
often carry presigned tokens in the query string; failed installs were
leaking those into HTTP responses and logs. Strip query and fragment
when building client-visible errors (origin + path only); log the full
URL server-side for debugging.
#8 (medium) Credentialed aggregator URLs accepted [config.ts]
validateAggregatorUrl accepted https://user:pass@example.com.
The normalized URL ends up in the admin manifest and is shipped to
every admin browser; browser fetch() also rejects credentialed URLs
outright. Reject them at config-validation time.
#4 (high, documented not fixed) Aggregator-trust-root scope [types.ts]
Full MST proof / publisher signature verification is not in this PR;
the server still trusts the aggregator-supplied (did, slug, checksum,
artifact URL). Expand the JSDoc on EmDashConfig.experimental.registry
to spell out exactly what the v1 trust contract is, what EmDash does
verify independently (checksum, manifest id/version/capabilities), and
what it doesn't (release-record signatures, replay). Recommendation:
point aggregatorUrl only at an aggregator you operate or trust at
centralized-source level until signature verification lands.
* fix(registry): adversarial review round 7 followups
Two LOW findings from the round-7 review (PR #1011 comment).
NSID exact-match in RegistryPluginDetail.tsx
Round-6 left a startsWith() match on the release-extension key.
RFC 0001 fixes the NSID for the release extension; accepting prefix
variants (...releaseExtensionV2, ...releaseExtension.deprecated)
would let a publisher render a different capability list than the
canonical key would. Use exact-equality keyed lookup.
Registry plugin uninstall affordance in PluginManager.tsx
Registry-installed plugins appear in PluginManager but the Uninstall
button is gated on isMarketplace. Admins see a permanent-looking
install with no way to remove it short of editing the DB and R2 by
hand. Add an inline note for source === 'registry' rows that says
uninstall isn't available yet and points the admin at the disable
toggle. Full uninstall handler lands in a follow-up PR.1 parent 5681eb2 commit dbaea9c
48 files changed
Lines changed: 4449 additions & 675 deletions
File tree
- .changeset
- apps/aggregator/src/routes/xrpc
- infra/blog-demo
- packages
- admin
- src
- components
- lib/api
- auth-atproto
- blocks
- playground
- core
- src
- api/handlers
- astro
- integration
- routes/api/admin/plugins
- [id]
- registry
- database
- migrations
- import
- plugins
- registry
- security
- tests/integration/database
- plugins
- ai-moderation
- field-kit
- forms
- registry-client/src/discovery
- registry-cli/src
- registry-lexicons
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
43 | 72 | | |
44 | 73 | | |
45 | 74 | | |
| |||
48 | 77 | | |
49 | 78 | | |
50 | 79 | | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
51 | 89 | | |
52 | | - | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
53 | 98 | | |
54 | 99 | | |
55 | 100 | | |
| |||
63 | 108 | | |
64 | 109 | | |
65 | 110 | | |
| 111 | + | |
66 | 112 | | |
67 | 113 | | |
68 | 114 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
25 | 27 | | |
26 | 28 | | |
27 | 29 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
35 | 37 | | |
36 | 38 | | |
37 | 39 | | |
38 | 40 | | |
| 41 | + | |
| 42 | + | |
39 | 43 | | |
40 | 44 | | |
41 | 45 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
229 | 229 | | |
230 | 230 | | |
231 | 231 | | |
| 232 | + | |
232 | 233 | | |
233 | 234 | | |
234 | 235 | | |
| |||
495 | 496 | | |
496 | 497 | | |
497 | 498 | | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
498 | 508 | | |
499 | 509 | | |
500 | 510 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
0 commit comments