Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .changeset/docs-restructure-require-resolve.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
"@aws-blocks/blocks": patch
"@aws-blocks/create-blocks-app": patch
---

docs: per-block docs folders + committed BB catalog with CI sync check; CLAUDE/agents docs resolved via require.resolve

`@aws-blocks/blocks` now ships one docs folder per Building Block under `docs/<block>/`
(`README.md` / `API.md` / `DESIGN.md`), plus a committed, marker-delimited Building Block
catalog in the package README that a `sync-docs --check` CI gate keeps in sync. The README's
catalog section and the scaffolded `AGENTS.md` (`@aws-blocks/create-blocks-app`) now direct
tools and agents to locate docs programmatically via
`require.resolve('@aws-blocks/blocks/docs/<block>/README.md')` (and
`require.resolve('@aws-blocks/blocks/docs/README.md')` for the catalog) rather than assuming a
`node_modules/` path or following the human-facing relative links. Also adds a Security
Considerations section to the package README.
25 changes: 25 additions & 0 deletions .github/workflows/block-catalog-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Block Catalog Check

# Fails the PR if the generated Building Block catalog table in
# packages/blocks/README.md is out of date. The script only reads the committed
# package READMEs + root README (Node builtins, zero deps), so no `npm ci` needed.
# On failure its stderr tells the engineer to run `npm run sync-docs` and commit.

on:
pull_request:
branches: [main]

permissions:
contents: read

jobs:
check:
name: Catalog in sync
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with:
node-version-file: '.nvmrc'
- name: Verify Building Block catalog is in sync
run: node scripts/sync-catalog.mjs --check
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@
"create-demo": "node packages/create-blocks-app/dist/index.js --template demo",
"update-template": "tsx scripts/update-template-from-demo.ts",
"build:tech-design-pdf": "node scripts/build-tech-design-pdf.mjs",
"sync-docs": "node scripts/sync-catalog.mjs --write",
"sync-docs:check": "node scripts/sync-catalog.mjs --check",
"review-pr": "bash scripts/review-pr.sh",
"publish:local": "tsx scripts/publish/publish.ts",
"version": "changeset version && npm install --package-lock-only",
Expand Down
111 changes: 86 additions & 25 deletions packages/blocks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,31 +109,77 @@ onAuthChange(authApi, (user) => {

## Building Blocks

Each block is its own package; full per-block docs ship in this package under **`docs/<package-name>.md`**, and **`docs/index.md`** has a decision tree to help you pick.
Start from what you need:

- **Store data**
- Simple key → value (caches, flags, user prefs) → `KVStore` (bb-kv-store)
- Structured records with indexes and queries → `DistributedTable` (bb-distributed-table) — **default for most data**
- Relational / SQL (joins, transactions) → see [Choosing a data block](#choosing-a-data-block) below
- Files, blobs, uploads, static assets → `FileBucket` (bb-file-bucket)
- A single config value or secret → `AppSetting` (bb-app-setting)
- **Authenticate users**
- Username/password, prototypes/MVPs → `AuthBasic` (bb-auth-basic)
- Cognito user pools, MFA, groups → `AuthCognito` (bb-auth-cognito)
- External identity provider (OIDC) → `AuthOIDC` (bb-auth-oidc)
- **Run work outside the request/response**
- Fire-and-forget background jobs → `AsyncJob` (bb-async-job)
- Scheduled / recurring tasks → `CronJob` (bb-cron-job)
- **Push live updates to browsers** (chat, presence, dashboards) → `Realtime` (bb-realtime)
- **Build AI features**
- Agent with tool use + conversation → `Agent` (bb-agent)
- Semantic document retrieval (RAG) → `KnowledgeBase` (bb-knowledge-base)
- **Send transactional email** → `EmailClient` (bb-email-client)
- **Observe and operate**
- Structured logs → `Logger` (bb-logger)
- Custom metrics → `Metrics` (bb-metrics)
- Distributed traces → `Tracer` (bb-tracer)
- Auto CloudWatch dashboard → `Dashboard` (bb-dashboard)

### Choosing a data block

Default to `DistributedTable` for your data models unless your domain specifically requires SQL engine capabilities.

Reach for one of the SQL blocks when you need to filter or join results across more than one related record, filter models on many dimensions with no preset hierarchy, store large objects, require transactions, or otherwise need the flexibility or familiarity of SQL that NoSQL does not offer.

If you need SQL, prefer `DistributedDatabase` for basic Postgres-compatible querying. Use `Database` specifically when you need a full (more expensive) Postgres implementation where the engine itself provides and enforces foreign keys, row level security, triggers, views, large transactions (more than 3,000 rows), or integration with an existing Postgres database. Note it carries an idle cost at minimum 0.5 ACU, or a cold start when scaling from zero, unlike the other two blocks.

## Building Block documentation

Every Building Block ships its full docs — `README.md`, `API.md`, and `DESIGN.md` — inside the `@aws-blocks/blocks` package under `docs/<block>/`. To read them, locate the bundled folder:

| Building Block | Import | Use it for |
|---|---|---|
| `Scope` | `@aws-blocks/blocks` | Resource boundaries / grouping for your backend |
| `ApiNamespace` | `@aws-blocks/blocks` | Type-safe APIs wired to the frontend automatically |
| `KVStore` | `@aws-blocks/blocks` | Simple key-value get/put/delete (prefs, flags, caches) |
| `DistributedTable` | `@aws-blocks/blocks` | Structured data with indexes and queries — **default for most data** |
| `DistributedDatabase` | `@aws-blocks/blocks` | Serverless SQL (Aurora DSQL) — zero-ops, scales to zero |
| `Database` | `@aws-blocks/blocks` | Full PostgreSQL (Aurora) — FKs, RLS, triggers, or an existing DB |
| `AuthBasic` | `@aws-blocks/blocks` | Username/password auth for prototypes, internal tools, MVPs |
| `AuthCognito` | `@aws-blocks/blocks` | Cognito User Pools — MFA, groups, hosted identity |
| `AuthOIDC` | `@aws-blocks/blocks` | Sign-in gated by an external OIDC identity provider |
| `Realtime` | `@aws-blocks/blocks` | Push to browsers — chat, presence, live dashboards |
| `AsyncJob` | `@aws-blocks/blocks` | Fire-and-forget background work (emails, uploads, reports) |
| `CronJob` | `@aws-blocks/blocks` | Scheduled / recurring tasks |
| `FileBucket` | `@aws-blocks/blocks` | File storage — uploads, downloads, presigned URLs |
| `AppSetting` | `@aws-blocks/blocks` | A single config value or secret (flags, API keys) |
| `KnowledgeBase` | `@aws-blocks/blocks` | Semantic document retrieval / RAG (Bedrock + S3 Vectors) |
| `Agent` | `@aws-blocks/blocks` | AI agent — tool use, streaming, conversation persistence |
| `EmailClient` | `@aws-blocks/blocks` | Transactional email (SES) |
| `Logger` / `Metrics` / `Tracer` / `Dashboard` | `@aws-blocks/blocks` | Observability — structured logs, metrics, traces, CloudWatch dashboard |
| `Hosting` | `@aws-blocks/blocks` | Deploy a frontend (SPA / static / Next.js SSR) on CloudFront + S3 |

> **Not sure which data block?** Start with `DistributedTable` (DynamoDB). Reach for SQL only when you need joins across records, many-dimensional filtering, large transactions, or an existing Postgres database — `DistributedDatabase` for serverless Postgres, `Database` for full Aurora Postgres (FKs, RLS, triggers; carries idle cost / cold starts the other two don't). The full rationale is in `docs/index.md`.
```bash
node -p "require('path').dirname(require.resolve('@aws-blocks/blocks/docs/README.md'))"
```

If resolution fails, fall back to `node_modules/@aws-blocks/blocks/docs`. That folder holds this guide (`README.md`) plus one subfolder per block; the catalog below lists every block.

<!-- BEGIN:block-catalog -->
| Block | What it does | Keywords |
|-------|--------------|----------|
| auth-common | Shared interfaces and UI components for all AWS Blocks auth Building Blocks. | — |
| bb-agent | AI agent with streaming, tool calling, and conversation persistence. | — |
| bb-app-setting | A single application configuration value backed by SSM Parameter Store. | — |
| bb-async-job | Background job processing backed by SQS and Lambda. | queue, job, background, async, worker, submit, batch, retry, SQS |
| bb-auth-basic | Simple username/password authentication with JWT sessions, password policy, and optional code-confirmed signup and password reset. | — |
| bb-auth-cognito | Authentication backed by Amazon Cognito User Pools. | — |
| bb-auth-oidc | OIDC sign-in gate for AWS Blocks applications. | — |
| bb-cron-job | Scheduled task execution backed by EventBridge Scheduler and Lambda. | cron, schedule, timer, periodic, recurring, rate, EventBridge, background, interval |
| bb-dashboard | Auto-generated CloudWatch Dashboard for application observability. | — |
| bb-data | Full PostgreSQL database — provisions Aurora Serverless v2 by default, or connects to an existing PostgreSQL database (Supabase, Neon, etc.) via `fromExisting()`. | — |
| bb-distributed-data | Serverless SQL database backed by Amazon Aurora DSQL. | — |
| bb-distributed-table | Structured data storage backed by DynamoDB with secondary indexes and rich query capabilities. | — |
| bb-email-client | Transactional email sending via Amazon SES. | — |
| bb-file-bucket | File storage backed by Amazon S3. | — |
| bb-knowledge-base | Semantic document retrieval backed by Amazon Bedrock Knowledge Bases. | — |
| bb-kv-store | Simple key-value storage backed by DynamoDB. | — |
| bb-logger | Structured logging with consistent JSON format, log levels, and contextual metadata. | — |
| bb-metrics | Custom application metrics backed by Amazon CloudWatch (via Embedded Metric Format). | — |
| bb-realtime | Real-time pub/sub messaging backed by API Gateway WebSocket + DynamoDB. | — |
| bb-tracer | Distributed tracing backed by AWS X-Ray. | — |
| core | Core primitives for building full-stack applications with the AWS Blocks. | — |
| hosting | Low-level CDK L3 constructs for deploying web applications on AWS | — |
| pipeline | CDK Pipelines-based CI/CD construct for AWS Blocks applications. | — |
<!-- END:block-catalog -->

## Local development and deploying

Expand All @@ -144,6 +190,8 @@ Each block is its own package; full per-block docs ship in this package under **
| Data | persists to `.bb-data/` (delete to reset) | lives in AWS |
| Use for | rapid iteration, tests | pre-production validation against real services |

> **Deploying needs AWS credentials.** `npm run dev` is fully local (no creds). `npm run sandbox` and `npm run deploy` provision real AWS resources, so configure credentials first — e.g. `aws configure sso` + `aws sso login`, or `aws configure` (verify with `aws sts get-caller-identity`). Use **least-privilege** credentials scoped to the services your blocks deploy — not broad `Administrator` access.

`npm run deploy` does a full production deploy; `npm run sandbox:destroy` tears the sandbox down. The same backend code runs in all three — blocks switch implementations automatically.

## Testing
Expand Down Expand Up @@ -181,9 +229,22 @@ Run with `npm run test:e2e`. Write the test first, iterate against mocks until i
- **`Database` when `DistributedTable` would do** — Aurora costs more and has cold starts; reach for SQL only when you need it.
- **Curling REST-style paths** — there is no `GET /api/getData`. All calls are JSON-RPC to a single `POST /aws-blocks/api`; use the typed import instead.

## Security Considerations

- Use `await auth.requireAuth(context)` in every method that shouldn't be public — ApiNamespace methods are **unauthenticated by default**
- Use `new AppSetting(scope, id, { secret: true })` for API keys and credentials — never hardcode or use `.env` files
- Always attach a schema to KVStore/AppSetting that accepts user data — the RPC layer validates structure but not business logic
- Do not add broad `*` IAM policies — each Building Block already grants least-privilege scoped to its own resources
- Never change `blockPublicAccess` on FileBucket — serve public files through CloudFront instead
- Configure `CORS_ALLOWED_ORIGINS` explicitly for production — avoid wildcards
- For cross-domain deployments, pass `crossDomain: true` to auth constructors (enables `SameSite=None; Secure; Partitioned`)
- Enable `monitoring: { enabled: true, snsTopicArn: '...' }` on Hosting for production alerts
- Add WAF and API Gateway throttling via CDK for public-facing apps — not included by default
- Logger provides serialization safety (circular refs, type coercion) but does NOT redact sensitive content — never pass raw credentials, tokens, or secrets to Logger methods; sanitize context objects before logging

## Reference

- **Per-block documentation:** `docs/<package-name>.md` (e.g. `docs/bb-distributed-table.md`); `docs/index.md` for the catalog + decision tree.
- **Per-block documentation:** `docs/<block>/README.md` (overview), `docs/<block>/API.md` (full API reference), `docs/<block>/DESIGN.md` (architecture & rationale) — e.g. `docs/bb-distributed-table/README.md`. The catalog + decision tree live in `docs/README.md`.
- **UI components** (`@aws-blocks/blocks/ui`): `Authenticator`, `AuthenticatedContent`, `AccountMenuBar`, `onAuthChange`, `broadcastAuthChange` — framework-agnostic, return DOM nodes. See the `@aws-blocks/auth-common` README.
- **SSR** (`@aws-blocks/blocks/server`): `withAuth` forwards browser cookies to API calls during server rendering. See the `@aws-blocks/core` README.
- **Wire protocol & debugging:** the client is JSON-RPC 2.0 over a single endpoint — you should never call it directly. For `curl`-level troubleshooting, see [TROUBLESHOOTING.md](./TROUBLESHOOTING.md).
5 changes: 3 additions & 2 deletions packages/blocks/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,11 @@
"./utils": {
"types": "./dist/utils.d.ts",
"default": "./dist/utils.js"
}
},
"./docs/*": "./docs/*"
},
"scripts": {
"prebuild": "node ../../scripts/sync-block-docs.mjs",
"prebuild": "node ../../scripts/gen-block-docs.mjs",
"build": "tsc --build",
"test": "node --test dist/conditional-exports.test.js dist/vendorize-map.test.js"
},
Expand Down
6 changes: 2 additions & 4 deletions packages/create-blocks-app/resources/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,7 @@
- **Backend:** `aws-blocks/index.ts` — APIs, auth, data models
- **Frontend:** `src/` — imports backend APIs via `import { api } from 'aws-blocks'`
- **Tests:** `test/e2e.test.ts` — run with `npm run test:e2e`
- **Full guide:** `node_modules/@aws-blocks/blocks/README.md` — architecture, workflow, best practices, common mistakes
- **Block catalog + decision tree:** `node_modules/@aws-blocks/blocks/docs/index.md`
- **Per-block docs:** `node_modules/@aws-blocks/blocks/docs/<package-name>.md`
- **AWS Blocks docs** ship inside the `@aws-blocks/blocks` package. Find the docs folder once: `node -p "require('path').dirname(require.resolve('@aws-blocks/blocks/docs/README.md'))"` (fallback: `node_modules/@aws-blocks/blocks/docs`). Read everything relative to it: `README.md` (dev guide + catalog + decision tree — start here), then `<block>/README.md`, `<block>/API.md`, `<block>/DESIGN.md`.

## Workflow

Expand All @@ -19,7 +17,7 @@
## Rules

- **Use Building Blocks** for all persistence and cloud abstractions — never local files, in-memory arrays, or local databases.
- **Read block docs** at `node_modules/@aws-blocks/blocks/docs/<package-name>.md` before using a block.
- **Read block docs** before using a block — start with its `README.md`, then `API.md` / `DESIGN.md` as needed (see the **AWS Blocks docs** bullet above for where the docs folder lives).
- **The JSON-RPC transport is invisible** — do not construct RPC payloads manually. Import and call the typed API directly.

## Deploying (requires AWS credentials)
Expand Down
79 changes: 79 additions & 0 deletions scripts/gen-block-docs.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
#!/usr/bin/env node
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0

/**
* Generates the gitignored, shipped `docs/` artifact for @aws-blocks/blocks. Runs
* as the packages/blocks `prebuild` hook so the published package always carries
* fresh docs without dirtying any tracked file.
*
* It produces two things under packages/blocks/docs/:
* 1. Per-block folders docs/<pkg>/ — mirror every root-level *.md of each
* included package EXCEPT the ones in SKIP_MARKDOWN (CHANGELOG.md), so
* block-specific docs (README.md, API.md, DESIGN.md, ...) ship automatically.
* 2. docs/README.md — a verbatim copy of the committed packages/blocks/README.md.
*
* This script NEVER modifies packages/blocks/README.md. The committed catalog
* table inside that README is managed separately by scripts/sync-catalog.mjs
* (`npm run sync-docs`); run that and commit before building if you added or
* removed a block.
*
* No flag is required — the default run generates docs/. `--docs-only` is accepted
* as a harmless alias for the same behavior.
*
* Inclusion rule: every package under packages/ that has a README.md and is not in
* EXCLUDED. (The package-discovery logic is intentionally duplicated from
* sync-catalog.mjs — the two scripts are kept independent on purpose.)
*/

import { readFileSync, writeFileSync, mkdirSync, readdirSync, existsSync, rmSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';

const __dirname = dirname(fileURLToPath(import.meta.url));
const packagesDir = join(__dirname, '..', 'packages');
const outDir = join(packagesDir, 'blocks', 'docs');
const readmePath = join(packagesDir, 'blocks', 'README.md');

const EXCLUDED = new Set(['blocks', 'data-common', 'foundations', 'create-blocks-app']);

// Root-level markdown that should NOT be mirrored into the per-block doc folder.
const SKIP_MARKDOWN = new Set(['CHANGELOG.md']);

const packages = getPackages();

generatePerBlockDocs();
writeFileSync(join(outDir, 'README.md'), readFileSync(readmePath, 'utf-8'));

console.log(`Generated ${packages.length} block docs → packages/blocks/docs/`);

// ─── docs/ artifact ──────────────────────────────────────────────────────────

function generatePerBlockDocs() {
// Clean and recreate so removed blocks/files don't linger.
rmSync(outDir, { recursive: true, force: true });
mkdirSync(outDir, { recursive: true });

for (const pkg of packages) {
const pkgDir = join(packagesDir, pkg);
const blockOutDir = join(outDir, pkg);
mkdirSync(blockOutDir, { recursive: true });

// Mirror every root-level markdown file (README.md, API.md, DESIGN.md, and any
// block-specific docs) except the skipped ones, so new docs ship automatically.
const mdFiles = readdirSync(pkgDir, { withFileTypes: true }).filter(
(entry) => entry.isFile() && entry.name.endsWith('.md') && !SKIP_MARKDOWN.has(entry.name),
);
for (const entry of mdFiles) {
writeFileSync(join(blockOutDir, entry.name), readFileSync(join(pkgDir, entry.name), 'utf-8'));
}
}
}

// ─── Package discovery (duplicated from sync-catalog.mjs) ──────────────────────

function getPackages() {
return readdirSync(packagesDir).filter(
(name) => !name.startsWith('.') && !EXCLUDED.has(name) && existsSync(join(packagesDir, name, 'README.md')),
);
}
Loading
Loading