Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# DID of your labeler account (e.g. did:plc:xxxx)
DID=did:plc:your-labeler-did

# Signing key for the labeler (hex-encoded private key)
# Private signing key for the labeler
SIGNING_KEY=your-signing-key

# Bluesky account credentials for the labeler bot
BSKY_IDENTIFIER=your-handle.bsky.social
BSKY_PASSWORD=your-app-password
# Labeler account credentials
LABELER_IDENTIFIER=orglabeler.certified.one
LABELER_PASSWORD=your-app-password

# Public HTTP port for Caddy. Hosted platforms usually set PORT automatically.
PORT=8080
Expand All @@ -29,12 +29,12 @@ ACTIVITY_DB_PATH=activity-log.db
# Path to the labels database (used by @skyware/labeler LabelerServer)
LABELS_DB_PATH=labels.db

# Public URL of the Caddy-fronted app/labeler endpoint (e.g. https://labeler.yourdomain.com)
# Public URL of the Caddy-fronted app/labeler endpoint (e.g. https://orglabeler.hypercerts.dev)
# /xrpc/* is routed to the labeler and dashboard/API routes are routed to Next.js.
NEXT_PUBLIC_LABELER_ENDPOINT=https://labeler.yourdomain.com
NEXT_PUBLIC_LABELER_ENDPOINT=https://orglabeler.hypercerts.dev

# PDS URL for the labeler account (auto-detected during setup, override if needed)
PDS_URL=https://climateai.org
PDS_URL=https://certified.one

# External Tap service URL. Tap runs as a separate service, not inside this app container.
TAP_URL=https://your-tap-service.example.com
Expand Down
37 changes: 21 additions & 16 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ This is the **Certified Organization Labeler** fork — an AT Protocol labeler f
```
src/
├── app/ # Next.js App Router (dashboard)
│ ├── api/ # API routes (read from activity-log.db)
│ ├── api/ # API routes (read from configured ACTIVITY_DB_PATH)
│ ├── feed/ # Feed page with tier filtering
│ ├── globals.css # Tailwind v4 design system (OKLCH)
│ ├── layout.tsx # Root layout with fonts + theme
│ └── page.tsx # Dashboard page
├── components/ # React components (all "use client")
├── components/ # React UI components (client components only where interactive)
├── labeler/ # Standalone labeler process (NO React/Next.js imports)
│ ├── start.ts # Entry point: spawns Tap sidecar + LabelerServer
│ ├── start.ts # Entry point: LabelerServer + workers + external Tap connection
│ ├── tap-consumer.ts # Tap client, SimpleIndexer record handler
│ ├── server.ts # Label application logic
│ ├── setup.ts # Account setup (PDS-aware)
Expand All @@ -34,26 +34,31 @@ src/
1. **src/labeler/ must NEVER import from React or Next.js** — it runs as a standalone Node.js process
2. **src/lib/ is shared** — importable by both labeler and Next.js, but must not import from src/labeler/
3. **src/components/ and src/app/ are Next.js only** — they can import from src/lib/ but never from src/labeler/
4. **Two databases**: labels.db (AT Proto, managed by @skyware/labeler) and activity-log.db (dashboard, managed by src/lib/db.ts)
4. **Two app databases**: `LABELS_DB_PATH` (AT Proto, managed by @skyware/labeler) and `ACTIVITY_DB_PATH` (dashboard, managed by src/lib/db.ts)
5. **Design system**: Tailwind CSS v4, OKLCH colors (hue 260), Syne display font, no component libraries
6. **Code style**: no semicolons, single quotes, 2-space indent

## Development

```bash
npm run dev:all # Start dashboard + labeler together
npm run build # Verify Next.js build passes
npm run reset # Clear databases for fresh start
npm run dev:service # Start dashboard + labeler together
npm run build # Verify Next.js build passes
npm run test # Run Node test suite
npm run reset # Clear local databases for fresh start
```

## Issue Tracking
## Documentation Maintenance

This project uses **hb** (heartbeads) for issue tracking.
When behavior, configuration, deployment, scripts, or operator workflow changes, check whether the relevant docs need updates in the same change.

```bash
hb ready # Find available work
hb show <id> # View issue details
hb update <id> --status in_progress # Claim work
hb close <id> # Complete work
hb sync # Sync with git
```
Current docs to consider:

- `README.md` — project overview, setup, scripts, env vars, production shape
- `DEPLOYMENT.md` — generic hosted deployment guidance
- `RAILWAY.md` — Railway-specific deployment guidance
- `AGENTS.md` — agent-facing architecture, workflow, and maintenance instructions
- `src/app/docs/page.tsx` — dashboard documentation shown in the app
- `.agents/skills/orglabeler/SKILL.md` — downstream consumer integration guidance
- `.env.example` — documented environment variable defaults/examples

If a new documentation file is introduced, add it to this list and include when it should be checked.
62 changes: 35 additions & 27 deletions DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,59 +40,67 @@ The direct Caddy route is required for `subscribeLabels`, because it is a WebSoc

## Environment variables

### Required for the running service
### Required for the running app service

- `DID` — labeler DID
- `SIGNING_KEY` — labeler private key used by `@skyware/labeler`

These are validated at startup by `src/lib/config.ts`.
- `DID` — labeler account DID
- `SIGNING_KEY` — private key material used by `@skyware/labeler` to sign labels
- `TAP_URL` — URL of the separate Tap service; there is no localhost fallback
- `NEXT_PUBLIC_LABELER_ENDPOINT` — public HTTPS base URL for the dashboard and labeler XRPC endpoint

### Required for first-time labeler setup / label sync

- `BSKY_IDENTIFIER`
- `BSKY_PASSWORD`
- `NEXT_PUBLIC_LABELER_ENDPOINT`

Use these with `npm run setup` and `npm run set-labels` when registering the labeler account and label definitions.

### Required for the app service to reach Tap
- `LABELER_IDENTIFIER` — labeler account identifier, for example `orglabeler.certified.one`
- `LABELER_PASSWORD` — labeler account password or app password
- `PDS_URL` — optional override for the labeler account PDS; setup normally resolves this from the DID document

- `TAP_URL` — required URL of the separate Tap service
Use these with `npm run setup` and `npm run set-labels` when registering the labeler account and label definitions. The labeler account identifier is the signing source; `NEXT_PUBLIC_LABELER_ENDPOINT` is the hosted app endpoint and may be a different domain.

### SQLite paths

Set these to persistent storage in hosted environments:

- `ACTIVITY_DB_PATH` (default `activity-log.db`)
- `LABELS_DB_PATH` (default `labels.db`)
- `ACTIVITY_DB_PATH` (code default `/data/activity-log.db`)
- `LABELS_DB_PATH` (code default `/data/labels.db`)

### Tap service storage
### Tap service settings

The Tap service manages its own persistent SQLite file and any Tap-specific settings outside this repo.

If your Tap deployment uses admin auth, configure `TAP_ADMIN_PASSWORD` on the Tap service itself.
If your Tap deployment uses admin auth, configure `TAP_ADMIN_PASSWORD` on the Tap service itself and set the same `TAP_ADMIN_PASSWORD` on the app service so health checks and the Tap WebSocket can authenticate.

### Optional
### Optional app settings

- `HF_TOKEN` — enables HuggingFace classification; if unset, HF scoring stays disabled
- `PDS_URL` — overrides PDS discovery from the DID document
- `HF_TOKEN` — enables Hugging Face classification; if unset, HF scoring stays disabled
- `HYPERSCAN_RECORD_URL_BASE` — base URL used when the dashboard links to source AT Protocol records; default `https://hyperscan.dev/data`
- `TEST_PDS_HOSTS` — comma-separated PDS hosts whose actors should always be labeled `likely-test`; URL enrichment waits for actor PDS resolution and skips matching test PDS hosts
- `PORT` — public Caddy HTTP port (hosted platforms usually set this; default fallback `8080`)
- `TRUSTED_PDS_HOSTS` — comma-separated PDS hosts whose actors receive the trusted-PDS score bonus; default `certified.one,gainforest.id`
- `TRUSTED_PDS_BONUS` — score points added for trusted actor PDS hosts; default `10`
- `PORT` — public Caddy HTTP port (hosted platforms usually set this; Caddy fallback `8080`)
- `NEXT_PORT` — internal Next.js HTTP port (`3000` by default)
- `HOST` — bind host for the labeler server (`127.0.0.1` is recommended when Caddy runs in the same container)
- `HOST` — bind host for the labeler server (`0.0.0.0` by default; `127.0.0.1` is recommended when only same-container Caddy should reach it)
- `LABELER_PORT` — internal labeler HTTP port (`4100` by default)
- `METRICS_PORT` — metrics port (`4101` by default)
- `RESET_DB=true` — deletes the SQLite files before startup
- `RESET_DB=true` — deletes configured app SQLite files plus WAL/SHM companions before startup; remove it after one reset
- `URL_ENRICHMENT_ENABLED` — enables async URL checks through `url_checks`; default `true`
- `URL_CHECK_INTERVAL_MS` — poll interval for processing due URL checks; default `1000`
- `URL_CHECK_DISCOVERY_INTERVAL_MS` — how often the URL worker scans local snapshots for newly referenced URLs; default `30000`
- `URL_CHECK_TIMEOUT_MS` — timeout for one URL resolution attempt; default `4000`
- `URL_CHECK_OK_TTL_MS` — freshness window for successful URL checks; default `604800000`
- `URL_CHECK_FAILED_TTL_MS` — downgrade window for hard failed URL checks before another attempt; default `86400000`
- `URL_CHECK_RETRY_BASE_MS` — initial retry delay for temporary URL check failures; default `300000`
- `URL_CHECK_MAX_RETRY_MS` — maximum retry delay for temporary URL check failures; default `3600000`
- `URL_CHECK_HARD_FAILURE_ATTEMPTS` — number of hard failures required before URL scoring removes resolve points; default `2`
- `URL_CHECK_MAX_URLS_PER_DID` — maximum profile/organization URLs cached and checked per DID; default `5`

## Persistence requirements

This fork uses two SQLite databases in the app service and one in the Tap service:

- `activity-log.db` — dashboard data and scoring history
- `labels.db` — AT Protocol label records
- `tap.db` — Tap cursor / replay state (Tap service)
- `ACTIVITY_DB_PATH` — dashboard data and scoring history
- `LABELS_DB_PATH` — AT Protocol label records
- Tap service database — Tap cursor / replay state

The app service should mount a volume for `activity-log.db` and `labels.db`. The Tap service should mount its own volume for `tap.db`. If the files are not persisted, redeploys will lose dashboard history and Tap will replay from an old or empty cursor.
The app service should mount a volume for the configured `ACTIVITY_DB_PATH` and `LABELS_DB_PATH`. The Tap service should mount its own volume for its database. If the files are not persisted, redeploys will lose dashboard history and Tap will replay from old or empty state.

Also persist the `-wal` and `-shm` companions that SQLite may create beside each DB file.

Expand Down
Loading
Loading