ContributionPulse

Privacy-first contribution verification for developers with private repositories.

ContributionPulse connects to GitLab, Azure DevOps, and GitHub using read-only tokens, computes daily contribution aggregates, and exposes proof-style dashboards/reports without storing source-level metadata.

Product goals

Verify contribution activity from private repositories.
Keep user data privacy-safe by design.
Support shareable public reports and PDF export.
Scale sync via background jobs and retries.

Core principles

Store only day-level aggregate counts (commitCount, mergeCount, prCount, pipelineCount).
Never store code, diffs, commit messages, repository names, or raw provider payloads.
Encrypt provider credentials at rest with AES-256-GCM.
Keep all secret handling server-side only.

Tech stack

Frontend/App shell: Next.js 14 (App Router), TypeScript, Tailwind, shadcn/ui
Auth: Supabase Auth (email magic link)
Database: PostgreSQL + Prisma
Queue/worker: BullMQ + Redis
Charts: Recharts
Client data/state: React Query (server communication), Zustand (UI state)

System architecture

High-level component diagram

flowchart LR
  U[User Browser] --> W[Next.js App]
  W --> SA[Supabase Auth]
  W --> API[App Route Handlers]
  API --> DB[(Postgres via Prisma)]
  API --> Q[(BullMQ Queue on Redis)]
  WRK[Worker Process] --> Q
  WRK --> GL[GitLab APIs]
  WRK --> AZ[Azure DevOps APIs]
  WRK --> GH[GitHub APIs]
  WRK --> DB
  WRK --> PUB[(Redis Pub/Sub)]
  API --> PUB
  U <-->|SSE /api/sync/events| API

Runtime boundaries

Web app process: UI rendering, API route handling, auth checks, report generation.
Worker process: all sync jobs, provider API calls, aggregation, upserts.
Redis: queue broker + realtime pub/sub channel.
Postgres: tenant-scoped persistent storage.

End-to-end flow

1) Auth + tenant resolution

User signs in via Supabase magic link.
Server calls requireAppUser() for protected pages/routes.
App user record is upserted by supabaseUserId.
All operations are scoped by appUser.id.

2) Onboarding

User submits provider credentials (GitLab/GitHub token, Azure token + org).
API encrypts token using AES-256-GCM (MASTER_KEY).
Integration is upserted per unique (userId, provider).
Optional author-email aliases are stored for commit matching.

3) Sync

User clicks Sync now or queues historical backfill.
API enqueues BullMQ sync-user job.
Worker pulls job and iterates user integrations.
For each integration:
- mark syncState=RUNNING
- decrypt token in memory
- fetch provider events with pagination + retries + pacing
- aggregate to UTC day buckets
- upsert into DailyActivity
- set syncState=IDLE, lastSyncedAt=now
On failure: mark syncState=FAILED, emit sanitized error logs.

4) Realtime job status

Worker publishes sync lifecycle events (started/completed/failed) to Redis pub/sub.
Browser listens through SSE endpoint /api/sync/events.
Dashboard shows toasts and refreshes updated data.

5) Dashboard/report

Dashboard reads only aggregate rows (DailyActivity).
React Query handles all API calls (sync, backfill, shares, highlights, settings actions).
Public report uses tokenized, read-only route; no private metadata is exposed.
PDF export renders aggregate-only proof.

Detailed sync design

Sync sequence diagram

sequenceDiagram
  autonumber
  participant UI as Browser UI
  participant API as Next.js API
  participant Q as BullMQ/Redis
  participant W as Worker
  participant P as Provider APIs
  participant DB as Postgres
  participant SSE as SSE Stream

  UI->>API: POST /api/sync (or /api/sync/backfill)
  API->>Q: enqueue sync-user {userId, options}
  API-->>UI: 200 {ok:true}

  W->>Q: consume job
  W->>SSE: publish sync_started
  SSE-->>UI: event: sync_started

  W->>DB: load user + integrations
  loop each provider integration
    W->>DB: set syncState=RUNNING
    W->>P: fetch paginated events with retry/rate pacing
    W->>W: aggregate events by UTC day
    W->>DB: upsert DailyActivity(user,provider,date)
    W->>DB: set syncState=IDLE,lastSyncedAt
  end

  W->>SSE: publish sync_completed
  SSE-->>UI: event: sync_completed
  UI->>API: refresh dashboard data

Retry, pagination, and rate limiting

Retries transient failures (429, 5xx) with incremental backoff.
Supports both page-based and continuation-token pagination.
Applies per-host minimum interval (minIntervalMs) between outbound requests.

Backfill

Backfill submits fixed year range (from=Jan 1, to=Dec 31) as job options.
Jobs are listed, retried, deleted, or cleaned from the dashboard.

Data model

ER diagram

erDiagram
  User ||--o{ Integration : has
  User ||--o{ DailyActivity : has
  User ||--o{ ManualHighlight : has
  User ||--o{ PublicShare : has
  User ||--o{ SyncJob : has

  User {
    string id PK
    string supabaseUserId UK
    string email
  }

  Integration {
    string id PK
    string userId FK
    enum provider
    string encryptedToken
    string tokenIv
    string tokenTag
    string gitlabBaseUrl
    string azureOrg
    string[] authorEmails
    enum syncState
    datetime lastSyncedAt
  }

  DailyActivity {
    string id PK
    string userId FK
    enum provider
    datetime date
    int commitCount
    int mergeCount
    int prCount
    int pipelineCount
  }

  ManualHighlight {
    string id PK
    string userId FK
    datetime date
    string note
  }

  PublicShare {
    string id PK
    string userId FK
    string token UK
    datetime expiresAt
    datetime revokedAt
  }

  SyncJob {
    string id PK
    string userId FK
    enum provider NULL
    datetime from NULL
    datetime to NULL
    int backfillYear NULL
    enum status
    int attemptCount
    int maxAttempts
    datetime availableAt
    datetime lockedAt NULL
    datetime startedAt NULL
    datetime finishedAt NULL
    string errorMessage NULL
    datetime createdAt
    datetime updatedAt
  }

Privacy boundaries in data model

Integration keeps encrypted secrets only.
DailyActivity is aggregate-only.
SyncJob stores orchestration metadata only (no provider raw payloads, no repo names, no commit messages).
No model stores repository names or commit messages.

Supabase sync job lifecycle (when `SYNC_QUEUE_BACKEND=supabase`)

QUEUED: job waiting for processing (availableAt <= now means runnable).
RUNNING: worker has lock and is executing provider sync.
COMPLETED: sync finished successfully, finishedAt set.
FAILED: retries exhausted or non-retryable failure, errorMessage recorded.

Retry/backoff behavior:

attemptCount increments when a worker locks a queued job.
if attemptCount < maxAttempts, job is re-queued with future availableAt.
backoff is exponential and capped (current implementation caps at 60 seconds).

API architecture

App route groups

/api/integrations/* -> connect/update/disconnect providers
/api/sync + /api/sync/backfill* -> queue operations
/api/sync/events -> SSE stream for realtime sync status
/api/share -> create/revoke public report links
/api/highlights -> manual highlight CRUD (currently create)
/api/report/pdf/[token] -> PDF export
/api/account/delete -> account/data deletion

Client communication pattern

React Query handles all client-initiated API communication.
Mutations update local UI state and trigger selective refresh.
Zustand handles non-server UI state (chart filters/year, etc.).

Security architecture

Credential encryption

Algorithm: AES-256-GCM
Key source: MASTER_KEY env var (base64, 32 bytes)
Token encryption on write, decryption only in worker sync path.

Logging safety

All structured logs go through sanitization.
Token-like values are redacted before output.

Secret exposure controls

Provider secrets are never sent to browser.
Sensitive operations run in server routes/worker only.

Public report controls

Tokenized URL
Optional expiration timestamp
Revocation support

Multi-tenant design

Primary tenant key is userId.
Uniques enforce per-tenant isolation (userId + provider, userId + provider + date).
Every query/mutation path uses authenticated appUser.id.

Provider integration notes

GitLab

Supports gitlab.com and self-hosted base URL.
Uses events API to derive commits/merges/pipelines aggregates.

Azure DevOps

Requires PAT + organization name.
Traverses projects/repositories/commits.
Skips inaccessible repos (403/404) without failing whole sync.

GitHub

Uses user/repos, commits, pulls, and workflow runs endpoints.
Handles empty/inaccessible repositories gracefully (e.g., 409/404/403 skip paths).

Local development

Prerequisites

Node.js 20+ (project currently targets modern runtime)
PostgreSQL
Redis
Supabase project (URL + anon + service role key)

Environment variables

Copy .env.example to .env.local and set:

DATABASE_URL
DIRECT_URL
NEXT_PUBLIC_SUPABASE_URL
NEXT_PUBLIC_SUPABASE_ANON_KEY
SUPABASE_SERVICE_ROLE_KEY
MASTER_KEY (base64-encoded 32-byte key)
REDIS_URL
SYNC_QUEUE_BACKEND (bull or supabase)
CRON_SECRET (required when SYNC_QUEUE_BACKEND=supabase)
APP_URL
NEXT_PUBLIC_APP_NAME (optional, default ContributionPulse)
NEXT_PUBLIC_APP_SLUG (optional, default derived from app name)

Generate MASTER_KEY:

openssl rand -base64 32

Install and run

npm install
npm run prisma:generate
npx prisma migrate deploy
npm run dev

Run worker(s)

npm run worker
npm run worker:nightly

worker:nightly registers the repeatable nightly sync schedule. Run it once per environment.

Supabase queue mode (no Redis worker required)

Replace BullMQ/Redis solution for low-cost environments:

Set:

SYNC_QUEUE_BACKEND=supabase
CRON_SECRET=<strong-random-secret>

Do not run npm run worker for queue processing.
Trigger queue processing by calling:

POST /api/internal/sync/process
Authorization: Bearer <CRON_SECRET>

Process manually in dev:

npm run queue:process

Schedule this endpoint with Supabase Cron (example every minute):

select
  cron.schedule(
    'process-contribution-sync-queue',
    '* * * * *',
    $$
    select
      net.http_post(
        url := 'https://contribution-pulse.vercel.app/api/internal/sync/process?limit=10',
        headers := jsonb_build_object(
          'Content-Type', 'application/json',
          'Authorization', 'Bearer CRON_SECRET'
        ),
        body := '{}'::jsonb
      );
    $$
  );

This keeps BullMQ code in the project for future scale, while allowing Redis-free operation today.

Testing

npm test

Current automated tests include:

encryption helper correctness (AES-256-GCM)
daily aggregation/upsert logic

Deployment topology

Web service: Next.js app
Worker service: BullMQ worker process
Scheduler service/job: worker:nightly (or one-time bootstrap)
Managed Postgres (e.g., Supabase Postgres)
Managed Redis

Required GitHub repository secrets

Set these in GitHub repo settings -> Secrets and variables -> Actions:

VERCEL_TOKEN
VERCEL_ORG_ID
VERCEL_PROJECT_ID
DATABASE_URL
DIRECT_URL
NEXT_PUBLIC_SUPABASE_URL
NEXT_PUBLIC_SUPABASE_ANON_KEY
SUPABASE_SERVICE_ROLE_KEY
MASTER_KEY
REDIS_URL
APP_URL

Docker deployment

Files added

Dockerfile -> production image for Next.js web app
docker-compose.yml -> web + worker + Redis local/prod-like stack
.dockerignore -> optimized build context

Docker architecture diagram

flowchart LR
  U[Browser] --> WEB[web container<br/>Next.js]
  WEB --> DB[(Postgres)]
  WEB --> R[(Redis)]
  WEB --> SA[Supabase Auth]
  WEB --> Q[(BullMQ queue)]
  WRK[worker container<br/>npm run worker] --> Q
  WRK --> DB
  WRK --> GL[GitLab API]
  WRK --> AZ[Azure DevOps API]
  WRK --> GH[GitHub API]
  WRK --> R

Run with Docker Compose

docker compose up --build -d

Services:

web -> app on http://localhost:3000
worker -> background sync worker
redis -> queue broker + pub/sub

Stop:

docker compose down

Notes

docker-compose.yml uses .env for app secrets.
Keep DATABASE_URL pointing to your managed Postgres (or add a Postgres service if desired).
Run npm run worker:nightly once per environment to register the repeatable nightly sync schedule.

Name		Name	Last commit message	Last commit date
Latest commit History 21 Commits
.github/workflows		.github/workflows
prisma		prisma
scripts		scripts
src		src
tests		tests
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
.nvmrc		.nvmrc
Dockerfile		Dockerfile
README.md		README.md
components.json		components.json
docker-compose.yml		docker-compose.yml
middleware.ts		middleware.ts
next-env.d.ts		next-env.d.ts
next.config.mjs		next.config.mjs
package-lock.json		package-lock.json
package.json		package.json
postcss.config.js		postcss.config.js
tailwind.config.ts		tailwind.config.ts
tsconfig.json		tsconfig.json
tsconfig.tsbuildinfo		tsconfig.tsbuildinfo
vitest.config.ts		vitest.config.ts

Folders and files

Latest commit

History

Repository files navigation

ContributionPulse

Product goals

Core principles

Tech stack

System architecture

High-level component diagram

Runtime boundaries

End-to-end flow

1) Auth + tenant resolution

2) Onboarding

3) Sync

4) Realtime job status

5) Dashboard/report

Detailed sync design

Sync sequence diagram

Retry, pagination, and rate limiting

Backfill

Data model

ER diagram

Privacy boundaries in data model

Supabase sync job lifecycle (when SYNC_QUEUE_BACKEND=supabase)

API architecture

App route groups

Client communication pattern

Security architecture

Credential encryption

Logging safety

Secret exposure controls

Public report controls

Multi-tenant design

Provider integration notes

GitLab

Azure DevOps

GitHub

Local development

Prerequisites

Environment variables

Install and run

Run worker(s)

Supabase queue mode (no Redis worker required)

Testing

Deployment topology

Required GitHub repository secrets

Docker deployment

Files added

Docker architecture diagram

Run with Docker Compose

Notes

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Supabase sync job lifecycle (when `SYNC_QUEUE_BACKEND=supabase`)

Packages