Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
287 lines (196 loc) · 11.9 KB

CONTRIBUTING.md

File metadata and controls

287 lines (196 loc) · 11.9 KB

Contribution Guidelines

This document assumes that you are using a MacOS system. The project has been set up with VSCode as the default editor.

Project Structure

This is a pnpm monorepo written in TypeScript with four packages:

  • extension/ – Chrome extension (React, Vite, MV3)
  • backend/ – Express.js API (LLM-based analysis)
  • shared/ – Shared TypeScript schemas and types
  • evals/ – LLM evaluation suite (accuracy, prompts)

Key Scripts (from project root)

Command Description
pnpm dev:extension Start extension dev server
pnpm dev:backend Start backend with hot reload
pnpm build Build all packages
pnpm build:extension Build extension only
pnpm test Run all unit tests
pnpm test:backend Run backend tests only
pnpm test:extension Run extension tests only
pnpm test:shared Run shared tests only
pnpm test:watch Run tests in watch mode
pnpm check Full pre-commit run (format, lint, typecheck, build)
pnpm start:backend Run built backend (after pnpm build)
pnpm eval Run detailed LLM evals
pnpm eval:latest Run detailed evals using prompts/latest.txt
pnpm eval:all Run all evals (snippets + pages)

Local Development Setup

Prerequisites

Before you begin, ensure you have the following installed:

  • Node.js (v22 or higher)
  • pnpm (v9.15.4 or higher)

To install Node.js, you can use a version manager like nvm (recommended) or Homebrew if you don't need to manage multiple versions. For more information, see the Node.js official download page.

If you already have npm installed but not pnpm, you can install it via:

npm install --global corepack@latest
corepack enable pnpm

If you don't have npm installed, you can install pnpm via:

brew install pnpm

Clone and Install

  1. Clone the repository:

    git clone <repository-url> patch
cd patch

  2. Install all dependencies:

    pnpm install

    This installs dependencies for the root workspace and all packages (extension, backend, shared, evals).

Running the Development Servers

Extension (Vite dev server):

pnpm dev:extension

Backend (Express API with hot reload):

pnpm dev:backend

The backend runs at http://localhost:3000 by default.

Configure the LLM backend (.env)

The backend uses an OpenAI‑compatible LLM (e.g., xAI Grok). Copy the example env file and fill in your own secrets:

cp backend/.env.example backend/.env

Update backend/.env with your credentials:

# backend/.env
# OpenAI-compatible endpoint for Grok
LLM_BASE_URL=https://api.x.ai/v1
LLL_MODEL=grok-4-1-fast-non-reasoning  # or grok-4-latest, etc.
LLM_API_KEY=your_xai_api_key_here      # do NOT commit real keys

Notes:

  • Do not commit real secrets. Keep backend/.env out of version control (see .gitignore). Share secrets via your team’s secret manager, not git.
  • backend/.env.example documents the variables we expect; keep it in sync when adding new env keys.

Testing the /analyze endpoint

With the backend running (pnpm dev:backend) and .env configured, you can call the API directly:

curl -s -X POST http://localhost:3000/analyze \
  -H 'Content-Type: application/json' \
  -d '{"domSnippet":"<html><body>Hello</body></html>"}' | jq .

The response is JSON conforming to AnalysisResponseSchema in shared/src/analyze/response.ts (patterns array with type, severity, description, evidence, fixes (array of { selector, strategy, fix }), and confidence).

Loading the Extension in Chrome

  1. Build the extension:

    pnpm dev:extension

  2. Open Chrome and navigate to chrome://extensions/

  3. Enable Developer mode (toggle in the top right)

  4. Click Load unpacked

  5. Select the extension/dist folder

After making changes, click the refresh icon on the extension card in chrome://extensions/. For hot-reloading to work, you must keep the dev:extension script running.

Building the Extension

pnpm build:extension

The extension will be built in the extension/dist folder. You can then load it in Chrome by following the steps above. Note that this is a one-time build and will not be updated automatically.

Running Evaluations

The evals/ package measures how accurately the LLM detects dark patterns. There are two eval modes:

Mode Command What it measures
Detailed pnpm eval Field-level accuracy: severity, selectors, evidence, fix quality
Latest pnpm eval:latest Same as detailed but always uses prompts/latest.txt
Standard pnpm eval:pages Pattern type and count correctness only
Snippets pnpm eval:snippets Single-pattern HTML snippets (fast, no fixture files)
All pnpm eval:all Both snippet and page standard evals

Prerequisites

The evals call the LLM API directly — no backend server needs to be running. You only need backend/.env configured with a valid LLM_API_KEY:

cp backend/.env.example backend/.env
# then set LLM_API_KEY in backend/.env

Then:

pnpm eval          # detailed eval (default, recommended)
pnpm eval -- -v    # detailed eval with verbose field-level output

Dataset

  • Snippet cases: evals/dataset/snippets/<pattern>.json — single-pattern inline HTML snippets
  • Page cases: evals/dataset/pages/complex-pages.json — full-page fixture tests (pattern type + count)
  • Detailed page cases: evals/dataset/pages/marketplace.json — field-level expected patterns (severity, evidence, selectors, fixes)

See evals/README.md for full details on adding test cases, dataset format, and prompt engineering.

Severity guidelines for dataset entries

When adding or updating expected patterns in the dataset, align severity with the system prompt's definitions:

  • "high" — direct financial harm: hidden subscriptions, pre-selected paid add-ons, deceptive checkout flows
  • "medium" — pressure without direct financial harm: countdown timers, scarcity badges, confirmshaming
  • "low" — mild/borderline: minor visual tricks, social proof claims, informational friction

Evidence guidelines

Write evidence as the text or HTML element a user would see in the rendered page, not as JavaScript source code. For example:

"evidence": "<span id=\"timer-1\">Deal ends in [MM:SS]</span>"

not

"evidence": "productTimers[1] = Math.floor(Math.random() * 3600) + 300"

Pre-Commit Checklist

Before committing, run:

pnpm check

This runs format check, lint, dependency checks (knip), typecheck, build, and tests. Simple fixes:

pnpm lint:fix        # lint
pnpm format          # format
pnpm check-deps:fix  # knip fixes

Ensure all checks pass before creating a merge request. For testing details, see TESTING.md.

Basic Branching Workflow

You should NEVER push directly to the main branch. Create a new branch and follow the steps below:

  1. All new branches should be associated with a task. You can create a task from the "Issues" section on GitLab.

  2. Create a new branch from the task's page on GitLab. Copy the branch name after creating it.

  3. Run git fetch origin and git checkout <branch-name> to switch to the new branch.

  4. Make and test your changes locally.

  5. Go through the Pre-Commit Checklist to check for errors.

  6. Commit your changes to the remote repository.

  7. To resolve potential conflicts with the main branch, make sure you rebase your branch onto the latest main branch:

    git config pull.rebase true  # (you only need to do this the first time)
git pull origin main

    You will need to resolve any conflicts that arise during the rebase either by editing the files then running git rebase --continue or by running git rebase --abort then starting over.

  8. Create a merge request to the main branch on GitLab.

  9. Request for a review on the MR and await approval before merging.

  10. Delete the branch after the MR is merged.

CI/CD Pipelines

When you open a merge request, GitLab CI runs a pipeline that only triggers on merge_request_event. You can see the pipeline status and logs on the MR page.

The pipeline has three stages. Jobs use the default image cimg/node:24.13 unless noted.

Stage 1: Prepare

  • install – Cleans, installs dependencies with pnpm install --frozen-lockfile, and pushes the cache (.pnpm-store, node_modules). All other jobs depend on this and pull the cache.

Stage 2: Linting

  • format – pnpm format:check
  • lint – pnpm lint
  • dependencies – pnpm check-deps (knip)
  • build – pnpm build; produces artifacts (shared/dist, backend/dist, extension/dist) for downstream jobs
  • typecheck – pnpm typecheck (needs install + build)

Stage 3: Testing

  • test-backend – Backend unit tests with coverage; Cobertura and JUnit artifacts (2 weeks)
  • test-extension – Extension unit tests with coverage; Cobertura and JUnit artifacts (2 weeks)
  • test-shared – Shared package unit tests with coverage; Cobertura and JUnit artifacts (2 weeks)
  • test-e2e – E2E tests with Puppeteer (pnpm test:e2e). Installs Puppeteer’s official Debian dependencies plus chromium; on Linux arm64 (e.g. Docker on Apple Silicon) uses system Chromium since Chrome doesn’t ship an arm64 Linux binary. Needs install + build.

Coverage is reported in the MR; JUnit and coverage artifacts are kept for 2 weeks (unit tests only).

The pipeline must pass before the merge request is merged. If it fails, fix the issues locally (re-run pnpm check and the relevant test commands as needed), push your changes, and the pipeline will run again.

For this project, we use self-hosted GitLab runners. If there are no runners active, the pipelines will not run when you open a merge request. When this happens, you can either contact someone who has a runner set up or configure your own runner by following the instructions below.

Configuring your own GitLab Runner

  1. Install GitLab Runner on your machine using the instructions here.
  2. (Optional but recommended) Install Docker on your machine if it is not already installed. This makes replicating the runner environment and cleaning up pipeline artifacts easier. For MacOS, you can either use the official Docker Desktop, OrbStack, or any other Docker alternative such as Colima, Rancher Desktop, or Podman.
  3. On the project's GitLab page, navigate to Settings > CI/CD > Runners.
  4. Click on Create Project Runner. All the fields can be left empty.
  5. Click on Create Runner.
  6. Copy the runner registration command.
  7. Run the registration command to register the runner. If this doesn't work, you can try to register the runner manually by running gitlab-runner register and following the prompts.
  8. Start the runner by running gitlab-runner run whenever you need to run a pipeline.