Skip to content

Commit 1baff25

Browse files
feat(umami-postgres): keploy compat lane sample (smoke-test only) (#96)
* feat(umami-postgres): keploy compat lane sample (scaffold) Mirrors the doccano-django sample shape: the sample owns orchestration (compose / bootstrap / traffic / coverage), keploy CI lanes consume it as a thin wrapper. This is a SCAFFOLD — the full traffic loop driven by the existing keploy/enterprise lane (`run_api_flow` in .ci/scripts/umami-linux.sh) needs to be ported into flow.sh::umami_record_traffic in a follow-up. The current loop is deliberately minimal (heartbeat / me / teams / websites CRUD) which is enough to prove the sample boots end-to-end without keploy. Layout: Dockerfile — pin to umami:postgresql-v2.18.1 docker-compose.yml — postgres-15 + umami v2, env-driven flow.sh — bootstrap | record-traffic | coverage | list-routes keploy.yml.template — globalNoise for createdAt/updatedAt/uuid id README.md — handoff + status notes Signed-off-by: Akash Kumar <meakash7902@gmail.com> * feat(umami-postgres): port full umami v2 traffic loop Replace the bootstrap-only stub in flow.sh::umami_record_traffic with the complete umami v2 API drive that the keploy compat lanes need to gate against on a record/replay round-trip. The sample now owns the entire traffic loop end-to-end; consuming lanes wrap `bootstrap | record-traffic | coverage` inside `keploy record` / `keploy test` and add no curls of their own. Surfaces driven by record-traffic: * auth: /api/auth/login (via bootstrap), /api/auth/verify, /api/auth/logout * identity: /api/me, /api/me/teams, /api/me/websites * admin: /api/admin/users, /api/admin/websites, /api/admin/teams (incl. paged + search variants) * users CRUD: POST /api/users, GET /api/users/{id}, POST /api/users/{id} (update), GET /api/users/{id}/websites, GET /api/users/{id}/teams * websites CRUD: POST /api/websites, GET /api/websites (paged), GET /api/websites/{id}, POST /api/websites/{id} (update), GET /api/websites/{id}/active, GET /api/websites/{id}/daterange, POST /api/websites/{id}/reset * events ingest: POST /api/send (event + identify variants), POST /api/batch * sessions deep-dive: GET /api/websites/{id}/sessions[, /stats, /weekly, /{sessionId}, /{sessionId}/activity, /{sessionId}/properties, /{sessionId}/replays], GET /api/websites/{id}/replays, GET /api/websites/{id}/session-data/properties * analytics: stats, pageviews (multiple unit/timezone variants), events (series/stats), event-data[/stats], values, realtime, metrics (path / referrer / browser / os / device / country / event + search/limit variants), metrics/expanded * reports: every type umami v2 ships — breakdown, goal, funnel, journey, retention, utm, attribution, performance — plus saved-report CRUD (create, read, update, delete) and the listing endpoints * teams CRUD lifecycle: POST/GET/POST(update)/DELETE on /api/teams/{id}, member attach/list/detach via /api/teams/{id}/users[/{userId}] * share tokens: POST /api/websites/{id}/shares + GET /api/share/{shareId} (unauthenticated public-share access) * boards: full CRUD + /api/boards/{id}/shares * pixel tracker: GET /api/pixels * heartbeat 405 path: POST /api/heartbeat Total: 78 distinct (method, path) tuples fired per record-traffic run. Resource ids/names are fixed UUIDs / deterministic strings so request bodies stay byte-stable across record/replay (keeps keploy's body equality check passing without per-field globalNoise entries). Each call goes through a small umami_http() helper that logs the (method, url) tuple to UMAMI_FIRED_ROUTES_FILE and tolerates non-2xx (|| true) so a single endpoint regression in umami itself does not abort the whole record run — keploy is the assertion layer at replay. Also strips the SCAFFOLD/handoff/follow-up language from flow.sh and README.md: the sample is now the complete reproducer, no out-of-tree porting remains. Signed-off-by: Akash Kumar <meakash7902@gmail.com> * ci(umami-postgres): add per-sample coverage gate workflow Adds a GitHub Actions workflow scoped via paths: filter to umami-postgres/** so it triggers ONLY on PRs and main-branch pushes that touch the umami-postgres sample (or the workflow file itself). Other samples in this repo keep their orthogonal CI; gating the whole repo on every umami change would slow them all down for no benefit. Three jobs: * build-coverage — runs the sample end-to-end against the PR's HEAD ref via flow.sh bootstrap + record-traffic, captures the route- coverage percentage from flow.sh coverage. * release-coverage — same end-to-end against the PR's base ref. Has a first-PR bootstrap escape hatch (sample-existed=false → coverage=0) so the introducing PR doesn't fail for lack of a baseline. * coverage-gate — fails the PR if build-coverage drops more than COVERAGE_THRESHOLD percentage points below release-coverage. Default 1.0pp; overridable via the UMAMI_COVERAGE_THRESHOLD repo variable. Sticky PR comment summarises the diff. The gate runs ONLY here, on the sample repo. The enterprise PR pipeline (.woodpecker/umami-linux.yml) calls flow.sh coverage informationally with || true and does NOT gate on coverage — that separation keeps the enterprise lane decoupled from sample- level coverage drift. Helper script .github/workflows/scripts/run-and-measure.sh is the keploy-independent measurement shared by both build- and release-coverage jobs: two-phase compose boot (UMAMI_SKIP_INIT=0 then =1) matching the lane scripts, then flow.sh bootstrap + record-traffic + coverage with UMAMI_FIRED_ROUTES_FILE wired in as the standalone numerator. Signed-off-by: Akash Kumar <meakash7902@gmail.com> * fix(umami-postgres): read route surface from compiled Next.js manifest The upstream umami image (ghcr.io/umami-software/umami:postgresql-v2.18.1) ships a compiled Next.js build, not the TypeScript source. The prior implementation greped src/app/api/**/route.ts inside the container, which doesn't exist there, so umami_list_routes returned zero rows and umami_report_coverage skipped with "WARNING: ...skipping coverage report". The route surface is fully derivable from the build artefacts: /app/.next/app-path-routes-manifest.json → URL paths /app/.next/server/app<url>/route.js → compiled handlers with method exports ({GET:...,POST:...}) Verified end-to-end against the running container: list-routes now emits 93 (method, path) rows; coverage gate has a real denominator. Signed-off-by: Akash Kumar <meakash7902@gmail.com> * ci(umami-postgres): drop coverage gate — upstream image is precompiled+minified The upstream `ghcr.io/umami-software/umami:postgresql-v2.18.1` image ships a heavily minified Next.js standalone build under /app/.next/server/app/api/**/route.js. The source tree (/app/src) and sourcemaps (.map) are stripped from the image. V8 / c8 line coverage on minified code is structurally meaningless — each "line" of the compiled output is many source statements concatenated by the bundler, so a coverage percentage doesn't map back to anything a reviewer can act on. Rather than ship a misleading metric (the prior route-surface "coverage" we removed elsewhere was exactly this kind of proxy), the umami sample is now smoke-test-only: - `flow.sh bootstrap` signs in as admin, persists the JWT - `flow.sh record-traffic` exercises the v2 API surface - `flow.sh coverage` is a no-op that prints an info message and exits 0 (so consumers' `flow.sh coverage || true` calls keep working) The keploy/enterprise compat lane already uses the resulting record/replay assertions as its correctness gate — that IS the meaningful test here, not source coverage of umami's frontend. If real source-line coverage becomes a hard requirement for this sample, the path is to rebuild umami from source inside a Dockerfile.coverage overlay (~5-10 min npm install + next build without minification + with sourcemaps). That's a separate ~hours-of-work change. Removed: - .github/workflows/umami-postgres.yml (coverage gate workflow) - .github/workflows/scripts/run-and-measure.sh (its helper) - umami_list_routes / umami_list_recorded_routes / the legacy route-surface umami_report_coverage in flow.sh. - list-routes subcommand. Replaced umami_report_coverage with a no-op stub. Signed-off-by: Akash Kumar <meakash7902@gmail.com> * docs(umami-postgres): document why coverage is not measured (precompiled image) Signed-off-by: Akash Kumar <meakash7902@gmail.com> * fix(umami-postgres): plumb SKIP_DB_CHECK + nest globalNoise schema The umami compat lane (consumed by keploy/enterprise PR #1889 for the parse-server / umami / restheart / doccano three-cell matrix) was failing all three umami cells with `getaddrinfo EAI_AGAIN binaries.prisma.sh` mid-startup. Two sample-side bugs feeding into the lane: 1. SKIP_DB_CHECK wasn't surfaced through the compose umami's scripts/check-db.js fetches the prisma `schema-engine` binary on every startup unless SKIP_DB_CHECK is set. Once keploy's DNS interception is on (record/replay phase), the fetch fails and umami crashes before serving HTTP. The compose exposed UMAMI_SKIP_INIT but not SKIP_DB_CHECK, so the lane had no way to suppress the second-phase prisma fetch. Fix: surface `SKIP_DB_CHECK: "${SKIP_DB_CHECK:-}"`. The default is empty (not "0") because check-db.js does a JS truthiness check `if (process.env.SKIP_DB_CHECK)` — the literal string "0" is truthy in JS and skips, defeating the gate. Empty string is falsy. Lane scripts can now render the bootstrap-phase compose with SKIP_DB_CHECK="" (migrations run against the fresh volume) and the keploy-phase compose with SKIP_DB_CHECK="1" (skip, volume already has the migrations + admin user). 2. globalNoise was the wrong shape keploy's GlobalNoise config type is `map[string]map[string][]string` — outer key is the section (`header` / `body`), inner key is the field name within that section, value is the list of values to ignore (empty list == match anything). The flat dotted form `header.Date: []` / `body.createdAt: []` doesn't parse into that shape — keploy's yaml decoder silently dropped it, so the lane had no effective noise filter. The only reason existing samples didn't notice is that keploy auto-stamps Date as per-test noise on every recording, masking the schema bug. Restructured to the correct nested form. Caught `body.timestamp` (Date.now() inside /api/realtime that --freezeTime can't intercept through V8) along the way — was the last failing test in the otherwise-clean replay run. Validation against keploy/enterprise's lane (concurrent 3-cell matrix): * record-stable-replay-pr (cell A): 104/104 PASSED * record-pr-replay-pr (cell B): 104/104 PASSED * record-pr-replay-stable (cell C): 104/104 PASSED Signed-off-by: Akash Kumar <meakash7902@gmail.com> --------- Signed-off-by: Akash Kumar <meakash7902@gmail.com>
1 parent 5094f03 commit 1baff25

5 files changed

Lines changed: 684 additions & 0 deletions

File tree

umami-postgres/Dockerfile

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Thin wrapper around umami's official image at the version this
2+
# sample tracks. Pin lives here (not in CI lane scripts) so a
3+
# future umami release that changes the bug-triggering shape is a
4+
# one-line retag, not a hunt across keploy/integrations and
5+
# keploy/enterprise.
6+
#
7+
# Upstream: https://github.com/umami-software/umami
8+
# Image: docker.io/umamisoftware/umami:postgresql-v2.18.1
9+
FROM ghcr.io/umami-software/umami:postgresql-v2.18.1

umami-postgres/README.md

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# umami-postgres — keploy compat lane sample
2+
3+
Reproducer for the umami / postgres-v3 compat lane. Mirrors the architectural pattern of the [doccano-django sample in `samples-python`](https://github.com/keploy/samples-python/tree/main/doccano-django): the sample owns orchestration (compose + bootstrap + traffic), the keploy CI lanes consume it as a thin wrapper.
4+
5+
The sample drives the full umami v2 API surface keploy needs to gate on a record/replay round-trip — auth + me + admin lists, users CRUD, websites CRUD, all eight report types, share tokens + public share access, batch + identify event ingest, sessions deep-dive, replays, boards lifecycle, pixel tracker, metric/pageview parser-branch variants, and logout.
6+
7+
## Layout
8+
9+
```
10+
umami-postgres/
11+
├── Dockerfile # FROM ghcr.io/umami-software/umami:postgresql-v2.18.1
12+
├── docker-compose.yml # postgres-15 + umami v2 on a fixed subnet, env-driven
13+
├── flow.sh # bootstrap | record-traffic | coverage
14+
├── keploy.yml.template # globalNoise for createdAt/updatedAt/Date/uuid id fields
15+
└── README.md # this file
16+
```
17+
18+
## Contract
19+
20+
The sample is keploy-independent: `docker compose up && bash flow.sh bootstrap && bash flow.sh record-traffic` runs end-to-end against bare umami. Lane scripts wrap that exact same path inside `keploy record` / `keploy test`.
21+
22+
* `bootstrap` — log in as admin via `/api/auth/login`, capture the JWT-style auth token, persist it to `/tmp/umami-token-${UMAMI_PHASE}` so subsequent calls share a deterministic Authorization header.
23+
* `record-traffic` — drive the umami v2 API. Calls are fire-and-forget (`|| true` semantics) so a single endpoint regression in umami itself does not abort the run — keploy is the assertion layer at replay.
24+
* `coverage` — no-op stub. The upstream umami image ships compiled+minified Next.js without sourcemaps, so source-line coverage is not meaningful without rebuilding from source. Returns 0 cleanly so `flow.sh coverage || true` informational hooks keep working.
25+
26+
## Local run
27+
28+
### Without keploy — smoke check
29+
30+
```sh
31+
docker compose up -d
32+
bash flow.sh bootstrap 240
33+
bash flow.sh record-traffic
34+
docker compose down -v
35+
```
36+
37+
This is what the keploy/enterprise compat lane wraps in `keploy record` / `keploy test` — the base compose runs unchanged inside that lane.
38+
39+
### With keploy — record + replay
40+
41+
```sh
42+
docker compose up -d
43+
bash flow.sh bootstrap 240
44+
45+
# In one shell:
46+
keploy record -c "docker compose up" --container-name umami_app \
47+
--proxy-port 13081 --dns-port 13082
48+
49+
# In another shell:
50+
bash flow.sh record-traffic
51+
# SIGINT keploy when traffic returns
52+
53+
keploy test -c "docker compose up" --containerName umami_app \
54+
--apiTimeout 60 --delay 30 --proxy-port 13081 --dns-port 13082
55+
```
56+
57+
### Coverage
58+
59+
This sample does not emit a coverage metric. The upstream `ghcr.io/umami-software/umami:postgresql-v2.18.1` image ships a compiled + minified Next.js standalone build with no source tree or sourcemaps; V8 line coverage on minified output doesn't map back to anything a reviewer can act on, so a coverage gate would be misleading. The keploy/enterprise compat lane uses the record/replay assertions as its correctness gate, which is the meaningful test here.
60+
61+
If real source-line coverage becomes a hard requirement, the path is to rebuild umami from its own source (npm install + `next build` without minification) inside a `Dockerfile.coverage` overlay — a separate, larger change.
62+
63+
## Consumers
64+
65+
* `keploy/enterprise` `.woodpecker/umami-linux.yml` — record/replay matrix delegates compose + bootstrap + traffic to this sample.
66+
* `keploy/integrations` may add a `.woodpecker/umami-postgres.yml` falsifying lane in a future PR.

umami-postgres/docker-compose.yml

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# umami-postgres sample compose. Postgres-15 + umami v2 on a fixed
2+
# subnet, every name env-driven so multiple matrix cells can run
3+
# in parallel on the same docker daemon. Two-phase boot pattern
4+
# matches the doccano-django sibling: SKIP_INIT=0 first time so
5+
# umami's `npx umami-app db:up` runs migrations and seeds; volume
6+
# is retained; SKIP_INIT=1 second time launches the app against
7+
# the populated volume.
8+
services:
9+
app:
10+
build:
11+
context: .
12+
dockerfile: Dockerfile
13+
container_name: ${UMAMI_APP_CONTAINER:-umami_app}
14+
init: true
15+
stop_grace_period: 5s
16+
ports:
17+
- "${UMAMI_APP_PORT:-3001}:3000"
18+
environment:
19+
DATABASE_URL: postgresql://umami:umami@${UMAMI_DB_IP:-172.35.0.10}:5432/umami
20+
DATABASE_TYPE: postgresql
21+
APP_SECRET: ${UMAMI_APP_SECRET:-keploy-fixed-app-secret-for-deterministic-recordings}
22+
DISABLE_TELEMETRY: "1"
23+
DISABLE_UPDATES: "1"
24+
UMAMI_SKIP_INIT: "${UMAMI_SKIP_INIT:-0}"
25+
# umami's scripts/check-db.js reads SKIP_DB_CHECK to decide
26+
# whether to skip the on-startup `prisma migrate deploy`. The
27+
# script tests `if (process.env.SKIP_DB_CHECK)` — a JS
28+
# truthiness check, so the literal string "0" is **truthy**
29+
# and skips. The only way to NOT skip is to leave the env
30+
# var unset or empty. Bootstrap phase wants migrations to
31+
# run (admin user creation), so the lane sets
32+
# SKIP_DB_CHECK="" (empty). Keploy record/replay phases need
33+
# the skip on (keploy's DNS interception blocks
34+
# binaries.prisma.sh, so the runtime prisma fetch fails with
35+
# EAI_AGAIN), so the lane sets SKIP_DB_CHECK="1".
36+
SKIP_DB_CHECK: "${SKIP_DB_CHECK:-}"
37+
depends_on:
38+
postgres:
39+
condition: service_healthy
40+
networks:
41+
- umami-net
42+
43+
postgres:
44+
image: postgres:15-alpine
45+
container_name: ${UMAMI_DB_CONTAINER:-umami_db}
46+
stop_grace_period: 5s
47+
environment:
48+
POSTGRES_USER: umami
49+
POSTGRES_PASSWORD: umami
50+
POSTGRES_DB: umami
51+
healthcheck:
52+
test: ["CMD-SHELL", "pg_isready -U umami -d umami"]
53+
interval: 5s
54+
timeout: 5s
55+
retries: 20
56+
volumes:
57+
- umami-db-data:/var/lib/postgresql/data
58+
networks:
59+
umami-net:
60+
ipv4_address: ${UMAMI_DB_IP:-172.35.0.10}
61+
62+
networks:
63+
umami-net:
64+
driver: bridge
65+
ipam:
66+
config:
67+
- subnet: ${UMAMI_NETWORK_SUBNET:-172.35.0.0/24}
68+
69+
volumes:
70+
umami-db-data:

0 commit comments

Comments
 (0)