Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions restheart-mongo/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Thin wrapper around RESTHeart's official image at the version
# this sample tracks. Pin lives here so a future RESTHeart release
# is a one-line retag, not a hunt across keploy CI lanes.
#
# Upstream: https://github.com/SoftInstigate/restheart
# Image: docker.io/softinstigate/restheart:9.2.1
FROM softinstigate/restheart:9.2.1
49 changes: 49 additions & 0 deletions restheart-mongo/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# restheart-mongo — keploy compat lane sample (work in progress)

Minimum reproducer scaffold for the RESTHeart / MongoDB compat lane. Mirrors the architectural pattern of the [doccano-django sample in `samples-python`](https://github.com/keploy/samples-python/tree/main/doccano-django): the sample owns orchestration (compose / bootstrap / traffic / noise filter / coverage), keploy CI lanes consume it as a thin wrapper.

## Status

**This is a SCAFFOLD.** The compose, bootstrap, and a minimal record-traffic loop work end-to-end against bare RESTHeart without keploy in the picture. The full traffic loop the existing keploy/enterprise lane drives (`compat_trigger_record_traffic` in `enterprise/.ci/scripts/restheart-linux.sh`, ~600 lines covering CRUD on `/<db>/<coll>` + GraphQL + files + ACL + users + bulk + aggregations) has **not been ported** into `flow.sh::restheart_record_traffic` yet. Lanes consuming this sample today should either:

1. Port the missing curls into `flow.sh::restheart_record_traffic` (preferred — that's the migration this scaffold is designed around).
2. Or call into `enterprise/.ci/scripts/restheart-linux.sh::compat_trigger_record_traffic` between `flow.sh bootstrap` and `flow.sh coverage` until the migration completes.

See the migration plan in this PR's description / linked issue.

## Layout

```
restheart-mongo/
├── Dockerfile # FROM softinstigate/restheart:9.2.1
├── docker-compose.yml # mongo:7 + restheart:9.2.1, fixed subnet, env-driven
├── flow.sh # bootstrap | record-traffic | coverage | list-routes
├── keploy.yml.template # globalNoise for _etag/_oid/lastModified/Date
└── README.md # this file
```

## Contract

The sample is keploy-independent: `docker compose up && bash flow.sh bootstrap && bash flow.sh record-traffic` runs end-to-end against bare RESTHeart. Lane scripts wrap that exact same path inside `keploy record` / `keploy test`.

* `bootstrap` — wait for RESTHeart to start serving, PUT the test database + collection so subsequent reads have something to find.
* `record-traffic` — drive RESTHeart's REST surface. Every call is logged to `${RESTHEART_FIRED_ROUTES_FILE}` (when set) so `coverage` has a numerator without a keploy recording.
* `coverage` — emits `(method, path)` coverage. Denominator is curated from RESTHeart's pattern-based mount table (see `restheart_list_routes` in `flow.sh`); not file-system-derivable like Next.js, so the list lives in source and must be updated alongside `record-traffic`.
* `list-routes` — diagnostic; prints the route table.

## Local run

```sh
docker compose up -d
bash flow.sh bootstrap 240
RESTHEART_FIRED_ROUTES_FILE=/tmp/fired.log bash flow.sh record-traffic
RESTHEART_FIRED_ROUTES_FILE=/tmp/fired.log bash flow.sh coverage
docker compose down -v
```

## Consumers

Lanes pinning to this sample (pinned via `--branch feat/restheart-mongo-sample` until merge):

* `keploy/enterprise` `.woodpecker/restheart-linux.yml` — being slimmed in a follow-up PR.
* No `keploy/integrations` consumer today; could be added if a RESTHeart-flavoured Mongo wire bug surfaces.
47 changes: 47 additions & 0 deletions restheart-mongo/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# restheart-mongo sample compose. RESTHeart 9.x + MongoDB 7 on a
# fixed subnet, every name env-driven so multiple matrix cells
# can run in parallel on the same docker daemon.
services:
restheart:
build:
context: .
dockerfile: Dockerfile
container_name: ${RESTHEART_APP_CONTAINER:-restheart_app}
init: true
stop_grace_period: 5s
ports:
- "${RESTHEART_APP_PORT:-8080}:8080"
environment:
RHO: >
/mclient/connection-string->"mongodb://${RESTHEART_MONGO_IP:-172.36.0.10}:27017",
/core/log-level->"INFO"
depends_on:
mongo:
condition: service_healthy
networks:
- restheart-net

mongo:
image: mongo:7
container_name: ${RESTHEART_MONGO_CONTAINER:-restheart_mongo}
stop_grace_period: 5s
healthcheck:
test: ["CMD", "mongosh", "--quiet", "--eval", "db.adminCommand('ping').ok"]
interval: 5s
timeout: 5s
retries: 20
volumes:
- restheart-mongo-data:/data/db
networks:
restheart-net:
ipv4_address: ${RESTHEART_MONGO_IP:-172.36.0.10}

networks:
restheart-net:
driver: bridge
ipam:
config:
- subnet: ${RESTHEART_NETWORK_SUBNET:-172.36.0.0/24}
Comment on lines +83 to +88

volumes:
restheart-mongo-data:
208 changes: 208 additions & 0 deletions restheart-mongo/flow.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
#!/usr/bin/env bash
#
# flow.sh — keploy-independent orchestration for the
# restheart-mongo sample. Modeled on
# samples-python/doccano-django/flow.sh.
#
# Subcommands:
# bootstrap — RESTHeart's default config has no admin auth
# setup needed; the bootstrap step here just
# creates the test database and seed
# collections so subsequent reads have
# something to find.
# record-traffic — drive RESTHeart's REST surface (Mongo / GraphQL
# / files / users / acl). Fire-and-forget;
# keploy is the assertion layer at replay.
# coverage — report (method, path) coverage. Denominator is
# derived from RESTHeart's known route-mounts
# (see SCOPE_PATHS in restheart_list_routes).
# list-routes — print the route table the coverage report
# uses as its denominator.
#
# HANDOFF NOTE: SCAFFOLD. The full traffic loop the existing keploy
# lane drives (`compat_trigger_record_traffic` in
# enterprise/.ci/scripts/restheart-linux.sh, ~600 lines covering
# CRUD on /<db>/<coll> + GraphQL + files + ACL + users + bulk +
# aggregations) needs to be ported into
# `restheart_record_traffic` here. The stub below covers enough
# to prove the sample boots end-to-end without keploy. See the
# migration plan in the PR description / linked issue.
set -Eeuo pipefail

RESTHEART_APP_PORT="${RESTHEART_APP_PORT:-8080}"
RESTHEART_APP_CONTAINER="${RESTHEART_APP_CONTAINER:-restheart_app}"
RESTHEART_MONGO_CONTAINER="${RESTHEART_MONGO_CONTAINER:-restheart_mongo}"
RESTHEART_DB="${RESTHEART_DB:-keploy}"
RESTHEART_PHASE="${RESTHEART_PHASE:-local}"
RESTHEART_FIRED_ROUTES_FILE="${RESTHEART_FIRED_ROUTES_FILE:-}"

# RESTHeart 9.x ships with an admin user (admin/secret) for protected
# endpoints; the unauthenticated paths are fine for the smoke set we
# drive in record-traffic. Override RESTHEART_ADMIN_AUTH to add
# `Authorization: Basic <b64>` to authenticated calls when porting
# the full lane traffic.
RESTHEART_ADMIN_AUTH="${RESTHEART_ADMIN_AUTH:-Basic YWRtaW46c2VjcmV0}"

base="http://127.0.0.1:${RESTHEART_APP_PORT}"
h_json='Content-Type: application/json'

log_fired() {
[ -z "$RESTHEART_FIRED_ROUTES_FILE" ] && return 0
printf '%s %s\n' "$1" "$2" >>"$RESTHEART_FIRED_ROUTES_FILE"
}

restheart_wait_for_app() {
local timeout=${1:-180}
local start_ts code
start_ts=$(date +%s)
while true; do
code=$(curl -sS -o /dev/null -w '%{http_code}' "${base}/" 2>/dev/null || echo "")
# 401 (auth required on root) is a SUCCESS signal — it
# means RESTHeart is up and responding to HTTP.
if [ "$code" = "200" ] || [ "$code" = "401" ]; then return 0; fi
if [ $(( $(date +%s) - start_ts )) -ge "$timeout" ]; then
echo "restheart_wait_for_app: timed out (last code: ${code:-<empty>})" >&2
return 1
fi
sleep 2
done
}

restheart_bootstrap() {
local timeout=${1:-180}
restheart_wait_for_app "$timeout"

# Create the test database. PUT on /<db> is idempotent —
# 201 first time, 200 on subsequent runs.
curl -sS -o /dev/null -H "$RESTHEART_ADMIN_AUTH" -X PUT "${base}/${RESTHEART_DB}" || true
# Seed a collection so reads have something to find.
curl -sS -o /dev/null -H "$RESTHEART_ADMIN_AUTH" -X PUT "${base}/${RESTHEART_DB}/items" || true
echo "restheart_bootstrap: db=${RESTHEART_DB} ready"
}

restheart_record_traffic() {
restheart_wait_for_app 60

log_fired GET "$base/"
curl -sS -H "$RESTHEART_ADMIN_AUTH" "$base/" >/dev/null || true

log_fired GET "$base/${RESTHEART_DB}"
curl -sS -H "$RESTHEART_ADMIN_AUTH" "$base/${RESTHEART_DB}" >/dev/null || true

log_fired GET "$base/${RESTHEART_DB}/items"
curl -sS -H "$RESTHEART_ADMIN_AUTH" "$base/${RESTHEART_DB}/items" >/dev/null || true

# Insert a document.
log_fired POST "$base/${RESTHEART_DB}/items"
curl -fsS -H "$RESTHEART_ADMIN_AUTH" -H "$h_json" -X POST \
"$base/${RESTHEART_DB}/items" \
-d "{\"_id\":\"keploy-${RESTHEART_PHASE}\",\"name\":\"sample item\",\"score\":42}" >/dev/null || true

# Read it back.
log_fired GET "$base/${RESTHEART_DB}/items/keploy-${RESTHEART_PHASE}"
curl -sS -H "$RESTHEART_ADMIN_AUTH" \
"$base/${RESTHEART_DB}/items/keploy-${RESTHEART_PHASE}" >/dev/null || true

# Update it.
log_fired PATCH "$base/${RESTHEART_DB}/items/keploy-${RESTHEART_PHASE}"
curl -sS -H "$RESTHEART_ADMIN_AUTH" -H "$h_json" -X PATCH \
"$base/${RESTHEART_DB}/items/keploy-${RESTHEART_PHASE}" \
-d '{"$set":{"score":100}}' >/dev/null || true

# Aggregation surface.
log_fired GET "$base/${RESTHEART_DB}/items/_size"
curl -sS -H "$RESTHEART_ADMIN_AUTH" "$base/${RESTHEART_DB}/items/_size" >/dev/null || true
log_fired GET "$base/${RESTHEART_DB}/_meta"
curl -sS -H "$RESTHEART_ADMIN_AUTH" "$base/${RESTHEART_DB}/_meta" >/dev/null || true
}

# RESTHeart's routes are pattern-mount based, not file-system
# based. The denominator is curated here from the upstream docs +
# the routes the lane intends to exercise. Update this list when
# adding new traffic to record-traffic so the coverage stays in
# lockstep.
restheart_list_routes() {
cat <<'ROUTES'
GET /
GET /{db}
PUT /{db}
DELETE /{db}
GET /{db}/_meta
GET /{db}/{coll}
PUT /{db}/{coll}
DELETE /{db}/{coll}
POST /{db}/{coll}
GET /{db}/{coll}/{docid}
PUT /{db}/{coll}/{docid}
PATCH /{db}/{coll}/{docid}
DELETE /{db}/{coll}/{docid}
GET /{db}/{coll}/_size
GET /{db}/{coll}/_aggrs/{name}
GET /{db}/{coll}/_indexes
ROUTES
}

restheart_list_recorded_routes() {
local f method route
local found_keploy=0
while IFS= read -r f; do
found_keploy=1
method=$(awk '/^ method:/{print $2; exit}' "$f")
route=$(awk '/^ url:/{print $2; exit}' "$f")
route="${route%%\?*}"
case "$route" in http://*|https://*) route="/${route#*://*/}" ;; esac
if [ -n "$method" ] && [ -n "$route" ]; then echo "$method $route"; fi
done < <(find keploy -type f -path '*/tests/*.yaml' 2>/dev/null) | sort -u
if [ "$found_keploy" = "1" ]; then return 0; fi

if [ -n "$RESTHEART_FIRED_ROUTES_FILE" ] && [ -f "$RESTHEART_FIRED_ROUTES_FILE" ]; then
while IFS= read -r line; do
method="${line%% *}"; route="${line#* }"
route="${route%%\?*}"
case "$route" in http://*|https://*) route="/${route#*://*/}" ;; esac
[ -n "$method" ] && [ -n "$route" ] && echo "$method $route"
done <"$RESTHEART_FIRED_ROUTES_FILE" | sort -u
fi
}

restheart_report_coverage() {
local routes_file recorded_file
routes_file="$(mktemp)"; recorded_file="$(mktemp)"
restheart_list_routes >"$routes_file"
restheart_list_recorded_routes >"$recorded_file"

local total covered missing pct
total=$(wc -l <"$routes_file" | tr -d ' '); covered=0; missing=""
while IFS= read -r line; do
local method="${line%% *}"
local route="${line#* }"
# Replace {param} placeholders with [^/]+ for matching.
local pattern
pattern="^${method} $(printf '%s' "$route" | sed -E 's/\{[^}]+\}/[^\/]+/g')$"
if grep -qE "$pattern" "$recorded_file"; then
covered=$((covered + 1))
else
missing+=" ${method} ${route}"$'\n'
fi
done <"$routes_file"
if [ "$total" -gt 0 ]; then
pct=$(awk -v c="$covered" -v t="$total" 'BEGIN{printf "%.1f", c*100/t}')
else pct="0.0"; fi
{
echo "================ RESTHeart API coverage ================"
echo "Covered ${covered}/${total} (${pct}%)"
if [ -n "$missing" ]; then echo "Uncovered:"; printf '%s' "$missing"; fi
echo "========================================================"
} | tee "${COVERAGE_REPORT_FILE:-coverage_report.txt}"
rm -f "$routes_file" "$recorded_file"
}

case "${1:-}" in
bootstrap) restheart_bootstrap "${2:-180}" ;;
record-traffic) restheart_record_traffic ;;
coverage) restheart_report_coverage ;;
list-routes) restheart_list_routes ;;
*)
echo "usage: $0 {bootstrap|record-traffic|coverage|list-routes}" >&2
exit 2 ;;
esac
21 changes: 21 additions & 0 deletions restheart-mongo/keploy.yml.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# keploy.yml template for the restheart-mongo sample.
#
# globalNoise covers fields whose value is non-deterministic
# across record/replay:
#
# header.Date runtime-stamped
# body._etag RESTHeart auto-stamped on each
# document; changes per write
# body._oid / body._id server-generated ObjectIds
# (when not set by client)
# body.lastModified auto-now timestamp
#
# Centralised here so a future RESTHeart version that adds another
# auto-stamped field is one edit, not a fan-out across lane scripts.
test:
globalNoise:
global:
header.Date: []
body._etag: []
body._oid: []
body.lastModified: []
Loading