|
| 1 | +# spring-aerospike — Aerospike-Java sample with Keploy record/replay |
| 2 | + |
| 3 | +A Spring Boot 2.7 service that talks to Aerospike CE over the |
| 4 | +clear-text service port (3000) using the official |
| 5 | +`aerospike-client-jdk8`. Recorded and replayed end-to-end with |
| 6 | +Keploy via three bundled scripts that mirror the |
| 7 | +`keploy/samples-go/aerospike-tls` shape one-to-one — same endpoints, |
| 8 | +same test-set layout, same record-then-replay loop. |
| 9 | + |
| 10 | +What the sample demonstrates: |
| 11 | + |
| 12 | +* **Keploy records binary Aerospike protocol traffic** — Info, |
| 13 | + AS_MSG (single-record PUT/GET/TOUCH/DELETE), BATCH_READ/WRITE, |
| 14 | + SCAN, QUERY, UDF, CDT — and replays them from `mocks.yaml` |
| 15 | + without needing the real cluster. |
| 16 | +* **Replay stays deterministic at any concurrency the app exposes** — |
| 17 | + single-client `/parallel`, multi-client round-robin, and per- |
| 18 | + request fresh-client construction all pass cleanly. |
| 19 | +* **A pipeline-friendly shape.** Three `scripts/script-{1,2,3}.sh` |
| 20 | + entry points each record and replay one test-set independently, |
| 21 | + so a CI matrix can call them as separate steps. |
| 22 | + |
| 23 | +## Layout |
| 24 | + |
| 25 | +``` |
| 26 | +spring-aerospike/ |
| 27 | +├── pom.xml # Spring Boot 2.7 + aerospike-client-jdk8 |
| 28 | +├── src/main/java/com/example/aerospike/ |
| 29 | +│ ├── SpringAerospikeApplication.java |
| 30 | +│ ├── config/ # client + multi-client pool, warmup, policies |
| 31 | +│ └── controller/ # one @RestController per endpoint group |
| 32 | +├── src/main/resources/ |
| 33 | +│ └── application.properties # port + Aerospike host/namespace/pool sizing |
| 34 | +├── aerospike-conf/ |
| 35 | +│ └── aerospike.conf # CE config: clear-text on 3000 |
| 36 | +├── docker-compose.yml # Aerospike CE + the Spring Boot app |
| 37 | +├── Dockerfile # eclipse-temurin 17 + mvn package |
| 38 | +├── keploy.yml # Keploy CLI config (command, ports) |
| 39 | +└── scripts/ |
| 40 | + ├── common.sh # shared boot/build/record/replay/normalise |
| 41 | + ├── script-1.sh # records + replays test-set-0 (CRUD) |
| 42 | + ├── script-2.sh # records + replays test-set-1 (/parallel) |
| 43 | + └── script-3.sh # records + replays test-set-2 (/multiclient + /freshclient) |
| 44 | +``` |
| 45 | + |
| 46 | +There is no committed `keploy/` directory — the scripts produce it |
| 47 | +from scratch every run. Each CI run validates the full |
| 48 | +record-then-replay loop instead of replaying stale captures. |
| 49 | + |
| 50 | +## Endpoints |
| 51 | + |
| 52 | +| Method | Path | What it does | |
| 53 | +| ------ | -------------------------- | ---------------------------------------------------------------------------- | |
| 54 | +| GET | `/health` | `info build + namespaces` | |
| 55 | +| POST | `/put` | single-record PUT | |
| 56 | +| GET | `/get/{key}` | single-record GET | |
| 57 | +| POST | `/batch/put` | sequential write loop | |
| 58 | +| GET | `/batch/get?k=a&k=b` | BATCH_READ | |
| 59 | +| POST | `/scan` | full namespace scan | |
| 60 | +| POST | `/query` | secondary-index range query | |
| 61 | +| POST | `/udf` | UDF_EXECUTE | |
| 62 | +| POST | `/cdt/list/append` | CDT list append | |
| 63 | +| POST | `/cdt/map/put` | CDT map put | |
| 64 | +| POST | `/touch/{key}` | TOUCH | |
| 65 | +| DELETE | `/key/{key}` | DELETE | |
| 66 | +| POST | `/parallel?n=N&prefix=P` | fans out N threads, each PUT+GET on a unique key — **one shared client** | |
| 67 | +| POST | `/multiclient?n=N&prefix=P`| same, but round-robins across **4 pre-built `AerospikeClient` instances** | |
| 68 | +| POST | `/freshclient?n=N&prefix=P`| **each thread builds its own `AerospikeClient`** inside the request | |
| 69 | + |
| 70 | +## Run it manually |
| 71 | + |
| 72 | +```bash |
| 73 | +# 1) Boot Aerospike CE on clear-text 3000. |
| 74 | +docker compose up -d aerospike |
| 75 | + |
| 76 | +# 2) Build + run the Spring Boot app. |
| 77 | +mvn -q -DskipTests package |
| 78 | +java -jar target/spring-aerospike.jar |
| 79 | + |
| 80 | +# 3) Hit it. |
| 81 | +curl -s localhost:8090/health |
| 82 | +curl -s -XPOST localhost:8090/put -H 'Content-Type: application/json' \ |
| 83 | + -d '{"key":"alice","bins":{"age":30}}' |
| 84 | +curl -s localhost:8090/get/alice |
| 85 | +curl -s -XPOST 'localhost:8090/parallel?n=24&prefix=run1' |
| 86 | +curl -s -XPOST 'localhost:8090/multiclient?n=24&prefix=mc1' |
| 87 | +curl -s -XPOST 'localhost:8090/freshclient?n=8&prefix=fc1' |
| 88 | +``` |
| 89 | + |
| 90 | +## Record + replay with the scripts |
| 91 | + |
| 92 | +```bash |
| 93 | +# Each script is self-contained: brings up Aerospike, builds the |
| 94 | +# JAR, records, replays. Exit code is non-zero if any case fails on |
| 95 | +# replay. |
| 96 | +sudo ./scripts/script-1.sh # test-set-0: single-endpoint CRUD |
| 97 | +sudo ./scripts/script-2.sh # test-set-1: /parallel n = 4..24 |
| 98 | +sudo ./scripts/script-3.sh # test-set-2: /multiclient + /freshclient |
| 99 | +``` |
| 100 | + |
| 101 | +Pipeline-friendly knobs (env vars): |
| 102 | + |
| 103 | +| Var | Default | What it does | |
| 104 | +|--------------|---------------|---------------------------------------------------------------| |
| 105 | +| `KEPLOY` | `sudo keploy` | binary + auth invocation. Override to `keploy` if root | |
| 106 | +| `PORT` | `8090` | HTTP port the recorded sample listens on | |
| 107 | +| `LOG_DIR` | `/tmp` | where to drop the keploy record log | |
| 108 | +| `SKIP_DOCKER`| (unset) | `=1` skips `docker compose up -d aerospike` (already running) | |
| 109 | +| `SKIP_BUILD` | (unset) | `=1` skips `mvn package` (JAR already in target/) | |
| 110 | + |
| 111 | +## Concurrency notes — why the warmup + retry matter |
| 112 | + |
| 113 | +Mocked replay through Keploy is roughly 10–20× faster than real |
| 114 | +Aerospike for the same op. A burst of N concurrent threads on a |
| 115 | +cold client pool then races to open N fresh sockets, and the |
| 116 | +thread that loses the race surfaces as `MAX_RETRIES_EXCEEDED` at |
| 117 | +the application — even though every peer in the same burst |
| 118 | +succeeds. |
| 119 | + |
| 120 | +`AerospikeConfig` paints over this with four layered changes; |
| 121 | +together they make `/parallel?n=24`, `/multiclient?n=24`, and |
| 122 | +`/freshclient?n=8` replay clean on every run: |
| 123 | + |
| 124 | +1. **Sized pool** — `ClientPolicy.maxConnsPerNode = 256`. The |
| 125 | + `OpeningConnectionThreshold` analogue is kept low (16) so a |
| 126 | + sudden burst doesn't outpace upstream connect rate. |
| 127 | +2. **Tolerant per-op policy** — `Policies.parallelWrite()` and |
| 128 | + `Policies.parallelRead()` set `socketTimeout 10s`, `totalTimeout |
| 129 | + 30s`, `maxRetries 10`, `sleepBetweenRetries 5ms`. |
| 130 | +3. **Two-phase warmup** on the main client at startup: a sequential |
| 131 | + prelude that walks the cluster past cold-start latencies, |
| 132 | + followed by a parallel fill that puts idle connections in the |
| 133 | + pool before the HTTP server accepts the first request. |
| 134 | +4. **App-level retry wrapper** (`RetryHelper.doOp`) around each PUT |
| 135 | + and GET in `/parallel`, `/multiclient`, and `/freshclient`. |
| 136 | + |
| 137 | +`/multiclient`'s extra clients are deliberately NOT warmed at |
| 138 | +startup — a hundred concurrent dials at boot can stall a record- |
| 139 | +time proxy. The retry wrapper covers their first burst instead. |
| 140 | + |
| 141 | +This sample is the Java counterpart of |
| 142 | +[`keploy/samples-go/aerospike-tls`](https://github.com/keploy/samples-go/tree/main/aerospike-tls); |
| 143 | +the script set is byte-for-byte the same shape so a single CI |
| 144 | +matrix can drive both languages with the same harness. |
0 commit comments