Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
d11c6f2
WIP: Outbox 엔티티 전환(OperationType+targetId) 및 후속 상태조회 전환 준비
lunarbae628 Mar 3, 2026
f795968
Refactor: Mongo 삭제 Outbox 워커 도입 및 claim/done/retry 상태 전이 분리
lunarbae628 Mar 4, 2026
b20b383
Refactor: Mongo 삭제 Outbox 안정화 (Save 삭제 전환, PROCESSING 복구, operationKe…
lunarbae628 Mar 4, 2026
1eb7020
Refactor: Mongo delete outbox 식별체계를 Trigger/Domain/Origin으로 전환 및 dedu…
lunarbae628 Mar 4, 2026
27e16ed
Chore: 서비스 계층 트랜잭션 정책 정비 (class-level + checked rollback)
lunarbae628 Mar 7, 2026
3ae1fa6
Test: Mongo delete outbox 전환 테스트 반영 (retry/maxRetry/done 경로 테스트 보강)
lunarbae628 Mar 8, 2026
d731b02
Test: Mongo delete outbox 핵심 시나리오(중복/timeout/no-op/배치한도) 보강
lunarbae628 Mar 8, 2026
8a0eec7
Feat: TestUserInitializer에 PERF_SEED_USER_COUNT 기반 대량 유저 시드 추가
lunarbae628 Mar 9, 2026
5b4392f
Test(perf): 삭제 성능측정용 k6 시나리오/비교 스크립트 추가
lunarbae628 Mar 9, 2026
6b292ef
Fix(test): MongoDeleteOutboxWorker 스케줄 경합 제거로 배치 한도 테스트 안정화
lunarbae628 Mar 9, 2026
523d3f0
Feat: stg 부하테스트용 TestUserInitializer 대량 유저 시드 추가 (PERF_SEED_USER_COUNT)
lunarbae628 Mar 9, 2026
1d636e7
Merge: stg 부하테스트용 TestUserInitializer 대량 유저 시드 추가
lunarbae628 Mar 9, 2026
137f8d9
Fix(perf): 통합 cleanup 스크립트 추가 및 k6 블록 포맷 보정
lunarbae628 Mar 10, 2026
82235eb
Merge: Mongo데이터 삭제 Outbox 도입 및 k6 부하테스트 스크립트
lunarbae628 Mar 10, 2026
916bfd5
Outbox 생성 충돌 테스트 보강 및 흐름 정리
lunarbae628 Mar 25, 2026
d27c551
리팩터링: Mongo 삭제 Outbox 패키지와 서비스 네이밍 정리
lunarbae628 Mar 25, 2026
51e2f4d
Merge: Mongo delete outbox 구조 정리 및 생성 충돌 테스트 보강
lunarbae628 Mar 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,7 @@ infra/mysql/logs/
*.log
*.pid
.*.swp

# Perf benchmark outputs
perf/**/results/*
!perf/**/results/.gitkeep
185 changes: 185 additions & 0 deletions perf/cleanup_perf_docs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
#!/usr/bin/env bash
set -euo pipefail

# Cleanup perf-created docs by API.
# Modes:
# - single: delete docs created by single-user benchmark (SDEL-*)
# - multi : delete docs created by multi-user benchmark (PDEL-*)
# - all : both (default)

BASE_URL="${BASE_URL:-http://localhost:8080}"
SESSION_COOKIE_NAME="${SESSION_COOKIE_NAME:-JSESSIONID}"
INSECURE_TLS="${INSECURE_TLS:-0}"

MODE="${MODE:-all}" # single | multi | all
USER_PASSWORD="${USER_PASSWORD:-Testtest1}"
TEST_EMAIL="${TEST_EMAIL:-test@test.com}"

USER_PREFIX="${USER_PREFIX:-perfdel}"
USER_DOMAIN="${USER_DOMAIN:-test.com}"
USER_COUNT="${USER_COUNT:-100}"
SEARCH_PAGE_SIZE="${SEARCH_PAGE_SIZE:-100}"

if [[ "${MODE}" != "single" && "${MODE}" != "multi" && "${MODE}" != "all" ]]; then
echo "MODE must be one of: single | multi | all" >&2
exit 1
fi

if ! [[ "${USER_COUNT}" =~ ^[0-9]+$ ]] || [[ "${USER_COUNT}" -lt 0 ]]; then
echo "USER_COUNT must be a non-negative integer" >&2
exit 1
fi

if [[ "${INSECURE_TLS}" == "1" ]]; then
CURL_BASE=(curl -k -sS)
else
CURL_BASE=(curl -sS)
fi

pad3() {
printf "%03d" "$1"
}

login_cookie() {
local email="$1"
local headers body status cookie
headers="$(mktemp)"
body="$(mktemp)"

status="$(
"${CURL_BASE[@]}" -o "${body}" -D "${headers}" -w '%{http_code}' \
-H 'Content-Type: application/json' \
-X POST "${BASE_URL}/api/user/login" \
-d "{\"email\":\"${email}\",\"password\":\"${USER_PASSWORD}\"}"
)"

if [[ "${status}" != "200" ]]; then
rm -f "${headers}" "${body}"
return 1
fi

cookie="$(
grep -i '^set-cookie:' "${headers}" \
| tr -d '\r' \
| sed -n "s/^set-cookie:[[:space:]]*${SESSION_COOKIE_NAME}=\\([^;]*\\).*/\\1/p" \
| head -n 1
)"
rm -f "${headers}" "${body}"

[[ -n "${cookie}" ]] || return 1
printf '%s' "${cookie}"
}

search_doc_ids() {
local cookie="$1"
local keyword="$2"
local body

body="$(
"${CURL_BASE[@]}" \
-H "Cookie: ${SESSION_COOKIE_NAME}=${cookie}" \
-H 'Content-Type: application/json' \
"${BASE_URL}/api/document/search?keyword=${keyword}&page=0&size=${SEARCH_PAGE_SIZE}&sort=updatedAt&order=desc"
)"

printf '%s' "${body}" | node -e '
const fs = require("fs");
const raw = fs.readFileSync(0, "utf8");
let json;
try {
json = JSON.parse(raw);
} catch {
process.exit(0);
}
const content = Array.isArray(json.content) ? json.content : [];
for (const row of content) {
if (row && row.id != null) console.log(row.id);
}
'
}

delete_doc() {
local cookie="$1"
local doc_id="$2"
local status

status="$(
"${CURL_BASE[@]}" -o /dev/null -w '%{http_code}' \
-H "Cookie: ${SESSION_COOKIE_NAME}=${cookie}" \
-H 'Content-Type: application/json' \
-X DELETE "${BASE_URL}/api/document/${doc_id}"
)"
[[ "${status}" == "204" ]]
}

build_users() {
if [[ "${MODE}" == "single" || "${MODE}" == "all" ]]; then
printf '%s\n' "${TEST_EMAIL}"
fi

if [[ "${MODE}" == "multi" || "${MODE}" == "all" ]]; then
local i=1
while [[ "${i}" -le "${USER_COUNT}" ]]; do
printf "%s_u%s@%s\n" "${USER_PREFIX}" "$(pad3 "${i}")" "${USER_DOMAIN}"
i=$((i + 1))
done
fi
}

build_keywords() {
if [[ "${MODE}" == "single" || "${MODE}" == "all" ]]; then
printf '%s\n' "SDEL-"
fi

if [[ "${MODE}" == "multi" || "${MODE}" == "all" ]]; then
printf '%s\n' "PDEL-"
fi
}

echo "[cleanup] BASE_URL=${BASE_URL} MODE=${MODE} INSECURE_TLS=${INSECURE_TLS}"

total_deleted=0
total_failed=0
users_processed=0

while IFS= read -r email; do
[[ -z "${email}" ]] && continue

cookie="$(login_cookie "${email}" || true)"
if [[ -z "${cookie}" ]]; then
echo "[cleanup][skip] login failed: ${email}"
continue
fi

users_processed=$((users_processed + 1))

while IFS= read -r keyword; do
[[ -z "${keyword}" ]] && continue
while true; do
ids="$(search_doc_ids "${cookie}" "${keyword}" || true)"
if [[ -z "${ids}" ]]; then
break
fi

deleted_this_round=0
while IFS= read -r id; do
[[ -z "${id}" ]] && continue
if delete_doc "${cookie}" "${id}"; then
total_deleted=$((total_deleted + 1))
deleted_this_round=$((deleted_this_round + 1))
else
total_failed=$((total_failed + 1))
echo "[cleanup][warn] delete failed: user=${email}, docId=${id}"
fi
done <<< "${ids}"

# Avoid infinite loop if all deletes fail in this page.
if [[ "${deleted_this_round}" -eq 0 ]]; then
break
fi
done
done < <(build_keywords)
done < <(build_users | awk '!seen[$0]++')

echo "[cleanup] users processed=${users_processed}, docs deleted=${total_deleted}, failed=${total_failed}"
echo "[cleanup] Mongo cleanup is async via outbox worker."
137 changes: 137 additions & 0 deletions perf/delete/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# Deletion-Only Performance Benchmark (Multi-user)

목표:

- 다수 유저가 실사용처럼 문서/커밋/브랜치를 먼저 생성
- 삭제 API 성능만 별도 측정
- `이벤트 기반 삭제` vs `outbox` 비교를 같은 조건으로 수행

## Files

- `perf/delete/seed_dataset.js`
- 유저별 문서/커밋/브랜치 데이터 생성
- `perf/delete/delete_only_benchmark.js`
- 생성된 문서를 찾아 삭제만 수행
- `perf/delete/compare_delete_summary.mjs`
- 두 브랜치 결과 비교 표 출력
- `perf/cleanup_perf_docs.sh`
- 생성된 perf 문서 정리 (`MODE=multi` 또는 `MODE=all`)

## 0) App run (example)

로컬 부팅 예시:

```bash
MONGO_URI='...' MYSQL_USERNAME='devbae' MYSQL_PASSWORD='' SPRING_PROFILES_ACTIVE='local' bash ./gradlew bootRun
```

`dev` 브랜치에서 메일 env가 필요하면 아래도 같이 추가:

```bash
MAIL_USERNAME='...' MAIL_PASSWORD='...'
```

## 1) Prepare users (no script)

유저 생성은 애플리케이션의 `TestUserInitializer`를 사용합니다.

```bash
PERF_SEED_USER_COUNT=100
```

유저 패턴:

- `perfdel_u001@test.com` ~ `perfdel_u100@test.com`
- 비밀번호: `Testtest1`

## 2) Seed realistic dataset

동일한 `RUN_ID`를 기록해 두고 삭제 벤치마크에서 재사용하세요.

```bash
RUN_ID=run01 \
BASE_URL=http://localhost:8080 \
USER_PREFIX=perfdel USER_DOMAIN=test.com USER_PASSWORD=Testtest1 \
USER_COUNT=100 DOCS_PER_USER=3 \
MAIN_COMMITS=6 FEATURE_COMMITS=4 BLOCKS_PER_COMMIT=20 \
SEED_VUS=20 \
k6 run perf/delete/seed_dataset.js
```

생성 규칙:

- 문서 제목: `PDEL-<RUN_ID>uNNNdNNN`
- 각 문서에 main 커밋 + feature 브랜치 커밋 생성
- 삭제 시 block 수가 충분히 커지도록 commit 당 block 수 지정

## 3) Delete-only benchmark

```bash
RUN_ID=run01 \
BASE_URL=http://localhost:8080 \
USER_PREFIX=perfdel USER_DOMAIN=test.com USER_PASSWORD=Testtest1 \
USER_COUNT=100 DOCS_PER_USER=3 \
DELETE_VUS=30 \
k6 run perf/delete/delete_only_benchmark.js \
--summary-export perf/delete/results/delete_only_summary.json
```

측정 포인트:

- `op_doc_delete_ms` (핵심)
- `http_req_duration`
- 실패율: `http_req_failed`, `delete_failed`

## 4) Branch comparison workflow

권장: 브랜치별로 아래 순서를 동일하게 반복

1. 브랜치 checkout
2. 서버 실행
3. 유저 준비(`PERF_SEED_USER_COUNT` 적용된 상태)
4. 데이터 생성 (`seed_dataset.js`)
5. 삭제 벤치 실행 (`delete_only_benchmark.js`)
6. 결과 파일 저장

예시 결과 파일명:

- `perf/delete/results/dev_delete_summary.json`
- `perf/delete/results/refactor_delete_summary.json`

비교:

```bash
node perf/delete/compare_delete_summary.mjs \
perf/delete/results/dev_delete_summary.json \
perf/delete/results/refactor_delete_summary.json
```

## 5) Fairness checklist

- 브랜치별 동일한 `USER_COUNT`, `DOCS_PER_USER`, `MAIN_COMMITS`, `FEATURE_COMMITS`, `BLOCKS_PER_COMMIT`
- 동일 DB 스펙, 동일 JVM 옵션
- 워밍업 1회 후 본측정 3회 이상 (평균/편차 사용)
- 브랜치 순서 바꿔서 반복 (`dev -> refactor`, `refactor -> dev`)

## 6) Cleanup

다중 유저 테스트 데이터(`PDEL-*`) 정리:

```bash
BASE_URL=http://localhost:8080 \
MODE=multi \
USER_PREFIX=perfdel USER_DOMAIN=test.com USER_COUNT=100 \
USER_PASSWORD=Testtest1 \
bash perf/cleanup_perf_docs.sh
```

staging HTTPS(자체 서명 인증서)면:

```bash
BASE_URL=https://<stg-domain-or-ip>:8443 \
INSECURE_TLS=1 \
MODE=multi \
USER_PREFIX=perfdel USER_DOMAIN=test.com USER_COUNT=100 \
USER_PASSWORD=Testtest1 \
bash perf/cleanup_perf_docs.sh
```
65 changes: 65 additions & 0 deletions perf/delete/compare_delete_summary.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
#!/usr/bin/env node
import fs from 'node:fs';
import path from 'node:path';

if (process.argv.length < 4) {
console.error('Usage: node perf/delete/compare_delete_summary.mjs <dev-summary.json> <refactor-summary.json>');
process.exit(1);
}

const dev = JSON.parse(fs.readFileSync(path.resolve(process.argv[2]), 'utf-8'));
const ref = JSON.parse(fs.readFileSync(path.resolve(process.argv[3]), 'utf-8'));

function getMetric(summary, name, key) {
const m = summary.metrics?.[name];
if (!m) return null;
if (m.values && typeof m.values === 'object') return m.values[key] ?? null;
if (key === 'rate') return m.value ?? null;
return m[key] ?? null;
}

function pct(base, target) {
if (base === null || target === null || base === 0) return null;
return ((target - base) / base) * 100;
}

function f(v, d = 2) {
if (v === null || Number.isNaN(v)) return '-';
return Number(v).toFixed(d);
}

function row(label, metric, key, digits = 2) {
const b = getMetric(dev, metric, key);
const t = getMetric(ref, metric, key);
const delta = pct(b, t);
const direction = delta === null ? '-' : delta < 0 ? 'improved' : delta > 0 ? 'regressed' : 'same';

return {
label,
base: f(b, digits),
target: f(t, digits),
delta: delta === null ? '-' : `${f(delta, 2)}%`,
direction,
};
}

const rows = [
row('http_req_duration p95 (ms)', 'http_req_duration', 'p(95)'),
row('http_req_duration avg (ms)', 'http_req_duration', 'avg'),
row('doc search p95 (ms)', 'op_doc_search_ms', 'p(95)'),
row('doc delete p50 (ms)', 'op_doc_delete_ms', 'med'),
row('doc delete p95 (ms)', 'op_doc_delete_ms', 'p(95)'),
row('doc delete avg (ms)', 'op_doc_delete_ms', 'avg'),
row('http_req_failed rate', 'http_req_failed', 'rate', 4),
row('delete_failed rate', 'delete_failed', 'rate', 4),
];

console.log('# Delete Benchmark Comparison');
console.log(`- base(dev): ${path.resolve(process.argv[2])}`);
console.log(`- target(refactor): ${path.resolve(process.argv[3])}`);
console.log('');
console.log('| Metric | dev | refactor | Delta | Direction |');
console.log('|---|---:|---:|---:|---|');
for (const r of rows) {
console.log(`| ${r.label} | ${r.base} | ${r.target} | ${r.delta} | ${r.direction} |`);
}
Loading
Loading