Skip to content

Commit 181f332

Browse files
bootjpbootjp
committed
Add redis-proxy Dockerfile, GHCR publishing workflow, and deployment guide for automated container deployment
1 parent 5085aaa commit 181f332

3 files changed

Lines changed: 362 additions & 0 deletions

File tree

‎.github/workflows/redis-proxy-docker.yml‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
name: redis-proxy Docker Image
2+
3+
permissions:
4+
contents: read
5+
packages: write
6+
7+
on:
8+
push:
9+
branches: [main]
10+
paths:
11+
- 'cmd/redis-proxy/**'
12+
- 'proxy/**'
13+
- 'Dockerfile.redis-proxy'
14+
- '.github/workflows/redis-proxy-docker.yml'
15+
- 'go.mod'
16+
- 'go.sum'
17+
pull_request:
18+
paths:
19+
- 'cmd/redis-proxy/**'
20+
- 'proxy/**'
21+
- 'Dockerfile.redis-proxy'
22+
- '.github/workflows/redis-proxy-docker.yml'
23+
- 'go.mod'
24+
- 'go.sum'
25+
26+
concurrency:
27+
group: ${{ github.workflow }}-${{ github.ref }}
28+
cancel-in-progress: true
29+
30+
jobs:
31+
build:
32+
runs-on: ubuntu-latest
33+
steps:
34+
- uses: actions/checkout@v6
35+
36+
- name: Set up Docker Buildx
37+
uses: docker/setup-buildx-action@v4
38+
39+
- name: Login to GitHub Container Registry
40+
uses: docker/login-action@v4
41+
with:
42+
registry: ghcr.io
43+
username: ${{ github.repository_owner }}
44+
password: ${{ secrets.GITHUB_TOKEN }}
45+
46+
- name: Docker metadata
47+
id: meta
48+
uses: docker/metadata-action@v6
49+
with:
50+
images: ghcr.io/${{ github.repository }}/redis-proxy
51+
tags: |
52+
type=sha
53+
type=ref,event=branch
54+
type=raw,value=latest,enable={{is_default_branch}}
55+
56+
- name: Build and push
57+
uses: docker/build-push-action@v7
58+
with:
59+
context: .
60+
file: ./Dockerfile.redis-proxy
61+
platforms: linux/amd64
62+
push: ${{ github.event_name != 'pull_request' }}
63+
tags: ${{ steps.meta.outputs.tags }}
64+
labels: ${{ steps.meta.outputs.labels }}
65+
cache-from: type=gha
66+
cache-to: type=gha,mode=max

‎Dockerfile.redis-proxy‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
FROM golang:latest AS build
2+
3+
WORKDIR $GOPATH/src/app
4+
COPY . .
5+
6+
RUN CGO_ENABLED=0 go build -o /redis-proxy ./cmd/redis-proxy/
7+
8+
FROM gcr.io/distroless/static:latest
9+
COPY --from=build /redis-proxy /redis-proxy
10+
11+
ENTRYPOINT ["/redis-proxy"]

‎docs/redis-proxy-deployment.md‎

Lines changed: 285 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,285 @@
1+
# redis-proxy Deployment Guide
2+
3+
redis-proxy is a Redis-protocol reverse proxy that enables gradual migration from Redis to ElasticKV through dual-write, shadow-read comparison, and phased primary cutover.
4+
5+
## Docker Image
6+
7+
Pre-built images are published to GitHub Container Registry on every push to `main`:
8+
9+
```
10+
ghcr.io/bootjp/elastickv/redis-proxy:latest
11+
ghcr.io/bootjp/elastickv/redis-proxy:sha-<commit>
12+
```
13+
14+
The CI workflow (`.github/workflows/redis-proxy-docker.yml`) builds the image automatically when files under `cmd/redis-proxy/`, `proxy/`, or `Dockerfile.redis-proxy` change.
15+
16+
### Building locally
17+
18+
```bash
19+
# Docker
20+
docker build -f Dockerfile.redis-proxy -t redis-proxy .
21+
22+
# Binary
23+
go build -o redis-proxy ./cmd/redis-proxy/
24+
```
25+
26+
## Command-Line Options
27+
28+
| Flag | Default | Description |
29+
|------|---------|-------------|
30+
| `-listen` | `:6479` | Proxy listen address |
31+
| `-primary` | `localhost:6379` | Primary (Redis) address |
32+
| `-primary-db` | `0` | Primary Redis DB number |
33+
| `-primary-password` | (empty) | Primary Redis password |
34+
| `-secondary` | `localhost:6380` | Secondary (ElasticKV) address |
35+
| `-secondary-db` | `0` | Secondary Redis DB number |
36+
| `-secondary-password` | (empty) | Secondary Redis password |
37+
| `-mode` | `dual-write` | Proxy mode (see below) |
38+
| `-secondary-timeout` | `5s` | Secondary write timeout |
39+
| `-shadow-timeout` | `3s` | Shadow read timeout |
40+
| `-sentry-dsn` | (empty) | Sentry DSN (empty = disabled) |
41+
| `-sentry-env` | (empty) | Sentry environment name |
42+
| `-sentry-sample` | `1.0` | Sentry sample rate |
43+
| `-metrics` | `:9191` | Prometheus metrics endpoint |
44+
45+
## Proxy Modes
46+
47+
Five modes support a phased migration strategy.
48+
49+
| Mode | Reads from | Writes to | Use case |
50+
|------|-----------|-----------|----------|
51+
| `redis-only` | Redis | Redis only | Transparent proxy. Route traffic through the proxy first |
52+
| `dual-write` | Redis | Redis + ElasticKV | Begin data sync. Populate ElasticKV |
53+
| `dual-write-shadow` | Redis (+ shadow compare from ElasticKV) | Redis + ElasticKV | Verify read consistency between backends |
54+
| `elastickv-primary` | ElasticKV (+ shadow compare from Redis) | ElasticKV + Redis | Promote ElasticKV to primary. Redis as fallback |
55+
| `elastickv-only` | ElasticKV | ElasticKV only | Migration complete. Decommission Redis |
56+
57+
### Recommended Migration Path
58+
59+
```
60+
redis-only -> dual-write -> dual-write-shadow -> elastickv-primary -> elastickv-only
61+
```
62+
63+
Monitor metrics at each stage and roll back to the previous mode if issues arise. Mode changes require a proxy restart.
64+
65+
## Deployment Examples
66+
67+
### Minimal (redis-only)
68+
69+
```bash
70+
docker run --rm \
71+
ghcr.io/bootjp/elastickv/redis-proxy:latest \
72+
-listen :6379 \
73+
-primary redis.internal:6379 \
74+
-mode redis-only
75+
```
76+
77+
Point your application at the proxy. Behavior is identical to connecting directly to Redis.
78+
79+
### Dual-Write with Shadow Comparison
80+
81+
```bash
82+
docker run --rm \
83+
-p 6379:6479 \
84+
-p 9191:9191 \
85+
ghcr.io/bootjp/elastickv/redis-proxy:latest \
86+
-listen :6479 \
87+
-primary redis.internal:6379 \
88+
-primary-password "${REDIS_PASSWORD}" \
89+
-secondary elastickv.internal:6380 \
90+
-mode dual-write-shadow \
91+
-secondary-timeout 5s \
92+
-shadow-timeout 3s \
93+
-sentry-dsn "${SENTRY_DSN}" \
94+
-sentry-env production \
95+
-metrics :9191
96+
```
97+
98+
### Docker Compose
99+
100+
```yaml
101+
services:
102+
redis-proxy:
103+
image: ghcr.io/bootjp/elastickv/redis-proxy:latest
104+
ports:
105+
- "6379:6479"
106+
- "9191:9191"
107+
command:
108+
- -listen=:6479
109+
- -primary=redis:6379
110+
- -secondary=elastickv:6380
111+
- -mode=dual-write-shadow
112+
- -metrics=:9191
113+
depends_on:
114+
- redis
115+
- elastickv
116+
117+
redis:
118+
image: redis:7
119+
ports:
120+
- "6379"
121+
122+
elastickv:
123+
image: ghcr.io/bootjp/elastickv:latest
124+
ports:
125+
- "6380"
126+
```
127+
128+
### Kubernetes
129+
130+
```yaml
131+
apiVersion: apps/v1
132+
kind: Deployment
133+
metadata:
134+
name: redis-proxy
135+
spec:
136+
replicas: 1
137+
selector:
138+
matchLabels:
139+
app: redis-proxy
140+
template:
141+
metadata:
142+
labels:
143+
app: redis-proxy
144+
annotations:
145+
prometheus.io/scrape: "true"
146+
prometheus.io/port: "9191"
147+
spec:
148+
containers:
149+
- name: redis-proxy
150+
image: ghcr.io/bootjp/elastickv/redis-proxy:latest
151+
args:
152+
- -listen=:6479
153+
- -primary=redis:6379
154+
- -secondary=elastickv:6380
155+
- -mode=dual-write-shadow
156+
- -metrics=:9191
157+
ports:
158+
- containerPort: 6479
159+
name: redis
160+
- containerPort: 9191
161+
name: metrics
162+
livenessProbe:
163+
exec:
164+
command: ["redis-cli", "-p", "6479", "PING"]
165+
initialDelaySeconds: 5
166+
periodSeconds: 10
167+
readinessProbe:
168+
exec:
169+
command: ["redis-cli", "-p", "6479", "PING"]
170+
initialDelaySeconds: 3
171+
periodSeconds: 5
172+
resources:
173+
requests:
174+
cpu: 100m
175+
memory: 128Mi
176+
limits:
177+
cpu: "1"
178+
memory: 512Mi
179+
```
180+
181+
> **Note:** The distroless base image does not include `redis-cli`. For Kubernetes probes, use a `tcpSocket` probe instead or add a sidecar/init container with `redis-cli`.
182+
183+
```yaml
184+
# Alternative: TCP socket probe (no redis-cli needed)
185+
livenessProbe:
186+
tcpSocket:
187+
port: 6479
188+
initialDelaySeconds: 5
189+
periodSeconds: 10
190+
```
191+
192+
## Health Checks
193+
194+
The proxy does not expose an HTTP health endpoint. Use the Redis `PING` command to verify availability:
195+
196+
```bash
197+
redis-cli -p 6479 PING
198+
# PONG
199+
```
200+
201+
## Prometheus Metrics
202+
203+
Available at `/metrics` on the address specified by `-metrics`.
204+
205+
### Key Metrics
206+
207+
| Metric | Type | Description |
208+
|--------|------|-------------|
209+
| `proxy_command_total` | Counter | Commands processed (labels: command, backend, status) |
210+
| `proxy_command_duration_seconds` | Histogram | Backend command latency |
211+
| `proxy_primary_write_errors_total` | Counter | Primary write errors |
212+
| `proxy_secondary_write_errors_total` | Counter | Secondary write errors |
213+
| `proxy_primary_read_errors_total` | Counter | Primary read errors |
214+
| `proxy_shadow_read_errors_total` | Counter | Shadow read errors |
215+
| `proxy_divergences_total` | Counter | Shadow read mismatches (labels: command, kind) |
216+
| `proxy_migration_gap_total` | Counter | Expected mismatches from incomplete migration (labels: command) |
217+
| `proxy_async_drops_total` | Counter | Async operations dropped due to backpressure |
218+
| `proxy_active_connections` | Gauge | Current active client connections |
219+
| `proxy_pubsub_shadow_divergences_total` | Counter | Pub/Sub shadow message mismatches (labels: kind) |
220+
| `proxy_pubsub_shadow_errors_total` | Counter | Pub/Sub shadow operation errors |
221+
222+
### Recommended Alerts
223+
224+
```yaml
225+
groups:
226+
- name: redis-proxy
227+
rules:
228+
- alert: ProxyDivergenceHigh
229+
expr: rate(proxy_divergences_total[5m]) > 0
230+
for: 10m
231+
annotations:
232+
summary: "Data mismatch detected between primary and secondary"
233+
234+
- alert: ProxySecondaryWriteErrors
235+
expr: rate(proxy_secondary_write_errors_total[5m]) > 1
236+
for: 5m
237+
annotations:
238+
summary: "Secondary backend write errors are elevated"
239+
240+
- alert: ProxyAsyncDrops
241+
expr: rate(proxy_async_drops_total[5m]) > 0
242+
for: 5m
243+
annotations:
244+
summary: "Async goroutine limit reached; secondary may be slow"
245+
```
246+
247+
## Internal Parameters
248+
249+
| Parameter | Value | Description |
250+
|-----------|-------|-------------|
251+
| Connection pool size | 128 | go-redis pool size per backend |
252+
| Dial timeout | 5s | Backend connection timeout |
253+
| Read timeout | 3s | Backend read timeout |
254+
| Write timeout | 3s | Backend write timeout |
255+
| Async write goroutine limit | 4096 | Max concurrent secondary writes |
256+
| Shadow read goroutine limit | 4096 | Max concurrent shadow comparisons |
257+
| PubSub compare window | 2s | Message matching window |
258+
| PubSub sweep interval | 500ms | Expired message scan interval |
259+
260+
## Graceful Shutdown
261+
262+
The proxy handles `SIGINT` / `SIGTERM` for graceful shutdown:
263+
264+
1. Stops accepting new connections
265+
2. Waits for in-flight async goroutines to complete
266+
3. Releases backend connection pools
267+
4. Flushes Sentry buffers (up to 2 seconds)
268+
269+
Recommended shutdown order: `redis-proxy -> application -> Redis / ElasticKV`.
270+
271+
## Troubleshooting
272+
273+
### Secondary writes are falling behind
274+
- Check `proxy_async_drops_total`. If increasing, the goroutine limit is being hit.
275+
- Reduce `-secondary-timeout` to fail fast on slow secondaries.
276+
- Investigate secondary (ElasticKV) performance.
277+
278+
### High divergence count
279+
- Also check `proxy_migration_gap_total`. Pre-migration missing keys are counted as gaps, not divergences.
280+
- In `dual-write-shadow` mode, inspect `proxy_divergences_total` labels to identify which commands are mismatched.
281+
282+
### Pub/Sub messages missing
283+
- Check `proxy_pubsub_shadow_divergences_total`.
284+
- `kind=data_mismatch`: message received by primary but not secondary.
285+
- `kind=extra_data`: message received by secondary only.

0 commit comments

Comments
 (0)