|
| 1 | +# RedisShake Installation & Usage Guide |
| 2 | + |
| 3 | +> This document helps LLM agents quickly install and configure RedisShake for Redis data migration tasks. |
| 4 | +> |
| 5 | +> - Official Documentation: [Chinese](https://tair-opensource.github.io/RedisShake/) | [English](https://tair-opensource.github.io/RedisShake/en/) |
| 6 | +> - Releases: https://github.com/tair-opensource/RedisShake/releases |
| 7 | +> - Docker Image: `ghcr.io/tair-opensource/redisshake` |
| 8 | +
|
| 9 | +## What is RedisShake |
| 10 | + |
| 11 | +RedisShake is a Redis data transformation and migration tool. It moves data between Redis instances with zero downtime, supporting Redis 2.8–8.x and Valkey 8.x–9.x across standalone, master-slave, sentinel, and cluster deployments. It also works with cloud services like Alibaba Cloud Tair, AWS ElastiCache, and AWS MemoryDB. |
| 12 | + |
| 13 | +## Install |
| 14 | + |
| 15 | +### Option 1: Download Binary |
| 16 | + |
| 17 | +Download the latest release from https://github.com/tair-opensource/RedisShake/releases |
| 18 | + |
| 19 | +### Option 2: Docker |
| 20 | + |
| 21 | +```shell |
| 22 | +docker run --network host \ |
| 23 | + -e SYNC=true \ |
| 24 | + -e SHAKE_SRC_ADDRESS=127.0.0.1:6379 \ |
| 25 | + -e SHAKE_DST_ADDRESS=127.0.0.1:6380 \ |
| 26 | + ghcr.io/tair-opensource/redisshake:latest |
| 27 | +``` |
| 28 | + |
| 29 | +### Option 3: Build from Source |
| 30 | + |
| 31 | +```shell |
| 32 | +git clone https://github.com/tair-opensource/RedisShake |
| 33 | +cd RedisShake |
| 34 | +sh build.sh |
| 35 | +``` |
| 36 | + |
| 37 | +## Usage |
| 38 | + |
| 39 | +Create a `shake.toml` config file, then run: |
| 40 | + |
| 41 | +```shell |
| 42 | +./redis-shake shake.toml |
| 43 | +``` |
| 44 | + |
| 45 | +## Configuration |
| 46 | + |
| 47 | +A config file has one **reader** section and one **writer** section. Choose one reader and one writer based on your scenario. |
| 48 | + |
| 49 | +### Readers (choose one) |
| 50 | + |
| 51 | +**sync_reader** — Real-time sync via PSync protocol. Best for migration with minimal downtime. See [sync_reader docs](https://tair-opensource.github.io/RedisShake/en/reader/sync_reader.html). |
| 52 | + |
| 53 | +```toml |
| 54 | +[sync_reader] |
| 55 | +cluster = false # set to true if source is a Redis cluster |
| 56 | +address = "127.0.0.1:6379" # for cluster, any node address works |
| 57 | +username = "" # keep empty if not using ACL |
| 58 | +password = "" # keep empty if no authentication |
| 59 | +tls = false |
| 60 | +sync_rdb = true # set to false to skip RDB phase |
| 61 | +sync_aof = true # set to false to skip AOF phase |
| 62 | +prefer_replica = false # set to true to sync from replica |
| 63 | +try_diskless = false # set to true for diskless sync |
| 64 | +``` |
| 65 | + |
| 66 | +**scan_reader** — Scan-based one-time migration. Use when PSync is unavailable. See [scan_reader docs](https://tair-opensource.github.io/RedisShake/en/reader/scan_reader.html). |
| 67 | + |
| 68 | +```toml |
| 69 | +[scan_reader] |
| 70 | +cluster = false |
| 71 | +address = "127.0.0.1:6379" |
| 72 | +username = "" |
| 73 | +password = "" |
| 74 | +tls = false |
| 75 | +dbs = [] # e.g. [0, 1, 2] to scan specific DBs; empty = all |
| 76 | +scan = true |
| 77 | +ksn = false # set to true to enable keyspace notifications |
| 78 | +count = 1 # keys per scan iteration |
| 79 | +``` |
| 80 | + |
| 81 | +**rdb_reader** — Import from an RDB file. See [rdb_reader docs](https://tair-opensource.github.io/RedisShake/en/reader/rdb_reader.html). |
| 82 | + |
| 83 | +```toml |
| 84 | +[rdb_reader] |
| 85 | +filepath = "/path/to/dump.rdb" |
| 86 | +``` |
| 87 | + |
| 88 | +**aof_reader** — Import from an AOF file. |
| 89 | + |
| 90 | +```toml |
| 91 | +[aof_reader] |
| 92 | +filepath = "/path/to/appendonly.aof" |
| 93 | +timestamp = 0 |
| 94 | +``` |
| 95 | + |
| 96 | +### Writers (choose one) |
| 97 | + |
| 98 | +**redis_writer** — Write to a Redis instance. |
| 99 | + |
| 100 | +```toml |
| 101 | +[redis_writer] |
| 102 | +cluster = false |
| 103 | +address = "127.0.0.1:6380" |
| 104 | +username = "" |
| 105 | +password = "" |
| 106 | +tls = false |
| 107 | +off_reply = false # turn off server reply for performance |
| 108 | +``` |
| 109 | + |
| 110 | +**file_writer** — Export to a file. |
| 111 | + |
| 112 | +```toml |
| 113 | +[file_writer] |
| 114 | +filepath = "/path/to/output.aof" |
| 115 | +type = "aof" # "cmd", "aof", or "json" |
| 116 | +``` |
| 117 | + |
| 118 | +### Filter |
| 119 | + |
| 120 | +Filter keys, databases, or commands during migration. See [filter docs](https://tair-opensource.github.io/RedisShake/en/filter/filter.html). |
| 121 | + |
| 122 | +```toml |
| 123 | +[filter] |
| 124 | +allow_keys = [] # e.g. ["user:1001", "product:2001"] |
| 125 | +allow_key_prefix = [] # e.g. ["user:", "product:"] |
| 126 | +allow_key_suffix = [] |
| 127 | +allow_key_regex = [] # e.g. [":\\d{11}:"] |
| 128 | +block_keys = [] |
| 129 | +block_key_prefix = [] # e.g. ["temp:", "cache:"] |
| 130 | +block_key_suffix = [] |
| 131 | +block_key_regex = [] |
| 132 | + |
| 133 | +allow_db = [] # e.g. [0, 1, 2] |
| 134 | +block_db = [] # e.g. [3, 4, 5] |
| 135 | + |
| 136 | +allow_command = [] |
| 137 | +block_command = [] # e.g. ["FLUSHALL", "FLUSHDB"] |
| 138 | +allow_command_group = [] |
| 139 | +block_command_group = [] |
| 140 | + |
| 141 | +# Lua function for custom data processing |
| 142 | +# See: https://tair-opensource.github.io/RedisShake/en/filter/function.html |
| 143 | +function = "" |
| 144 | +``` |
| 145 | + |
| 146 | +### Advanced |
| 147 | + |
| 148 | +```toml |
| 149 | +[advanced] |
| 150 | +dir = "data" |
| 151 | +ncpu = 0 # 0 = use all CPU cores |
| 152 | +pprof_port = 0 # 0 = disable |
| 153 | +status_port = 0 # set a port to enable HTTP status monitoring |
| 154 | + |
| 155 | +log_file = "shake.log" |
| 156 | +log_level = "info" # debug, info, warn |
| 157 | +log_interval = 5 # seconds |
| 158 | + |
| 159 | +# How to handle "Target key name is busy" during RESTORE |
| 160 | +# "panic" = stop, "rewrite" = replace, "skip" = skip the key |
| 161 | +rdb_restore_command_behavior = "panic" |
| 162 | + |
| 163 | +pipeline_count_limit = 1024 |
| 164 | +target_redis_max_qps = 300000 |
| 165 | +target_redis_proto_max_bulk_len = 512_000_000 |
| 166 | + |
| 167 | +empty_db_before_sync = false |
| 168 | +``` |
| 169 | + |
| 170 | +## Example: Migrate between Two Redis Instances |
| 171 | + |
| 172 | +```toml |
| 173 | +[sync_reader] |
| 174 | +address = "10.0.0.1:6379" |
| 175 | +password = "source_password" |
| 176 | + |
| 177 | +[redis_writer] |
| 178 | +address = "10.0.0.2:6380" |
| 179 | +password = "target_password" |
| 180 | + |
| 181 | +[filter] |
| 182 | +block_key_prefix = ["temp:", "cache:"] |
| 183 | +``` |
| 184 | + |
| 185 | +```shell |
| 186 | +./redis-shake shake.toml |
| 187 | +``` |
| 188 | + |
| 189 | +## Example: Docker with Environment Variables |
| 190 | + |
| 191 | +For the simplest sync scenario, no config file is needed: |
| 192 | + |
| 193 | +```shell |
| 194 | +docker run --network host \ |
| 195 | + -e SYNC=true \ |
| 196 | + -e SHAKE_SRC_ADDRESS=10.0.0.1:6379 \ |
| 197 | + -e SHAKE_DST_ADDRESS=10.0.0.2:6380 \ |
| 198 | + ghcr.io/tair-opensource/redisshake:latest |
| 199 | +``` |
| 200 | + |
| 201 | +## Limitations |
| 202 | + |
| 203 | +- **No checkpoint/resume**: RedisShake 4.x does not support resumable transfer. Restart means full resync from scratch. |
| 204 | +- **Static cluster topology**: Cluster topology changes (failover, scaling, slot migration) will cause panic. RedisShake is best suited for one-time migration, not long-term synchronization. |
0 commit comments