使用 Docker-Compose 部署 Redis 三主三从集群(含 Exporter 监控
-
- 一、环境准备
- 二、Docker Compose 配置
- 三、Redis 配置文件
- 四、ACL 权限配置
- 五、系统优化配置
- 六、启动服务
- 七、创建集群
- 八、验证集群状态
- 九、监控接入 Prometheus
- 十、常见的场景运维
- 十一、总结
本文将介绍如何在三台服务器上使用 Docker Compose 部署 Redis 三主三从集群,并配置 Redis Exporter 用于 Prometheus 监控。 部署的 Redis 版本为 7.2.4,并启用 ACL 用户权限控制与密码认证。
一、环境准备
三台服务器(示例):
主机名 IP domp-server-001 172.20.191.185 domp-server-002 172.20.191.186 domp-server-003 172.20.191.187
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Master #1 │ │ Master #2 │ │ Master #3 │
│ 172.20.191.185:6379│ │ 172.20.191.186:6379│ │ 172.20.191.187:6379│
└───────┬──────┘ └───────┬──────┘ └───────┬──────┘
│ │ │
│ │ │
┌───────▼──────┐ ┌───────▼──────┐ ┌───────▼──────┐
│ Replica #1 │ │ Replica #2 │ │ Replica #3 │
│ 172.20.191.185:6380│ │ 172.20.191.186:6380│ │ 172.20.191.187:6380│
└─────────────────────┘ └─────────────────────┘ └─────────────────────┘
说明:
•每个 Master 对应一个 Replica(主从配对),总共 6 个节点。
•Exporter 部署在每台服务器上,分别采集 6379、6380 实例的监控数据。
•Redis Cluster 负责数据分片和故障转移,节点通过 cluster bus 端口(16379/16380)通信。
Exporter: – redis-exporter-1 (6379 metrics) → Prometheus – redis-exporter-2 (6380 metrics) → Prometheus
软件环境: • Docker ≥ 20.10 • Docker Compose ≥ 1.29 • Redis 镜像:reg.deeplearning.cn/mmwei3/redis:7.2.4 • Redis Exporter 镜像:reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest-20250723_x86
目录规划:
/data/redis/
├── 6379
│ ├── conf
│ └── data
└── 6380
├── conf
└── data
在每台服务器上执行:
mkdir –vp /data/redis/6379/conf /data/redis/6379/data
mkdir –vp /data/redis/6380/conf /data/redis/6380/data
⸻
二、Docker Compose 配置
以下示例以第一台服务器 172.20.191.185 为例(其他两台服务器需替换 cluster-announce-ip 为对应 IP)。
docker–compose.yml:
version: '3'
services:
redis-server-6379:
image: reg.deeplearning.cn/mmwei3/redis:7.2.4
container_name: redis–server–6379
restart: unless–stopped
ports:
– 6379:6379
– 16379:16379
environment:
TZ: "Asia/Shanghai"
command: ["redis-server", "/6379/conf/redis.conf"]
volumes:
– /data/redis/6379/conf/:/6379/conf/
– /data/redis/6379/data/:/6379/data/
redis-server-6380:
image: reg.deeplearning.cn/mmwei3/redis:7.2.4
container_name: redis–server–6380
restart: unless–stopped
ports:
– 6380:6380
– 16380:16380
environment:
TZ: "Asia/Shanghai"
command: ["redis-server", "/6380/conf/redis.conf"]
volumes:
– /data/redis/6380/conf/:/6380/conf/
– /data/redis/6380/data/:/6380/data/
redis-exporter-1:
image: reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest–20250723_x86
restart: unless–stopped
ports:
– 9121:9121
environment:
TZ: Asia/Shanghai
REDIS_ADDR: redis://172.20.191.185:6379
REDIS_USER: redis_monitor
REDIS_PASSWORD: <监控用户密码已脱敏>
redis-exporter-2:
image: reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest–20250723_x86
restart: unless–stopped
ports:
– 9122:9121
environment:
TZ: Asia/Shanghai
REDIS_ADDR: redis://172.20.191.185:6380
REDIS_USER: redis_monitor
REDIS_PASSWORD: <监控用户密码已脱敏>
⸻
三、Redis 配置文件
以 6379 节点 为例(/data/redis/6379/conf/redis.conf):
bind 0.0.0.0
port 6379
timeout 3600
tcp–keepalive 300
loglevel notice
databases 16
# 持久化策略
save 600 100
save 120 10000
save 7200 1
dir /6379/data/
masteruser clusterrepl
masterauth <复制用户密码已脱敏>
aclfile /6379/conf/users.acl
replica–read–only yes
cluster–announce–ip "172.20.191.185"
cluster–announce–port 16379
maxclients 10000
maxmemory 10gb
maxmemory–policy noeviction
appendonly yes
appendfsync everysec
auto–aof–rewrite–percentage 100
auto–aof–rewrite–min–size 64mb
# 开启集群模式
cluster–enabled yes
cluster–config–file nodes–6379.conf
cluster–node–timeout 15000
cluster–announce–bus–port 16379
6380 节点配置 /data/redis/6380/conf/redis.conf 类似,只需修改:
port 6380
dir /6380/data/
cluster–announce–port 6380
cluster–config–file nodes–6380.conf
cluster–announce–bus–port 16380
⸻
四、ACL 权限配置
/data/redis/6379/conf/users.acl(所有节点相同,密码已脱敏):
user default off nopass &* –@all
user root on >[Root密码] allkeys allchannels +@all
user redis_monitor on >[监控用户密码] +client +ping +info +config|get +cluster|info +slowlog +latency +memory +select +get +scan +xinfo +type +pfcount +strlen +llen +scard +zcard +hlen +xlen +eval allkeys
user clusterrepl on >[复制用户密码] +psync +replconf +ping
user appuser on >[应用用户密码] allkeys allchannels +@all –@admin –@dangerous
⸻
五、系统优化配置
# 系统参数优化
sed –i '/vm.overcommit_memory/d' /etc/sysctl.conf
echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
echo "net.core.somaxconn=2048" >> /etc/sysctl.conf
sysctl –p
# 禁用透明大页
cat <<EOF >> /etc/rc.d/rc.local
if test –f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
EOF
chmod +x /etc/rc.d/rc.local
# 数据目录权限
chown –R polkitd:root /data/redis
⸻
六、启动服务
在每台服务器执行:
docker-compose up -d
[root@domp–server–003 redis]# docker compose up -d
WARN[0000] /data/docker_up/redis/docker–compose.yaml: `version` is obsolete
[+] Running 14/14
✔ redis–server–6380 Pulled 2.9s
✔ 09f376ebb190 Pull complete 1.8s
✔ f89f5b53601e Pull complete 1.8s
✔ 45d9097e8d82 Pull complete 1.8s
✔ b14e0d774d3d Pull complete 1.9s
✔ e8f49d7b64bc Pull complete 2.4s
✔ 17bfefd8913f Pull complete 2.4s
✔ 4f4fb700ef54 Pull complete 2.4s
✔ d05f3a965cb3 Pull complete 2.4s
✔ redis–exporter Pulled 1.0s
✔ 4f3b0056e2b7 Pull complete 0.5s
✔ 93ace162418f Pull complete 0.5s
✔ cf0c47f0538a Pull complete 0.5s
✔ redis–server–6379 Pulled 2.9s
[+] Running 4/4
✔ Network redis_default Created 0.0s
✔ Container redis–redis–exporter–1 Started 0.5s
✔ Container redis–server–6380 Started 0.6s
✔ Container redis–server–6379 Started 0.6s
[root@domp–server–003 redis]#
[root@domp–server–003 redis]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
57d66719bc31 reg.deeplearning.cn/mmwei3/redis:7.2.4 "docker-entrypoint.s…" 25 seconds ago Up 25 seconds 6379/tcp, 0.0.0.0:6380–>6380/tcp, :::6380–>6380/tcp redis–server–6380
18ab476e8c64 reg.deeplearning.cn/mmwei3/redis:7.2.4 "docker-entrypoint.s…" 25 seconds ago Up 25 seconds 0.0.0.0:6379–>6379/tcp, :::6379–>6379/tcp redis–server–6379
c8d4239de5a2 reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest–20250723_x86 "/redis_exporter" 25 seconds ago Up 25 seconds 0.0.0.0:9121–>9121/tcp, :::9121–>9121/tcp redis–redis–exporter–1
747435ab6bdb reg.deeplearning.cn/mmwei3/haproxy:latest–20250725 "kolla_start" 20 hours ago Up 14 minutes haproxy
59a91a0b5223 reg.deeplearning.cn/mmwei3/keepalived:latest–20250725 "kolla_start" 29 hours ago Up 14 minutes keepalived
[root@domp–server–003 redis]#
docker exec –it redis–server–6379 redis–cli \\
–c \\
–h 172.20.191.185 \\
–p 6379 \\
––user root \\
–a Rootpass_c2B2h15D1PfHTOEjZd \\
cluster nodes
redis–cli –h <host> –p <port> ––user root –a <password> cluster nodes
[root@domp–server–001 redis]# docker compose up -d
WARN[0000] /data/docker_up/redis/docker–compose.yaml: `version` is obsolete
[+] Running 3/3
✔ Container redis–redis–exporter–1 Running 0.0s
✔ Container redis–server–6379 Started 0.5s
✔ Container redis–server–6380 Started 0.5s
[root@domp–server–001 redis]# docker exec -it redis-server-6379 bash
root@ab02a014ade5:/data# history
1 history
root@ab02a014ade5:/data# redis-cli \\
––user root –a Rootpass_c2B2h15D1PfHTOEjZd \\
––cluster create \\
172.20.191.185:6379 172.20.191.185:6380 \\
172.20.191.186:6379 172.20.191.186:6380 \\
172.20.191.187:6379 172.20.191.187:6380 \\
––cluster–replicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] –> Slots 0 – 5460
Master[1] –> Slots 5461 – 10922
Master[2] –> Slots 10923 – 16383
Adding replica 172.20.191.186:6380 to 172.20.191.185:6379
Adding replica 172.20.191.187:6380 to 172.20.191.186:6379
Adding replica 172.20.191.185:6380 to 172.20.191.187:6379
M: 119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379
slots:[0–5460] (5461 slots) master
S: 7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380
replicates 933f4247253f383476c37f4a76f3fc3b4374c6d8
M: 2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379
slots:[5461–10922] (5462 slots) master
S: 779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380
replicates 119195a14a2fc9ca186ce439c0d15d587553004e
M: 933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379
slots:[10923–16383] (5461 slots) master
S: 02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380
replicates 2636af29cc2d6c6badf76355c923c5c2c6503a33
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 172.20.191.185:6379)
M: 119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379
slots:[0–5460] (5461 slots) master
1 additional replica(s)
S: 779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380
slots: (0 slots) slave
replicates 119195a14a2fc9ca186ce439c0d15d587553004e
S: 02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380
slots: (0 slots) slave
replicates 2636af29cc2d6c6badf76355c923c5c2c6503a33
M: 933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379
slots:[10923–16383] (5461 slots) master
1 additional replica(s)
M: 2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379
slots:[5461–10922] (5462 slots) master
1 additional replica(s)
S: 7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380
slots: (0 slots) slave
replicates 933f4247253f383476c37f4a76f3fc3b4374c6d8
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
root@ab02a014ade5:/data#
root@ab02a014ade5:/data#
⸻
七、创建集群
在其中一台服务器执行:
docker exec -it redis-server-6379 redis-cli –user root -a <Root密码已脱敏> –cluster create 172.20.191.185:6379 172.20.191.185:6380 172.20.191.186:6379 172.20.191.186:6380 172.20.191.187:6379 172.20.191.187:6380 –cluster-replicas 1
执行过程中会提示确认,输入 yes 即可完成集群初始化。
⸻
八、验证集群状态
docker exec –it redis–server–6379 redis–cli \\
––user root –a <Root密码已脱敏> \\
cluster info
docker exec –it redis–server–6379 redis–cli \\
––user root –a <Root密码已脱敏> \\
cluster nodes
[root@domp–server–001 redis]# cat /data/redis/6379/data/nodes-6379.conf
779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380@16380,,tls–port=0,shard–id=99b8a25667e3a4caaee705c15c9056196f417d9e slave 119195a14a2fc9ca186ce439c0d15d587553004e 0 1753877534397 1 connected
02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380@16380,,tls–port=0,shard–id=1f2d74eabc1a9193db35317864bc5e6e0840523a slave 2636af29cc2d6c6badf76355c923c5c2c6503a33 0 1753877533000 3 connected
119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379@16379,,tls–port=0,shard–id=99b8a25667e3a4caaee705c15c9056196f417d9e myself,master – 0 1753877533000 1 connected 0–5460
933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379@16379,,tls–port=0,shard–id=de5219610a55a82ba3361d902ecec170a6e54dae master – 0 1753877534000 5 connected 10923–16383
2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379@16379,,tls–port=0,shard–id=1f2d74eabc1a9193db35317864bc5e6e0840523a master – 0 1753877535399 3 connected 5461–10922
7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380@16380,,tls–port=0,shard–id=de5219610a55a82ba3361d902ecec170a6e54dae slave 933f4247253f383476c37f4a76f3fc3b4374c6d8 0 1753877533000 5 connected
vars currentEpoch 6 lastVoteEpoch 0
[root@domp–server–001 redis]#
[root@domp–server–001 redis]# docker exec -it redis-server-6379 redis-cli \\
> –c \\
> –h 172.20.191.185 \\
> –p 6379 \\
> ––user root \\
> –a Rootpass_c2B2h15D1PfHTOEjZd \\
> cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380@16380 slave 119195a14a2fc9ca186ce439c0d15d587553004e 0 1753877694771 1 connected
02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380@16380 slave 2636af29cc2d6c6badf76355c923c5c2c6503a33 0 1753877691764 3 connected
119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379@16379 myself,master – 0 1753877692000 1 connected 0–5460
933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379@16379 master – 0 1753877693769 5 connected 10923–16383
2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379@16379 master – 0 1753877691000 3 connected 5461–10922
7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380@16380 slave 933f4247253f383476c37f4a76f3fc3b4374c6d8 0 1753877692766 5 connected
[root@domp–server–001 redis]#
[root@domp–server–001 redis]# docker exec -it redis-server-6379 redis-cli -c -h 172.20.191.185 -p 6379 –user root -a Rootpass_c2B2h15D1PfHTOEjZd cluster info
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:180
cluster_stats_messages_pong_sent:185
cluster_stats_messages_sent:365
cluster_stats_messages_ping_received:180
cluster_stats_messages_pong_received:180
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:365
total_cluster_links_buffer_limit_exceeded:0
[root@domp–server–001 redis]#
export CNNL_LOG_LEVEL=error
⸻
九、监控接入 Prometheus
Redis Exporter 已在 Compose 中启动:
•9121 端口对应 6379
•9122 端口对应 6380
Prometheus 配置示例:
– job_name: 'redis-6379'
static_configs:
– targets: ['172.20.191.185:9121']
– job_name: 'redis-6380'
static_configs:
– targets: ['172.20.191.185:9122']
⸻
十、常见的场景运维
1. 集群扩容 / 缩容
扩容
# 新节点加入集群(作为master)
redis-cli –cluster add-node 172.20.191.188:6379 172.20.191.185:6379
# 分配槽位(例如新节点分配 5000 槽)
redis-cli –cluster reshard 172.20.191.185:6379 \\
–cluster-from all \\
–cluster-to <新节点ID> \\
–cluster-slots 5000 \\
–cluster-yes
缩容
# 将指定节点的槽位迁移给其他节点
redis-cli –cluster reshard 172.20.191.185:6379 \\
–cluster-from <要下线节点ID> \\
–cluster-to <目标节点ID> \\
–cluster-slots <迁移槽位数> \\
–cluster-yes
# 删除节点
redis-cli –cluster del-node 172.20.191.185:6379 <要下线节点ID>
⸻
2. 主从切换与故障转移
# 手动触发某个 Replica 升级为 Master
redis-cli -h 172.20.191.186 -p 6380 CLUSTER FAILOVER
# 查看当前节点角色
redis-cli -h 172.20.191.185 -p 6379 ROLE
⸻
3. 数据迁移与重分片
# 进入交互式重分片
redis-cli –cluster reshard 172.20.191.185:6379
按照提示输入:
1.接收槽位的节点ID
2.迁移的槽位数量
3.来源节点ID列表(all 表示全部节点)
⸻
4. 密码与 ACL 管理
# 设置集群访问密码(配置文件 redis.conf)
requirepass MyStrongPass
# 创建监控用户,仅允许 INFO/PING
redis-cli -a MyStrongPass ACL SETUSER monitor_user on >monitor_pass +INFO +PING
⸻
5. 性能优化
# 查看当前内存淘汰策略
redis-cli CONFIG GET maxmemory-policy
# 设置淘汰策略为 allkeys-lru
redis-cli CONFIG SET maxmemory-policy allkeys-lru
# 批量写入(pipeline示例)
(echo -en "SET key1 val1\\r\\nSET key2 val2\\r\\nSET key3 val3\\r\\n"; sleep 1) | nc 172.20.191.185 6379
⸻
6. 监控与告警
# 查看内存使用情况
redis-cli INFO memory
# 查看主从延迟
redis-cli INFO replication
# 查看慢查询日志
redis-cli SLOWLOG GET 10
⸻
7. 升级与维护
# 升级流程示例(先从库后主库)
# Replica 升级完成后手动切换为 Master
redis-cli -h 172.20.191.186 -p 6380 CLUSTER FAILOVER
⸻
8. 故障排查
# 查看集群状态
redis-cli -h 172.20.191.185 -p 6379 CLUSTER INFO
redis-cli -h 172.20.191.185 -p 6379 CLUSTER NODES
# 检查 AOF 文件完整性
redis-check-aof appendonly.aof
# 检查 RDB 文件
redis-check-rdb dump.rdb
⸻
十一、总结
本文介绍了如何: • 使用 Docker Compose 部署多实例 Redis • 配置 三主三从集群 • 启用 ACL 用户与密码认证 • 接入 Redis Exporter 进行监控
评论前必须登录!
注册