云计算百科
云计算领域专业知识百科平台

使用 Docker-Compose 部署 Redis 三主三从集群(含 Exporter 监控)

使用 Docker-Compose 部署 Redis 三主三从集群(含 Exporter 监控

    • 一、环境准备
    • 二、Docker Compose 配置
    • 三、Redis 配置文件
    • 四、ACL 权限配置
    • 五、系统优化配置
    • 六、启动服务
    • 七、创建集群
    • 八、验证集群状态
    • 九、监控接入 Prometheus
    • 十、常见的场景运维
    • 十一、总结

本文将介绍如何在三台服务器上使用 Docker Compose 部署 Redis 三主三从集群,并配置 Redis Exporter 用于 Prometheus 监控。 部署的 Redis 版本为 7.2.4,并启用 ACL 用户权限控制与密码认证。

一、环境准备

三台服务器(示例):

主机名 IP domp-server-001 172.20.191.185 domp-server-002 172.20.191.186 domp-server-003 172.20.191.187

在这里插入图片描述

┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Master #1 │ │ Master #2 │ │ Master #3 │
│ 172.20.191.185:6379│ │ 172.20.191.186:6379│ │ 172.20.191.187:6379│
└───────┬──────┘ └───────┬──────┘ └───────┬──────┘
│ │ │
│ │ │
┌───────▼──────┐ ┌───────▼──────┐ ┌───────▼──────┐
│ Replica #1 │ │ Replica #2 │ │ Replica #3 │
│ 172.20.191.185:6380│ │ 172.20.191.186:6380│ │ 172.20.191.187:6380│
└─────────────────────┘ └─────────────────────┘ └─────────────────────┘

说明:
•每个 Master 对应一个 Replica(主从配对),总共 6 个节点。
•Exporter 部署在每台服务器上,分别采集 63796380 实例的监控数据。
•Redis Cluster 负责数据分片和故障转移,节点通过 cluster bus 端口(16379/16380)通信。

Exporter: – redis-exporter-1 (6379 metrics) → Prometheus – redis-exporter-2 (6380 metrics) → Prometheus

软件环境: • Docker ≥ 20.10 • Docker Compose ≥ 1.29 • Redis 镜像:reg.deeplearning.cn/mmwei3/redis:7.2.4 • Redis Exporter 镜像:reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest-20250723_x86

目录规划:

/data/redis/
├── 6379
│ ├── conf
│ └── data
└── 6380
├── conf
└── data

在每台服务器上执行:

mkdir vp /data/redis/6379/conf /data/redis/6379/data
mkdir vp /data/redis/6380/conf /data/redis/6380/data

二、Docker Compose 配置

以下示例以第一台服务器 172.20.191.185 为例(其他两台服务器需替换 cluster-announce-ip 为对应 IP)。

dockercompose.yml:

version: '3'

services:
redis-server-6379:
image: reg.deeplearning.cn/mmwei3/redis:7.2.4
container_name: redisserver6379
restart: unlessstopped
ports:
6379:6379
16379:16379
environment:
TZ: "Asia/Shanghai"
command: ["redis-server", "/6379/conf/redis.conf"]
volumes:
/data/redis/6379/conf/:/6379/conf/
/data/redis/6379/data/:/6379/data/

redis-server-6380:
image: reg.deeplearning.cn/mmwei3/redis:7.2.4
container_name: redisserver6380
restart: unlessstopped
ports:
6380:6380
16380:16380
environment:
TZ: "Asia/Shanghai"
command: ["redis-server", "/6380/conf/redis.conf"]
volumes:
/data/redis/6380/conf/:/6380/conf/
/data/redis/6380/data/:/6380/data/

redis-exporter-1:
image: reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest20250723_x86
restart: unlessstopped
ports:
9121:9121
environment:
TZ: Asia/Shanghai
REDIS_ADDR: redis://172.20.191.185:6379
REDIS_USER: redis_monitor
REDIS_PASSWORD: <监控用户密码已脱敏>

redis-exporter-2:
image: reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest20250723_x86
restart: unlessstopped
ports:
9122:9121
environment:
TZ: Asia/Shanghai
REDIS_ADDR: redis://172.20.191.185:6380
REDIS_USER: redis_monitor
REDIS_PASSWORD: <监控用户密码已脱敏>

三、Redis 配置文件

6379 节点 为例(/data/redis/6379/conf/redis.conf):

bind 0.0.0.0
port 6379
timeout 3600
tcpkeepalive 300
loglevel notice
databases 16

# 持久化策略
save 600 100
save 120 10000
save 7200 1

dir /6379/data/
masteruser clusterrepl
masterauth <复制用户密码已脱敏>

aclfile /6379/conf/users.acl
replicareadonly yes

clusterannounceip "172.20.191.185"
clusterannounceport 16379

maxclients 10000
maxmemory 10gb
maxmemorypolicy noeviction

appendonly yes
appendfsync everysec
autoaofrewritepercentage 100
autoaofrewriteminsize 64mb

# 开启集群模式
clusterenabled yes
clusterconfigfile nodes6379.conf
clusternodetimeout 15000
clusterannouncebusport 16379

6380 节点配置 /data/redis/6380/conf/redis.conf 类似,只需修改:

port 6380
dir /6380/data/
clusterannounceport 6380
clusterconfigfile nodes6380.conf
clusterannouncebusport 16380

四、ACL 权限配置

/data/redis/6379/conf/users.acl(所有节点相同,密码已脱敏):

user default off nopass &* @all

user root on >[Root密码] allkeys allchannels +@all
user redis_monitor on >[监控用户密码] +client +ping +info +config|get +cluster|info +slowlog +latency +memory +select +get +scan +xinfo +type +pfcount +strlen +llen +scard +zcard +hlen +xlen +eval allkeys
user clusterrepl on >[复制用户密码] +psync +replconf +ping
user appuser on >[应用用户密码] allkeys allchannels +@all @admin @dangerous

五、系统优化配置

# 系统参数优化
sed i '/vm.overcommit_memory/d' /etc/sysctl.conf
echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
echo "net.core.somaxconn=2048" >> /etc/sysctl.conf
sysctl p

# 禁用透明大页
cat <<EOF >> /etc/rc.d/rc.local
if test f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
EOF
chmod +x /etc/rc.d/rc.local

# 数据目录权限
chown R polkitd:root /data/redis

六、启动服务

在每台服务器执行:

docker-compose up -d

[root@dompserver003 redis]# docker compose up -d
WARN[0000] /data/docker_up/redis/dockercompose.yaml: `version` is obsolete
[+] Running 14/14
✔ redisserver6380 Pulled 2.9s
09f376ebb190 Pull complete 1.8s
✔ f89f5b53601e Pull complete 1.8s
45d9097e8d82 Pull complete 1.8s
✔ b14e0d774d3d Pull complete 1.9s
✔ e8f49d7b64bc Pull complete 2.4s
17bfefd8913f Pull complete 2.4s
4f4fb700ef54 Pull complete 2.4s
✔ d05f3a965cb3 Pull complete 2.4s
✔ redisexporter Pulled 1.0s
4f3b0056e2b7 Pull complete 0.5s
93ace162418f Pull complete 0.5s
✔ cf0c47f0538a Pull complete 0.5s
✔ redisserver6379 Pulled 2.9s
[+] Running 4/4
✔ Network redis_default Created 0.0s
✔ Container redisredisexporter1 Started 0.5s
✔ Container redisserver6380 Started 0.6s
✔ Container redisserver6379 Started 0.6s
[root@dompserver003 redis]#
[root@dompserver003 redis]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
57d66719bc31 reg.deeplearning.cn/mmwei3/redis:7.2.4 "docker-entrypoint.s…" 25 seconds ago Up 25 seconds 6379/tcp, 0.0.0.0:6380>6380/tcp, :::6380>6380/tcp redisserver6380
18ab476e8c64 reg.deeplearning.cn/mmwei3/redis:7.2.4 "docker-entrypoint.s…" 25 seconds ago Up 25 seconds 0.0.0.0:6379>6379/tcp, :::6379>6379/tcp redisserver6379
c8d4239de5a2 reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest20250723_x86 "/redis_exporter" 25 seconds ago Up 25 seconds 0.0.0.0:9121>9121/tcp, :::9121>9121/tcp redisredisexporter1
747435ab6bdb reg.deeplearning.cn/mmwei3/haproxy:latest20250725 "kolla_start" 20 hours ago Up 14 minutes haproxy
59a91a0b5223 reg.deeplearning.cn/mmwei3/keepalived:latest20250725 "kolla_start" 29 hours ago Up 14 minutes keepalived
[root@dompserver003 redis]#

docker exec it redisserver6379 rediscli \\
c \\
h 172.20.191.185 \\
p 6379 \\
user root \\
a Rootpass_c2B2h15D1PfHTOEjZd \\
cluster nodes

rediscli h <host> p <port> user root a <password> cluster nodes

[root@dompserver001 redis]# docker compose up -d
WARN[0000] /data/docker_up/redis/dockercompose.yaml: `version` is obsolete
[+] Running 3/3
✔ Container redisredisexporter1 Running 0.0s
✔ Container redisserver6379 Started 0.5s
✔ Container redisserver6380 Started 0.5s
[root@dompserver001 redis]# docker exec -it redis-server-6379 bash
root@ab02a014ade5:/data# history
1 history
root@ab02a014ade5:/data# redis-cli \\
user root a Rootpass_c2B2h15D1PfHTOEjZd \\
cluster create \\
172.20.191.185:6379 172.20.191.185:6380 \\
172.20.191.186:6379 172.20.191.186:6380 \\
172.20.191.187:6379 172.20.191.187:6380 \\
clusterreplicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] > Slots 0 5460
Master[1] > Slots 5461 10922
Master[2] > Slots 10923 16383
Adding replica 172.20.191.186:6380 to 172.20.191.185:6379
Adding replica 172.20.191.187:6380 to 172.20.191.186:6379
Adding replica 172.20.191.185:6380 to 172.20.191.187:6379
M: 119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379
slots:[05460] (5461 slots) master
S: 7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380
replicates 933f4247253f383476c37f4a76f3fc3b4374c6d8
M: 2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379
slots:[546110922] (5462 slots) master
S: 779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380
replicates 119195a14a2fc9ca186ce439c0d15d587553004e
M: 933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379
slots:[1092316383] (5461 slots) master
S: 02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380
replicates 2636af29cc2d6c6badf76355c923c5c2c6503a33
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 172.20.191.185:6379)
M: 119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379
slots:[05460] (5461 slots) master
1 additional replica(s)
S: 779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380
slots: (0 slots) slave
replicates 119195a14a2fc9ca186ce439c0d15d587553004e
S: 02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380
slots: (0 slots) slave
replicates 2636af29cc2d6c6badf76355c923c5c2c6503a33
M: 933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379
slots:[1092316383] (5461 slots) master
1 additional replica(s)
M: 2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379
slots:[546110922] (5462 slots) master
1 additional replica(s)
S: 7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380
slots: (0 slots) slave
replicates 933f4247253f383476c37f4a76f3fc3b4374c6d8
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
root@ab02a014ade5:/data#
root@ab02a014ade5:/data#

七、创建集群

在其中一台服务器执行:

docker exec -it redis-server-6379 redis-cli –user root -a <Root密码已脱敏> –cluster create 172.20.191.185:6379 172.20.191.185:6380 172.20.191.186:6379 172.20.191.186:6380 172.20.191.187:6379 172.20.191.187:6380 –cluster-replicas 1

执行过程中会提示确认,输入 yes 即可完成集群初始化。

八、验证集群状态

docker exec it redisserver6379 rediscli \\
user root a <Root密码已脱敏> \\
cluster info

docker exec it redisserver6379 rediscli \\
user root a <Root密码已脱敏> \\
cluster nodes

[root@dompserver001 redis]# cat /data/redis/6379/data/nodes-6379.conf
779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380@16380,,tlsport=0,shardid=99b8a25667e3a4caaee705c15c9056196f417d9e slave 119195a14a2fc9ca186ce439c0d15d587553004e 0 1753877534397 1 connected
02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380@16380,,tlsport=0,shardid=1f2d74eabc1a9193db35317864bc5e6e0840523a slave 2636af29cc2d6c6badf76355c923c5c2c6503a33 0 1753877533000 3 connected
119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379@16379,,tlsport=0,shardid=99b8a25667e3a4caaee705c15c9056196f417d9e myself,master 0 1753877533000 1 connected 05460
933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379@16379,,tlsport=0,shardid=de5219610a55a82ba3361d902ecec170a6e54dae master 0 1753877534000 5 connected 1092316383
2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379@16379,,tlsport=0,shardid=1f2d74eabc1a9193db35317864bc5e6e0840523a master 0 1753877535399 3 connected 546110922
7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380@16380,,tlsport=0,shardid=de5219610a55a82ba3361d902ecec170a6e54dae slave 933f4247253f383476c37f4a76f3fc3b4374c6d8 0 1753877533000 5 connected
vars currentEpoch 6 lastVoteEpoch 0
[root@dompserver001 redis]#
[root@dompserver001 redis]# docker exec -it redis-server-6379 redis-cli \\
> c \\
> h 172.20.191.185 \\
> p 6379 \\
> user root \\
> a Rootpass_c2B2h15D1PfHTOEjZd \\
> cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380@16380 slave 119195a14a2fc9ca186ce439c0d15d587553004e 0 1753877694771 1 connected
02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380@16380 slave 2636af29cc2d6c6badf76355c923c5c2c6503a33 0 1753877691764 3 connected
119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379@16379 myself,master 0 1753877692000 1 connected 05460
933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379@16379 master 0 1753877693769 5 connected 1092316383
2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379@16379 master 0 1753877691000 3 connected 546110922
7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380@16380 slave 933f4247253f383476c37f4a76f3fc3b4374c6d8 0 1753877692766 5 connected
[root@dompserver001 redis]#
[root@dompserver001 redis]# docker exec -it redis-server-6379 redis-cli -c -h 172.20.191.185 -p 6379 –user root -a Rootpass_c2B2h15D1PfHTOEjZd cluster info
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:180
cluster_stats_messages_pong_sent:185
cluster_stats_messages_sent:365
cluster_stats_messages_ping_received:180
cluster_stats_messages_pong_received:180
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:365
total_cluster_links_buffer_limit_exceeded:0
[root@dompserver001 redis]#

export CNNL_LOG_LEVEL=error

九、监控接入 Prometheus

Redis Exporter 已在 Compose 中启动:
9121 端口对应 6379
9122 端口对应 6380

Prometheus 配置示例:

job_name: 'redis-6379'
static_configs:
targets: ['172.20.191.185:9121']

job_name: 'redis-6380'
static_configs:
targets: ['172.20.191.185:9122']

十、常见的场景运维

1. 集群扩容 / 缩容

扩容

# 新节点加入集群(作为master)
redis-cli –cluster add-node 172.20.191.188:6379 172.20.191.185:6379

# 分配槽位(例如新节点分配 5000 槽)
redis-cli –cluster reshard 172.20.191.185:6379 \\
–cluster-from all \\
–cluster-to <新节点ID> \\
–cluster-slots 5000 \\
–cluster-yes

缩容

# 将指定节点的槽位迁移给其他节点
redis-cli –cluster reshard 172.20.191.185:6379 \\
–cluster-from <要下线节点ID> \\
–cluster-to <目标节点ID> \\
–cluster-slots <迁移槽位数> \\
–cluster-yes

# 删除节点
redis-cli –cluster del-node 172.20.191.185:6379 <要下线节点ID>

2. 主从切换与故障转移

# 手动触发某个 Replica 升级为 Master
redis-cli -h 172.20.191.186 -p 6380 CLUSTER FAILOVER

# 查看当前节点角色
redis-cli -h 172.20.191.185 -p 6379 ROLE

3. 数据迁移与重分片

# 进入交互式重分片
redis-cli –cluster reshard 172.20.191.185:6379

按照提示输入:
1.接收槽位的节点ID
2.迁移的槽位数量
3.来源节点ID列表(all 表示全部节点)

4. 密码与 ACL 管理

# 设置集群访问密码(配置文件 redis.conf)
requirepass MyStrongPass

# 创建监控用户,仅允许 INFO/PING
redis-cli -a MyStrongPass ACL SETUSER monitor_user on >monitor_pass +INFO +PING

5. 性能优化

# 查看当前内存淘汰策略
redis-cli CONFIG GET maxmemory-policy

# 设置淘汰策略为 allkeys-lru
redis-cli CONFIG SET maxmemory-policy allkeys-lru

# 批量写入(pipeline示例)
(echo -en "SET key1 val1\\r\\nSET key2 val2\\r\\nSET key3 val3\\r\\n"; sleep 1) | nc 172.20.191.185 6379

6. 监控与告警

# 查看内存使用情况
redis-cli INFO memory

# 查看主从延迟
redis-cli INFO replication

# 查看慢查询日志
redis-cli SLOWLOG GET 10

7. 升级与维护

# 升级流程示例(先从库后主库)
# Replica 升级完成后手动切换为 Master
redis-cli -h 172.20.191.186 -p 6380 CLUSTER FAILOVER

8. 故障排查

# 查看集群状态
redis-cli -h 172.20.191.185 -p 6379 CLUSTER INFO
redis-cli -h 172.20.191.185 -p 6379 CLUSTER NODES

# 检查 AOF 文件完整性
redis-check-aof appendonly.aof

# 检查 RDB 文件
redis-check-rdb dump.rdb

十一、总结

本文介绍了如何: • 使用 Docker Compose 部署多实例 Redis • 配置 三主三从集群 • 启用 ACL 用户与密码认证 • 接入 Redis Exporter 进行监控

赞(0)
未经允许不得转载:网硕互联帮助中心 » 使用 Docker-Compose 部署 Redis 三主三从集群(含 Exporter 监控)
分享到: 更多 (0)

评论 抢沙发

评论前必须登录!