526互联

在kubernetes中部署redis-cluster集群

发布时间 2023-04-26 17:43:36作者: 天海沙

使用k8s的DaemonSet创建6个节点的redis-cluster集群, 集群为3主3从, 所以3个控制平台(主)也要部署上pod, redis集群IP如下: 

(注:用StatfulSet也可以, 大同小异)

    192.168.10.34:7000 
    192.168.10.35:7000 
    192.168.10.37:7000 
    192.168.10.38:7000 
    192.168.10.39:7000 
    192.168.10.40:7000 

 

1. 所有节点,都需要创建目录 

cat >release-redis-cluster.sh<<DD
mkdir -p /data/kubernetes/redis-cluster/
mkdir -p /data/kubernetes/redis-cluster/dump
mkdir -p /data/kubernetes/redis-cluster/conf
mkdir -p /data/kubernetes/redis-cluster/logs
touch /data/kubernetes/redis-cluster/logs/redis-node-log.txt
cd /data/kubernetes/redis-cluster/
chmod -R 777 *
DD
bash release-redis-cluster.sh

 

2.所有节点,都需要写入redis配置, 注意每个节点的配置文件都必需要修改参数: "cluster-announce-ip 当前节点IP地址"

cat >/data/kubernetes/redis-cluster/conf/redis.conf<<DD
# 端口号
port 7000

# 绑定IP地址
bind 0.0.0.0

# 关闭守护进程
daemonize no
protected-mode no

# 最大内存设置
maxmemory 5g
maxmemory-policy allkeys-lru

# PID文件路径
pidfile /var/redis/run/redis.pid

# 日志级别
loglevel warning

# 日志文件路径
logfile /logs/redis-node-log.txt

# 启用集群模式
cluster-enabled yes
cluster-config-file /etc/redis/redis-cluster.conf
cluster-node-timeout 5000
# 每个节点的IP地址
cluster-announce-ip 192.168.10.38
cluster-announce-port 7000
cluster-announce-bus-port 17000

# Redis DB备份配置
dir /dump
dbfilename nodes-dump.rdb

# AOF备份配置
appendonly yes
appendfilename appendonly.aof
appendfsync no
aof-load-truncated yes
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# 配置密码
requirepass bai_redis_7000 
masterauth bai_redis_7000
DD

 

3. 创建DaemonSet资源yaml文件, 确保每台节点上都会运行一个副本

kubectl label nodes $(hostname) redis-cluster=true  #修改 --overwrite, 查看 -l 标签=值
mkdir -p /data/software/redis-clustr_k8s/

cat >/data/software/redis-clustr_k8s/redis-cluster_DeamonSet.yaml<<DD
apiVersion: v1
kind: Namespace  #定义一个命名空间
metadata:
  name: redis-cluster
---
apiVersion: apps/v1
kind: DaemonSet  #定义资源,每个节点部署一个redis
metadata:
  name: redis-cluster
  namespace: redis-cluster
spec:
  selector:    #pod标签选择器
    matchLabels:
      project: new-bigdata
  template:
    metadata:
      labels:  #pod标签,尽量二元以上
        app: redis
        project: new-bigdata
    spec: 
      nodeSelector:  #节点标签选择器
        redis-cluster: "true"
      containers:
      - name: redis-cluster-bigdata
        image: redis:latest
        imagePullPolicy: IfNotPresent  #优先使用本地镜像
        command: ["redis-server","/etc/redis/redis.conf"]
        ports:
        - name: redis-port
          containerPort: 7000  #pod容器暴露端口
          hostPort: 7000  #暴露的节点端口
          protocol: TCP
        - name: cluster-port
          containerPort: 17000
          hostPort: 17000
          protocol: TCP
        volumeMounts:
        - name: conf  #redis配置文件目录
          mountPath: /etc/redis/  
        - name: logs  #redis日志目录
          mountPath: /logs/
        - name: dump  #redis数据持久化备份
          mountPath: /dump/
        - name: timezone  #加载本地时区
          mountPath: /etc/localtime
      tolerations:  #允许此pod调度到控制平面master节点
      - key: node-role.kubernetes.io/control-plane
        operator: Exists
        effect: NoSchedule
      volumes:  #挂载本地目录,name与pod模板中的挂载目录必须同名对应
      - name: conf
        hostPath:
          path: /data/kubernetes/redis-cluster/conf/
      - name: logs
        hostPath:
          path: /data/kubernetes/redis-cluster/logs/
      - name: dump
        hostPath:
          path: /data/kubernetes/redis-cluster/dump/
      - name: timezone
        hostPath:
          path: /etc/localtime
DD

4.创建kubernetes资源

kubectl apply -f /data/software/redis-clustr_k8s/redis-cluster_DeamonSet.yaml

 

5. 登陆进任意pod容器执行创建redis-cluster命令

redis-cli --cluster create \
    192.168.10.34:7000 \
    192.168.10.35:7000 \
    192.168.10.37:7000 \
    192.168.10.38:7000 \
    192.168.10.39:7000 \
    192.168.10.40:7000 \
    --cluster-replicas 1 \
    -a bai_redis_7000


    
6. 登陆redis-cluster, 测试集群:
redis-cli -c -h 192.168.10.35 -p 7000 -a bai_redis_7000
>cluster nodes
>cluster info
>set name king
>get name