Keepalived高可用集群

高可用集群简介

什么是高可用集群？

高可用集群 (High Availability;Cluster，简称HA Cluster) ，是指以减少服务中断时间为目的的服务器集群技术。它通过保护用户的业务程序对外不间断提供的服务，把因软件、硬件、人为造成的故障对业务的影响降低到最小程度。

自动切换/故障转移（FailOver）

自动切换阶段某一主机如果确认对方故障，则正常主机除继续进行原来的任务还将依据各种容错备援模式接管预先设定的备援作业程序，并进行后续的程序及服务。
通俗地说，即当A无法为客户服务时，系统能够自动地切换，使B能够及时地顶上继续为客户提供服务，且客户感觉不到这个为他提供服务的对象已经更换
通过上面判断节点故障后，将高可用集群资源（如VIP、httpd等）从该不具备法定票数的集群节点转移到故障转移域( Failover Domain，可以接收故障资源转移的节点）。

自动侦测/脑裂

自动侦测阶段由主机上的软件通过冗余侦测线，经由复杂的监听程序，逻辑判断，来相互侦测对方运行的情况。
常用的方法是:集群各节点间通过心跳信息判断节点是否出现故障。
脑裂：在高可用（HA）系统中，当联系2个节点的“心跳线"断开时，本来为一整体、动作协调的HA系统，就分裂成为2个独立的个体。由于相互失去了联系，都以为是对方出了故障。两个节点上的HA软件像“裂脑人"一样，争抢“"共享资源"、争起“应用服务"，就会发生严重后果——或者共享资源被瓜分、2边"服务"都起不来了"或者2边"服务"都起来了，但同时读写“共享存储"，导致数据损坏（常见如数据库轮询着的联机日志出错）。
脑裂解决方案：1.添加冗余的心跳线 2.启用磁盘锁 3. 设置仲裁机制 4. 脑裂的监控报警

其他高可用方案：heartbeat、pacemaker、piranha（web页面）

Keepalived

keepalived是什么？
keepalived是集群管理中保证集群高可用的一个服务软件，用来防止单点故障.

keepalived工作原理

keepalived是以VRRP协议为实现基础的，VRRP全称Virtual Router Redundancy Protocol，即虚拟路由冗余协议。

将N台提供相同功能的服务器组成一个服务器组，这个组里面有一个master和个backup，master上面有一个对外提供服务的vip(该服务器所在局域网内其他机器的默认路由为该vip) ,master会发组播，当backup收不到vrrp包时就认为master宕掉了，这时就需要根据VRRP的优先级来选举一个backup当master

keepalived主要有三个模块

分别是core. check和vrrp。
core模块为keepalived的核心，负责主进程的启动、维护以及全局配置文件的加载和解析。check负责健康检查,包括常见的各种检查方式。vrrp模块是来实现VRRP协议的。

实战案例1 keepalived + nginx

准备：server1 server2 关闭防火墙 selinux

server1:

yum install -y keepalived
cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf-backup  //备份原文件

vim /etc/keepalived/keepalived.conf  //把内容全删了 ggdG 然后配置如下
! Configuration File for keepalived
global_defs {
router_id 1
}

#vrrp_script chk_nginx {
#script "/etc/keepalived/ck_ng.sh"
#interval 2
#weight -5
#fall 3
#}

vrrp_instance VI_1 {
state MASTER
interface ens33
mcast_src_ip 192.168.70.130
virtual_router_id 55
priority 100
advert_int 1

authentication {
auth_type PASS
auth_pass 123456
}

virtual_ipaddress {
192.168.70.140
}

#track_script {
#chk_nginx
#}

}

yum install -y nginx
systemctl enable nginx
systemctl start nginx
vim /var/share/nginx/html/index.html  //自行修改页面 以便区分server2的nginx
curl -i 192.168.70.130
systemctl start keepalived
systemctl enable keepalived

server2：

yum install -y keepalived
cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf-backup  //备份原文件

vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id 2
}

#vrrp_script chk_nginx {
#script "/etc/keepalived/ck_ng.sh"
#interval 2
#weight -5
#fall 3
#}

vrrp_instance VI_1 {
state BACKUP
interface ens33
mcast_src_ip 192.168.70.132
virtual_router_id 55
priority 99
advert_int 1

authentication {
auth_type PASS
auth_pass 123456
}

virtual_ipaddress {
192.168.70.140
}

#track_script {
#chk_nginx
#}

}

yum install -y nginx
systemctl enable nginx
systemctl start nginx
curl -i localhost
systemctl start keepalived
systemctl enable keepalived

测试：

[root@localhost local]# curl -i 192.168.70.140  //返回的应该是server1 nginx的内容

可以试着把server1断网，vmware设置 取消网络连接 再测试访问 这时候返回的应该是server2 nginx的内容

关于keepalived对nginx状态未知的问题

恢复之前的实验。启动两台主机的keepalived和nginx。确保页面访问正常。关闭master的nginx服务。systemctl stop nginx继续访问VIP，请问页面是否会切换到backup呢?keepalived并不会关心nginx的状态，原因是keepalived监控的是接口ip状态。无法监控nginx服务状态。解决方案：

1、监控脚本

server1 server2 添加nginx监控脚本

vim /etc/keepalived/ck_ng.sh
#!/bin/bash
#检查nginx进程是否存在
counter=`ps -C nginx --no-heading | wc -l`
if [ ${counter} = 0 ] ;then
        systemctl restart nginx
        sleep 5
        counter2=`ps -C nginx --no-heading | wc -l`
        if [ ${counter2} = 0 ] ;then
                systemctl stop keepalived
        fi
fi

chmod +x /etc/keepalived/ck_ng.sh

修改keepalived.conf文件 把上述写的注释都取消 server1 server2 都取消注释 其他内容不变

vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id 1
}

vrrp_script chk_nginx {
script "/etc/keepalived/ck_ng.sh"
interval 2
weight -5
fall 3
}

vrrp_instance VI_1 {
state BACKUP
interface ens33
mcast_src_ip 192.168.70.130
virtual_router_id 55
priority 100
advert_int 1

authentication {
auth_type PASS
auth_pass 123456
}

virtual_ipaddress {
192.168.70.140
}

track_script {
chk_nginx
}

}

systemctl restart keepalived

测试：

systemctl stop nginx
systemctl status nginx

如果测试结果并没按预期执行

在vrrp_script chk_nginx{} 中间加debug
tail -f /var/log/messages  //查看日志
如果出现Aug 27 20:59:44 localhost Keepalived_vrrp[51703]: /etc/keepalived/ck_ng.sh exited due to signal 15
说明生命探测advert_int设置时间太短了 增加5秒试试 相应interval必须大于advert_int的时间设置6秒试试，两台server都必须改过来！

实战案例2 keepalived + lvs集群

1.在master上安装配置keepalived ipvsadm

yum install keepalived ipvsadm -y

2.在master上修改配置文件

vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived
global_defs {
router_id Director 1
}

#Keepalived
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 100
advert_int 1

authentication {
auth_type PASS
auth_pass 123456
}

virtual_ipaddress {
192.168.70.140/24 dev ens33
}
}

#LVS
virtual_server 192.168.70.140 80 {
delay_loop 3  # 将 Keepalived 故障转移时的延迟检测循环次数设置为 5 次
lb_algo rr
lb_kind DR
protocol TCP

real_server 192.168.70.133 80 {
weight 1
TCP_CHECK {
connect_timeout 5
}
}

real_server 192.168.70.134 80 {
weight 1
TCP_CHECK{
connect_timeout 3
}
}

}

3.在backup安装配置keepalived ipvsadm

yum install keepalived ipvsadm -y

4.在backup上修改配置文件

vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived
global_defs {
router_id Director 2
}

#Keepalived
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 99
advert_int 1

authentication {
auth_type PASS
auth_pass 123456
}

virtual_ipaddress {
192.168.70.140/24 dev ens33
}
}

#LVS
virtual_server 192.168.70.140 80 {
delay_loop 3  # 将 Keepalived 故障转移时的延迟检测循环次数设置为 5 次
lb_algo rr
lb_kind DR
protocol TCP

real_server 192.168.70.133 80 {
weight 1
TCP_CHECK {
connect_timeout 5
}
}

real_server 192.168.70.134 80 {
weight 1
TCP_CHECK{
connect_timeout 3
}
}

}

5.启动两台设备的keepalived

systemctl start keepalived
systemctl enable keepalived

6.两台realserver 安装并启动httpd

yum install -y httpd
systemtl start httpd
systemtl enable httpd

7.新建lo:0文件 回环接口

vim /etc/sysconfig/network-scripts/ifcfg-lo:0  //配置如下

DEVICE=lo:0
IPADDR=192.168.70.140
NETMASK=255.255.255.255
ONBOOT=yes

8.配置路由 让每次开机都配置上回环接口
不管谁访问140 都让回环接口来处理

vim /etc/rc.local  //添加如下

/sbin/route add -host 192.168.70.140 dev lo:0

9.配置 sysctl.conf文件

vim /etc/sysctl.conf

net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2

10.将lo:0文件拷贝到另一台realserver

scp /etc/sysconfig/network-scripts/ifcfg-lo:0 192.168.70.134:/etc/sysconfig/network-scripts/ifcfg-lo:0
scp /etc/sysctl.conf 192.168.70.134:/etc/sysctl.conf

10.另一台一样配置rc.local文件

vim /etc/rc.local  //添加

/sbin/route add -host 192.168.70.140 dev lo:0

11.一样配置sysctl.conf文件

vim /etc/sysctl.conf

net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2

12.测试

浏览器访问192.168.70.140 再关闭master的网络再试试 还能访问表示我们实验成功了

LVS+Keepalived 常见面试题

1.什么是集群?集群分为哪些类型?列举代表的产品。2.有些负载均衡集群服务?他们有什么区别?
3.LVS-DR和LVS-NAT的工作原理。
4.keepalived的工作原理。
5.高可用集群有哪些产品。他们的区别。
6.负载均衡集群的策略有哪些?能否举例说明?

本栏目推荐文章
• Spark版本不兼容导致Standalone集群无法连接问题
• Linux 部署redis集群（三主三从）
• Prometheus监控k8s集群节点主机内存/CPU使用率
• TiDB 多集群告警监控-中章-融合多集群 Grafana
• TiDB 在单机上模拟部署生产环境集群
• k8s集群手动更换证书过程
• Kubernetes 多集群网络方案系列 1 -- Submariner 介绍
• Kubernetes 多集群网络方案系列 2 -- Submariner 监控
• 一篇文章彻底搞懂TiDB集群各种容量计算方式
• K8s 多集群实践思考和探索

集群 Keepalived Keepalive LVS集群keepalived keepalive lvs 集群keepalived企业lvs 可用性 集群keepalived lvs 集群keepalived lvs keepalive lvs web keepalived lvs keepalived架构lvs keepalived实战lvs keepalived lvs 16 keepalived linux lvs