keeppalived

VRRP技术

虚拟路由冗余协议,解决静态网关单点风险

物理层:路由器、三层交换机
软件层:keepalived

术语：
虚拟路由器：Virtual Router 
虚拟路由器标识：VRID(0-255)，唯一标识虚拟路由器
VIP：Virtual IP 
VMAC：Virutal MAC (00-00-5e-00-01-VRID)
物理路由器：
	master：主设备
	backup：备用设备
	priority：优先级
技术：
通告：心跳，优先级等；周期性
工作方式：抢占式，非抢占式
安全认证：
	无认证
	简单字符认证：预共享密钥
	MD5 
工作模式：
	主/备：单虚拟路由器
	主/主：主/备（虚拟路由器1），备/主（虚拟路由器2）

keeppalived介绍

vrrp 协议的软件实现，原生设计目的为了高可用 ipvs服务

功能：

基于vrrp协议完成地址流动
为vip地址所在的节点生成ipvs规则(在配置文件中预先定义)
为ipvs集群的各RS做健康状态检测
基于脚本调用接口完成脚本中定义的功能，进而影响集群事务，以此支持nginx、haproxy等服务

核心组件：
vrrp stack：VIP消息通告
checkers：监测real server
system call：实现 vrrp 协议状态转换时调用脚本的功能
SMTP：邮件组件
IPVS wrapper：生成IPVS规则
Netlink Reflector：网络接口
WatchDog：监控进程
控制组件：提供keepalived.conf 的解析器，完成Keepalived配置
IO复用器：针对网络目的而优化的自己的线程抽象
内存管理组件：为某些通用的内存管理功能（例如分配，重新分配，发布等）提供访问权限

文件

软件包名：keepalived 
主程序文件：/usr/sbin/keepalived
主配置文件：/etc/keepalived/keepalived.conf
配置文件示例：/usr/share/doc/keepalived/
Unit File：/lib/systemd/system/keepalived.service
Unit File的环境配置文件：
/etc/sysconfig/keepalived CentOS
/etc/default/keepalived Ubuntu

更新配置问题：
systemctl restart keepalived #新配置可能无法生效
systemctl stop keepalived;systemctl start keepalived #无法停止进程，需要 kill 停止

keeppalived编译安装

环境：
各节点时间必须同步：ntp, chrony
关闭防火墙及SELinux
各节点之间可通过主机名互相通信：非必须
建议使用/etc/hosts文件实现：非必须
各节点之间的root用户可以基于密钥认证的ssh服务完成互相通信：非必须

编译安装：
yum -y install gcc make wget openssl-devel libnl3-devel net-snmp-devel
wget https://keepalived.org/software/keepalived-2.2.4.tar.gz
tar -xvf keepalived-2.2.4.tar.gz
cd keepalived-2.2.4/
./configure --prefix=/usr/local/src/keepalived --disable-fwmark
make && make install
/usr/local/src/keepalived/sbin/keepalived -v
cat /lib/systemd/system/keepalived.service
[Unit]
Description=LVS and VRRP High Availability Monitor
After=network-online.target syslog.target 
Wants=network-online.target 
[Service]
Type=forking
PIDFile=/apps/keepalived/run/keepalived.pid
KillMode=process
EnvironmentFile=-/apps/keepalived/etc/sysconfig/keepalived
ExecStart=/apps/keepalived/sbin/keepalived $KEEPALIVED_OPTIONS
ExecReload=/bin/kill -HUP $MAINPID
[Install]
WantedBy=multi-user.target

root@centos7 keepalived]# cat /usr/local/src/keepalived-2.2.4/keepalived/keepalived.service
[Unit]
Description=LVS and VRRP High Availability Monitor
After=network-online.target syslog.target 
Wants=network-online.target 
Documentation=man:keepalived(8)
Documentation=man:keepalived.conf(5)
Documentation=man:genhash(1)
Documentation=https://keepalived.org

[Service]
Type=forking
PIDFile=/run/keepalived.pid
KillMode=process
EnvironmentFile=-/apps/keepalived/etc/sysconfig/keepalived
ExecStart=/apps/keepalived/sbin/keepalived  $KEEPALIVED_OPTIONS
ExecReload=/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target


cp /usr/local/src/keepalived/etc/keepalived/keepalived.conf  /etc/keepalived/
vim /etc/keepalived/keepalived.conf
#   vrrp_strict
systemctl start keepalived.service

keeppalived配置

配置文件：/etc/keepalived/keepalived.conf

配置文件组成
GLOBAL CONFIGURATION
Global definitions：定义邮件配置，route_id，vrrp配置，多播地址等
VRRP CONFIGURATION
VRRP instance(s)：定义每个vrrp虚拟路由器
LVS CONFIGURATION
Virtual server group(s)
Virtual server(s)：LVS集群的VS和RS

# man keepalived.conf

全局配置

#/etc/keepalived/keepalived.conf 
global_defs {
 notification_email {
 root@localhost #keepalived 发生故障切换时邮件发送的目标邮箱，可以按行区分写多个
 }
 notification_email_from keepalived@localhost  #发邮件的地址
 smtp_server 127.0.0.1     #邮件服务器地址
 smtp_connect_timeout 30   #邮件服务器连接timeout
 router_id ka1.example.com #每个keepalived主机唯一标识，建议使用当前主机名，但多节点重名不影响
 vrrp_skip_check_adv_addr  #对所有通告报文都检查，会比较消耗性能，启用此配置后，如果收到的通告报文和上一个报文是同一个路由器，则跳过检查，默认值为全检查
 vrrp_strict #严格遵守VRRP协议,启用此项后以下状况将无法启动服务:1.无VIP地址 2.配置了单播邻居 3.在VRRP版本2中有IPv6地址，开启动此项并且没有配置vrrp_iptables时会自动开启iptables防火墙规则，默认导致VIP无法访问,建议不加此项配置
 vrrp_garp_interval 0 #gratuitous ARP messages 报文发送延迟，0表示不延迟
 vrrp_gna_interval 0  #unsolicited NA messages （不请自来）消息发送延迟
 vrrp_mcast_group4 224.0.0.18 #指定组播IP地址范围：224.0.0.0到239.255.255.255,默认
值：224.0.0.18 
 vrrp_iptables        #此项和vrrp_strict同时开启时，则不会添加防火墙规则,如果无配置vrrp_strict项,则无需启用此项配置
}

开启日志功能
[root@ka1 ~]#vim /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-D -S 6"
[root@ka1 ~]#vim /etc/rsyslog.conf
local6.*                                               /var/log/keepalived.log 

实现子配置
include /path/file

虚拟路由器

vrrp_instance <STRING> { #<String>为vrrp的实例名,一般为业务名称
 配置参数
 ......
 }
#配置参数：
state MASTER|BACKUP#当前节点在此虚拟路由器上的初始状态，状态为MASTER或者BACKUP
interface IFACE_NAME #绑定为当前虚拟路由器使用的物理接口，如：eth0,bond0,br0,可以和VIP不在一个网卡
virtual_router_id VRID #每个虚拟路由器惟一标识，范围：0-255，每个虚拟路由器此值必须唯一，否则服务无法启动，同属一个虚拟路由器的多个keepalived节点必须相同,务必要确认在同一网络中此值必须唯一
priority 100 #当前物理节点在此虚拟路由器的优先级，范围：1-254，值越大优先级越高,每个keepalived主机节点此值不同
advert_int 1 #vrrp通告的时间间隔，默认1s
authentication { #认证机制
 auth_type AH|PASS   #AH为IPSEC认证(不推荐),PASS为简单密码(建议使用)
 auth_pass <PASSWORD> #预共享密钥，仅前8位有效，同一个虚拟路由器的多个keepalived节点必须一样
}
virtual_ipaddress { #虚拟IP,生产环境可能指定上百个IP地址
 <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>
 192.168.200.100 #指定VIP，不指定网卡，默认为eth0,注意：不指定/prefix,默认为/32
 192.168.200.101/24 dev eth1   #指定VIP的网卡，建议和interface指令指定的岗卡不在一个网卡
 192.168.200.102/24 dev eth2 label eth2:1 #指定VIP的网卡label 
}
track_interface { #配置监控网络接口，一旦出现故障，则转为FAULT状态实现地址转移
 eth0
 eth1
 …
}

keepalived单主，双主架构

1.单主

1.修改主服务配置文件
[root@centos7 ~]# cat /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node1.example.com
   vrrp_skip_check_adv_addr
#   vrrp_strict   #单播模式需注释此行
   vrrp_iptables   #防火墙无效
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   #vrrp_mcast_group4 224.0.0.18  #注释此行，另定义单播地址
}

vrrp_instance VI_1 {
    state MASTER     #标识主机，只做标识，具体主从由优先级决定，当集群都为BACKUP,添加nopreempt为非抢占模式
    interface eth0
    virtual_router_id 51  #虚拟路由id，必须唯一
    priority 100   #当前主机优先级
    advert_int 1
    nopreempt  #非抢占模式
    preempt_delay 10  #抢占延迟
    authentication {
        auth_type PASS    #验证，主从一致
        auth_pass 123456
    }
    virtual_ipaddress {   
     10.0.0.19 dev eth0 label eth0:1   #VIP
    }
unicast_src_ip 10.0.0.17  #使用单播模式，本机IP
unicast_peer {   #单播模式集群主机
   10.0.0.27
    }
notify_master "/data/notify.sh master"   #通知脚本
notify_backup "/data/notify.sh backup"
notify_fault "/data/notify.sh fault"
}

修改从服务配置文件
[root@centos7 ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     root@localohost
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
  # vrrp_strict
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   #vrrp_mcast_group4 224.0.0.18
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 80
    advert_int 1
    preempt_delay 10 
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
       10.0.0.19 dev eth0 label eth0:1 

    }
unicast_src_ip 10.0.0.27
unicast_peer{
   10.0.0.17
   }
}

2.双主

双主模式：两个或以上VIP运行在keepalived主机上，提高利用率
10.0.0.17配置
[root@centos7 data]# cat /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node1.example.com
   #vrrp_skip_check_adv_addr
#  vrrp_strict
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.0.100
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
     10.0.0.19 dev eth0 label eth0:1
    }
}
vrrp_instance VI_2 {
    state BACKUP
    interface eth0
    virtual_router_id 88
    priority 80
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass 1234
    }
    virtual_ipaddress {
     10.0.0.20 dev eth0 label eth0:2
    }

10.0.0.18配置
[root@localhost ~]# cat /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     root@example.local
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL2
#   vrrp_skip_check_adv_addr
   #vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.0.100
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 60
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        10.0.0.19/24 dev eth0 label eth0:1
    }
}
vrrp_instance VI_2 {
    state MASTER
    interface eth0
    virtual_router_id 88
    priority 100
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass 1234
    }
    virtual_ipaddress {
        10.0.0.20/24 dev eth0 label eth0:2
    }
}

VRRP Script

keepalived利用 VRRP Script 技术，可以调用外部的辅助脚本进行资源监控，并根据监控的结果实现优先动态调整，从而实现其它应用的高可用性功能

分两步实现：
定义脚本：定义脚本独立与虚拟路由
vrrp_script：自定义资源监控脚本，vrrp实例根据脚本返回值，公共定义，可被多个实例调用，定义在vrrp实例之外的独立配置块，一般放在global_defs设置块之后。通常此脚本用于监控指定应用的状态。一旦发现应用的状态异常，则触发对MASTER节点的权重减至低于SLAVE节点，从而实现 VIP 切换到 SLAVE 节点
vrrp_script <SCRIPT_NAME> { #定义一个检测脚本，在global_defs 之外配置
     script <STRING>|<QUOTED-STRING> #shell命令或脚本路径
     interval <INTEGER> #间隔时间，单位为秒，默认1秒
     timeout <INTEGER> #超时时间
     weight <INTEGER:-254..254> #默认为0,如果设置此值为负数，当上面脚本返回值为非0时，会将此值与本节点权重相加可以降低本节点权重，即表示fall. 如果是正数，当脚本返回值为0，会将此值与本节点权重相加可以提高本节点权重，即表示 rise.通常使用负值
     fall <INTEGER>       #执行脚本连续几次都失败,则转换为失败，建议设为2以上
     rise <INTEGER>       #执行脚本连续几次都成功，把服务器从失败标记为成功
     user USERNAME [GROUPNAME] #执行监测脚本的用户或组      
     init_fail         #设置默认标记为失败状态，监测成功之后再转换为成功状态
}

调用 VRRP script
vrrp_instance VI_1 {
 …
 track_script {
 chk_down
 }
}

脚本实现VIP切换

[root@centos7 ~]# cat /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     root@localohost
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL2
   vrrp_skip_check_adv_addr
  # vrrp_strict
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.0.100
}
vrrp_script chk_httpd {
   script "/usr/bin/killall -0 httpd"   #可以使用 "命令"  或者 "[脚本路径]"  脚本需执行权限
   interval 1
   weight -30
   fail 3
   rise 2
   timeout 5
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
       10.0.0.19 dev eth0 label eth0:1 
    }
track_script {   #调用前面定义脚本
   chk_httpd 
  }
}


nginx反代 单主
[root@centos7 ~]# cat /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     root@localohost
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL2
   vrrp_skip_check_adv_addr
  # vrrp_strict
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.0.100
}
vrrp_script chk_nginx {
   script "/usr/bin/killall -0 nginx"
   interval 1
   weight -30
   fail 3
   rise 2
   timeout 5
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
       10.0.0.19 dev eth0 label eth0:1 
    }
track_script {
   chk_nginx
  }
}

    upstream web_srv {
      server 10.0.0.18:80 weight=1;
      server 10.0.0.28:80 weight=1;
    }

   
    server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
           # root   html;
            #index  index.html index.htm;
           proxy_pass http://web_srv;
        }

案例：

1.keepalived+lvs

[root@centos7 ~]# cat /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     root@localohost
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL2
   vrrp_skip_check_adv_addr
  # vrrp_strict
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.0.100
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100   #BACKUP服务器修改优先级
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
       10.0.0.19 dev eth0 label eth0:1 
    }
}
virtual_server 10.0.0.19 80 {
    delay_loop 3
    lb_algo rr
    lb_kind DR
    protocol TCP
    real_server 10.0.0.18 80 {
      weight 1
      HTTP_GET {
        url {
          path /
          status_code 200
        }
      connect_timeout 5
      nb_get_retry 3
      delay_before_retry 1
      }
    }
    real_server 10.0.0.28 80 {
      weight 1
      TCP_CHECK {
        connect_timeout 5
        nb_get_retry 3
        delay_before_retry 1
        connect_port 80
      }
    }
  }

2.keepalived+haproxy

virtual_server 10.0.0.100 80 {
  delay_loop 3
  lb_algo rr
  lb_kind DR
  protocol TCP
  real_server 10.0.0.47 80 {
    weight 1
    HTTP_GET {          
      url {
        path /
        status_code 200
      }
    connect_timeout 1
    nb_get_retry 3
    delay_before_retry 1
  }
}