keepalived双主模型的实现

一、简介

 keepalived 简介:是服务器高可用的一个重要软件，它的核心组件有vrrp ，stack， checker ，ipvs， warpper， watch dog 

它是vrrp协议的实现，原生设计目的为高可用ipvs服务；keepalived能够通过配置文件中定义生成ipvs规则

并能够对RS的健康状态进行检测；vrrp_script,vrrp_track; 

双主模型的实现

简介:双方模型(主/备，备/主)的这里的意思是，一个keepalived配置中，一个虚拟IP地址为主，另一个为备。而在另一个

keepalived的配置中，与其它主机则恰恰相反，一个虚拟IP地址为备，另一个为主

以下为此次双主模型实现的拓扑

二、HA Cluster配置的前提；

(1)要点:各节点之间的时间秘需要同步: 

# ntpdate 172.16.0.1 (注此:处可自己在网上找个时间同步服务器)

(2)确保iptables及selinux不会阻碍:

# iptables -F && setenforce 0

(3)各节点之间可通过主机名互相通信(对keepalived并非必须):

# vim /etc/hosts 

172.16.250.140 kpl1

172.16.250.158 kpl2 两台keepalived主机都要修改 

(4)各节点之间root用户可以基于密钥认证的ssh通信

# ssh-copy-id -i 172.16.250.140

# ssh-copy-id -i 172.16.250.158

三、dr集群配置

此处使用dr集群类型

首先将dr模型配置好 

编辑setkp.sh 

#!/bin/bash 

vip=172.16.26.126

vip2=172.16.26.127

mask=255.255.255.255

interface=’lo:0′

interface2=’lo:1′

eth=’eno16777736:0′ 

case $1 in 

start) 

echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore

echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore

echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce

ifconfig $interface $vip netmask $mask broadcast $vip up 

ifconfig $interface2 $vip2 netmask $mask broadcast $vip2 up 

route add -host $vip dev $interface 

route add -host $vip2 dev $interface2

;; 

dstart) 

ifconfig $eth $vip/32 netmask $mask broadcast $vip up

;;

dstop)

ifconfig $eth down

;;

stop)

ifconfig $interface down 

ifconfig $interface2 down

echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore

echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore

echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce

echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce

;; 

status) 

ifconfig 

cat /proc/sys/net/ipv4/conf/all/arp_ignore

cat /proc/sys/net/ipv4/conf/lo/arp_ignore

cat /proc/sys/net/ipv4/conf/all/arp_announce

cat /proc/sys/net/ipv4/conf/lo/arp_announce

;; 

*)

echo “Usage: $(basename $0) {dstart|dstop|start|stop}”

exit 1 

esac

RS1、RS2主机中都执行执行此脚本 

# sh setkp.sh start 

# sh setkp.sh status 可用来查看当前网络的配置状态 

RS1、RS2主机中安装httpd并启动 

# yum -y install httpd && systemctl start httpd  

RS1 # echo “<h1>RS1</h1>” > /usr/share/nginx/html/index.html 

RS2 # echo “<h1>RS2</h1>” > /usr/share/nginx/html/index.html

四、keepalived配置

keepalived,keepalived2 中安装nginx(httpd和nginx都可以,此处用来做为keepalived的本身的提供sorry server)

当dr集群的两台节点都停止时，会由keepalived本身来提供一个页面

# yum -y install nginx keepalived ipvsadm && systemctl start nginx 

# echo “<h1>sorry server keepalived 1 </h1>” > /usr/share/nginx/html/index.html 

# echo “<h1>sorry server keepalived 2 </h2>” > /usr/share/nginx/html/index.html 

在一台keepalived主机上进行测试

# curl 172.16.251.232

<h1>RS1</h2>

# curl 172.16.250.159

<h1>RS2</h1>

keepalived,keepalived2 中在 /etc/keepalived/目录下编辑一个脚本来用来在主备变化，或服务down掉时发邮件给系统用户 

# vim kmail.sh 

#!/bin/bash

contact=’root@localhost’

notify(){

mailsubject=”$(hostname) to be $1:vip floating”

mailbody=”$(date +’%F %T’):vrrp transition, $(hostname) change to be $1″

echo $mailbody | mail -s “$mailsubject” $contact

}

case $1 in 

master )

notify master

;;

backup )

notify backup 

;;

fault )

notify fault 

;;

*)

echo “Usage:$(basename $0) {master |backup|fault}”

;;

esac

#chmod +x kmail.sh 

注:keepalived的两台主机的配置基本相同，当配置模型为主/备模型的时候，主备之间需要修改三个指令 

分别为 router_id kpl1 在keepalived2上时需要修改为 router_id kpl2 

  state MASTER 在keepalived2上时需要修改为 state BACKUP 

  priority 100 在keepalived2上时需要修改为  priority 90 (此处的值比主服务器的小便可以)

以下主/备的配置示例

先把备的服务器启动

# tcpdump -i eno33554984 -nn host 224.0.61.61

可使用tcpdump 抓取组播地址 来查看其通过组播方式传递的心跳信息

以下的配置文件便是keepavlied双主模型的实现 

对防火墙打标记 

keepalived 1

# iptables -t mangle -A PREROUTING -d 172.16.26.126 -p tcp –dport 80 -j MARK –set-mark 3 

# iptables -t mangle -A PREROUTING -d 172.16.26.127 -p tcp –dport 80 -j MARK –set-mark 3

# vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

  notification_email {

root@localhost

}

  notification_email_from [email protected]

  smtp_server 127.0.0.1

  smtp_connect_timeout 30

  router_id kpl1

  vrrp_mcast_group4 224.0.61.61

}

vrrp_instance VI_1 {

state MASTER

interface eno33554984

virtual_router_id 55

priority 100

advert_int 1

notify_master “/etc/keepalived/kmail.sh master”

notify_backup “/etc/keepalived/kmail.sh backup”

notify_fault  “/etc/keepalived/kmail.sh fault”

authentication {

auth_type PASS

auth_pass zE2kNsRQ

}

virtual_ipaddress {

172.16.26.126 dev eno33554984 label eno33554984:0    

}

}

vrrp_instance VI_2 {

state BACKUP

interface eno33554984

virtual_router_id 66

priority 90

advert_int 1

notify_master “/etc/keepalived/kmail.sh master”

notify_backup “/etc/keepalived/kmail.sh backup”

notify_fault  “/etc/keepalived/kmail.sh fault”

authentication {

auth_type PASS

auth_pass zE2kfsRQ

}

virtual_ipaddress {

172.16.26.127 dev eno33554984 label eno33554984:1    

}

}

virtual_server fwmark 3 {

delay_loop 2

lb_algo wrr 

lb_kind DR

nat_mask 255.255.0.0

protocol TCP

sorry_server 127.0.0.1 80

real_server 172.16.251.232 80 {

weight 3

HTTP_GET {

url { 

 path /

 status_code 200  

 }

connect_timeout 2

nb_get_retry 3

delay_before_retry 3

}

}

real_server 172.16.250.159 80 {

weight 1

HTTP_GET {

url { 

 path /

 status_code 200  

 }

connect_timeout 2

nb_get_retry 3

delay_before_retry 3

}

}

keepalived 2 

# iptables -t mangle -A PREROUTING -d 172.16.26.126 -p tcp –dport 80 -j MARK –set-mark 3

# iptables -t mangle -A PREROUTING -d 172.16.26.127 -p tcp –dport 80 -j MARK –set-mark 3

# vim /etc/keepavlied/keepalived.conf 

! Configuration File for keepalived

global_defs {

  notification_email {

root@localhost

}

  notification_email_from [email protected]

  smtp_server 127.0.0.1

  smtp_connect_timeout 30

  router_id kpl2

  vrrp_mcast_group4 224.0.61.61

}

vrrp_instance VI_1 {

state BACKUP

interface eno33554984

virtual_router_id 55

priority 90

advert_int 1

notify_master “/etc/keepalived/kmail.sh master”

notify_backup “/etc/keepalived/kmail.sh backup”

notify_fault  “/etc/keepalived/kmail.sh fault”

authentication {

auth_type PASS

auth_pass zE2kNsRQ

}

virtual_ipaddress {

172.16.26.126 dev eno33554984 label eno33554984:0    

}

}

vrrp_instance VI_2 {

state MASTER

interface eno33554984

virtual_router_id 66

priority 100

advert_int 1

notify_master “/etc/keepalived/kmail.sh master”

notify_backup “/etc/keepalived/kmail.sh backup”

notify_fault  “/etc/keepalived/kmail.sh fault”

authentication {

auth_type PASS

auth_pass zE2kfsRQ

}

virtual_ipaddress {

172.16.26.127 dev eno33554984 label eno33554984:1    

}

}

virtual_server fwmark 3  {

delay_loop 2

lb_algo wrr

lb_kind DR

nat_mask 255.255.0.0

protocol TCP

sorry_server 127.0.0.1 80

real_server 172.16.251.232 80 {

weight 3

HTTP_GET {

url {

 path /

 status_code 200

 }

connect_timeout 2

nb_get_retry 3

delay_before_retry 3

}

}

real_server 172.16.250.159 80 {

weight 1

HTTP_GET {

url {

 path /

 status_code 200

 }

connect_timeout 2

nb_get_retry 3

delay_before_retry 3

}

}

五、测试

此时可以测试其访问

当一个keepavlied停止时 

当RS2停止时 

当RS1，RS2都停止时

   

六、keepavlied配置指令说明 

虚拟路由器段

state MASTER：当前节点在虚拟路由器中的初始状态；

interface ETHERCARD: vrrp实际工作的网卡接口

virtual_route_id 51 :虚拟路由器ID，范围0-255；

priority 100 :当前物理节点在此虚拟路由器中的优先级；

advert_int 1:每隔多久发送心跳(通行的时间间隔)

auth_type PASS ：选择认证机制 

auth_pass 1111 ：密码 八位有效

virtual_ipaddress ：定义虚拟IP 

track_interface ： 定义要监控的接口

notify_master <STRING> | <QUOTED-STRING> ：当前节点变为主节点时用STRING脚本通告

notfy_backup<STRING> | <QUOTED-STRING> : 当前节点变为主节时用 STRING脚本通告

notify_fault<STRING> | <QUOTED-STRING> : 当前节点上不了线时用STRING脚本通告

notify<STRING> | <QUOTED-STRING> : 如果三种状态用一个脚本来实现用STRING脚本通告 

虚拟服务段

lb_algo rr | wrr|lc|lblc|sh|dh :定义负载均衡调度算法

delay_loop<INT>：：定义服务轮询时间间隔

bl_kind NAT |DR |TUN ：集群的类型

persistence_time_out<INT> ：持久连接时长

protocol TCP : 服务协议

sorry_server<IPADDR><PORT>：所有RS均故障时，提供sorry server的服务器；

real_server<IPADDR><PORT>:

weight<INT>：权重

notify_up<STRING>|<QUOTED-STRING> ： 节点上线通知脚本

notify_down <STRING>|<QUOTED-STRNG>：节点离线通知脚本；

#HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK ：支持的所以健康状态的检测方式

url:健康状态检测时请求的资源的URL 

delay_before_retry<INT> ：两次尝试之间的时间间隔 

connect_timeoute<STRING>：连接的超时时长

connect_ip<IP ADDRESS>：向此处指定的地址发测试请求

connect_port<PORT>：向此处指定的PORT发测试请求

bindto<IP ADDRESS>：指定测试请求报文的源IP 

bind_port<PORT>： 指定测试请求报文的源PORT