KingbaseES V8R3集群运维案例之---流复制异步同步及全同步模式配置

发布时间 2023-09-18 16:19:06作者: KINGBASE研究院

案例说明:
通过案例描述KingbaseES V8R3集群异步、同步及全同步强一致性配置,本案例为一主二备的架构。

适用版本:
KingbaseES V8R3

集群架构:

集群复制配置参数说明:
1)sync_flag

[kingbase@node101 bin]$ cat ../etc/HAmodule.conf |grep -i sync_
#1->synchronous cluster,0->asynchronous cluster ,default 1.example:SYNC_FLAG=1
SYNC_FLAG=0

说明:此参数强制配置流复制同异步模式,0为异步模式,1为同步模式。

2)synchronous_standby_names

[kingbase@node101 bin]$ cat ../data/kingbase.conf |grep -i standby_name
##synchronous_standby_names = ''        # standby servers that provide sync rep
#synchronous_standby_names='1 (node1, node2)'

说明:此参数用于配置在流复制同步模式下,参与同步复制的备库节点;
例如一主二备集群:'1 (node1,node2,node3)',其中node1是主库,则node2为同步备库,node3为潜在同步备库;
'2 (node1,node2,node3)',其中node1是主库,则node2为同步备库,node3为同步备库;
synchronous_standby_names = '' ,则node2,node3均为异步备库。

3)synchronous_commit

[kingbase@node101 bin]$ cat ../data/kingbase.conf |grep _commit
#synchronous_commit = on                # synchronization level;

说明:用于同步模式下接收到wal日志流的处理方式:
remote_write:备库将wal日志写入到文件系统缓存后,给主库返回commit的ack信息。
on:          备库将wal日志写入到wal日志文件后,给主库返回commit的ack信息。
remote_apply:备库将wal日志写入到wal日志文件并apply后,给主库返回commit的ack信息,
              强制主备事务同步。

如下图所示:

一、流复制异步模式配置

1、sync_flag配置

[kingbase@node101 bin]$ cat ../etc/HAmodule.conf |grep -i flag
#1->synchronous cluster,0->asynchronous cluster ,default 1.example:SYNC_FLAG=1
SYNC_FLAG=0

2、集群启动后流复制状态查看

TEST=# select * from sys_stat_replication;
  PID  | USESYSID | USENAME | APPLICATION_NAME |  CLIENT_ADDR  | CLIENT_HOSTNAME | CLIENT_PORT |         BACKEND_STAR
T         | BACKEND_XMIN |   STATE   | SENT_LOCATION | WRITE_LOCATION | FLUSH_LOCATION | REPLAY_LOCATION | SYNC_PRIOR
ITY | SYNC_STATE
-------+----------+---------+------------------+---------------+-----------------+-------------+---------------------
----------+--------------+-----------+---------------+----------------+----------------+-----------------+-----------
----+------------
 31144 |       10 | SYSTEM  | node2            | 192.168.1.102 |                 |       16989 | 2023-05-07 00:55:00.
110856+08 |              | streaming | 1/500000D0    | 1/500000D0     | 1/500000D0     | 1/500000D0      |
  0 | async
 31275 |       10 | SYSTEM  | node3            | 192.168.1.103 |                 |       30046 | 2023-05-07 00:55:03.
014221+08 |              | streaming | 1/500000D0    | 1/500000D0     | 1/500000D0     | 1/500000D0      |
  0 | async
(2 rows)

# SYNCHRONOUS_STANDBY_NAMES默认配置 
[kingbase@node101 data]$ cat kingbase.auto.conf
# Do not edit this file manually!
# It will be overwritten by ALTER SYSTEM command.
SYNCHRONOUS_STANDBY_NAMES = ''

TEST=# show synchronous_standby_names ;
 synchronous_standby_names
---------------------------

(1 row)

如下图所示,两个备库节点流复制均为async模式:

二、流复制同步模式配置
1、sync_flag配置

[kingbase@node101 bin]$ cat ../etc/HAmodule.conf |grep -i flag
#1->synchronous cluster,0->asynchronous cluster ,default 1.example:SYNC_FLAG=1
SYNC_FLAG=1

2、集群启动后流复制状态查看

TEST=# select * from sys_stat_replication;
 PID  | USESYSID | USENAME | APPLICATION_NAME |  CLIENT_ADDR  | CLIENT_HOSTNAME | CLIENT_PORT |         BACKEND_START
         | BACKEND_XMIN |   STATE   | SENT_LOCATION | WRITE_LOCATION | FLUSH_LOCATION | REPLAY_LOCATION | SYNC_PRIORI
TY | SYNC_STATE
------+----------+---------+------------------+---------------+-----------------+-------------+----------------------
---------+--------------+-----------+---------------+----------------+----------------+-----------------+------------
---+------------
 3966 |       10 | SYSTEM  | node2            | 192.168.1.102 |                 |       18965 | 2023-05-07 01:01:12.3
76208+08 |              | streaming | 1/510000D0    | 1/510000D0     | 1/510000D0     | 1/510000D0      |
 2 | sync
 3970 |       10 | SYSTEM  | node3            | 192.168.1.103 |                 |       30106 | 2023-05-07 01:01:14.9
62029+08 |              | streaming | 1/510000D0    | 1/510000D0     | 1/510000D0     | 1/510000D0      |
 3 | potential
(2 rows)

说明:node2同步模式sync,node3同步模式potential(潜在同步,实为异步)

# SYNCHRONOUS_STANDBY_NAMES默认配置 
[kingbase@node101 bin]$ cat ../data/kingbase.auto.conf
# Do not edit this file manually!
# It will be overwritten by ALTER SYSTEM command.
SYNCHRONOUS_STANDBY_NAMES = '1 (node1, node2, node3)'

TEST=# show synchronous_standby_names ;
 synchronous_standby_names
---------------------------
 1 (node1, node2, node3)
(1 row)

如下图,默认流复制同步模式配置,其中一个节点为sync,其余为potential:

三、集群流复制全同步模式配置
1、sync_flag配置

[kingbase@node101 bin]$ cat ../etc/HAmodule.conf |grep -i flag
#1->synchronous cluster,0->asynchronous cluster ,default 1.example:SYNC_FLAG=1
SYNC_FLAG=1

2、配置 synchronous_standby_names

TEST=# alter system set synchronous_standby_names='2 (node1, node2, node3)';
ALTER SYSTEM

TEST=# select sys_reload_conf();
 SYS_RELOAD_CONF
-----------------
 t
(1 row)

TEST=# show synchronous_standby_names ;
 synchronous_standby_names
---------------------------
 2 (node1, node2, node3)
(1 row)

[kingbase@node101 bin]$ cat ../data/kingbase.auto.conf
# Do not edit this file manually!
# It will be overwritten by ALTER SYSTEM command.
SYNCHRONOUS_STANDBY_NAMES = '2 (node1, node2, node3)'

3、查看流复制同步模式

TEST=# select * from sys_stat_replication;
 PID  | USESYSID | USENAME | APPLICATION_NAME |  CLIENT_ADDR  | CLIENT_HOSTNAME | CLIENT_PORT |         BACKEND_START
         | BACKEND_XMIN |   STATE   | SENT_LOCATION | WRITE_LOCATION | FLUSH_LOCATION | REPLAY_LOCATION | SYNC_PRIORI
TY | SYNC_STATE
------+----------+---------+------------------+---------------+-----------------+-------------+----------------------
---------+--------------+-----------+---------------+----------------+----------------+-----------------+------------
---+------------
 3966 |       10 | SYSTEM  | node2            | 192.168.1.102 |                 |       18965 | 2023-05-07 01:01:12.3
76208+08 |              | streaming | 1/510000D0    | 1/510000D0     | 1/510000D0     | 1/510000D0      |
 2 | sync
 3970 |       10 | SYSTEM  | node3            | 192.168.1.103 |                 |       30106 | 2023-05-07 01:01:14.9
62029+08 |              | streaming | 1/510000D0    | 1/510000D0     | 1/510000D0     | 1/510000D0      |
 3 | sync
(2 rows)

如下图所示,两个备库节点node2和node3流复制均为sync模式:

4、读写分离配置备库强一致性

[kingbase@node101 log]$ cat ../db/data/kingbase.conf|grep _commit
synchronous_commit = remote_apply                # synchronization level;

说明:配置synchronous_commit = remote_apply,主库事务commit,需要等待所有备库
接收到wal日志流并写入到wal日志文件,执行apply完成事务的操作后,向主库返回ack,
主库完成事务的commit。
因此,此时主库及备库查询到的事务的结果是一致的,但是对事务繁忙的业务, 性能
会有很大的影响。

四、总结
KingbaseES V8R3和KingbaseES V8R6集群在流复制同异步模式配置不同,通过以上案例的描述,可以了解到在KingbaseES V8R3集群下流复制模式的详细配置过程。