一、opengauss 的背景和行业现状
2022 年，七大 openGauss 商业版发布，是基于 openGauss3.0 推出商业发行版
目前海量数据库 Vastbase 表现最佳，一直是 TOP 1
作者认为之所以海量数据库 Vastbase 目前无法被同行超越，和各家研发实力和技术背景有关
众所周知，opengauss 起源于 postgresql，在此基础上做了改良
海量数据库从 postgresql 就已经开始做商业版，一直做到 opengauss，经验最丰富，其他产品例如 MongoDB 是紧随其后 (在此不讨论其存在的意义)，目前产品百花齐放，更显锦上添花
Vastbase G100 采用业界证明稳定高效的日志流复制技术实现集群内主备库的数据和状态传输，
并通过一主多备的部署和配置架构实现了多点可读、数据不丢失、异地灾备等关键高可用场景。
二、openGauss5.0.0企业版一主一备一级联部署（步骤全）
1.准备软硬件安装环境
准备三台硬件环境：虚拟机4核心CPU，16GB内存，100G硬盘
操作系统环境：CentOS7.9
1.1检查CPU核心
cat /proc/cpuinfo | grep processor
1.2检查内存大小
free -h
1.3检查操作系统版本
cat /etc/centos-release
1.4以CentOS操作系统为例，主机信息如下表所示
主机名称 IP地址
momo488m 192.168.14.6
momo488s1 192.168.14.7
momo488s2 192.168.14.8
2.修改/etc/hosts
2.1设置所有虚拟机主机名
momo488m
# hostnamectl set-hostname momo488m
momo488s1
# hostnamectl set-hostname momo488s1
momo488s2
# hostnamectl set-hostname momo488s2
2.2所有虚拟机添加对应主机名和IP地址
echo "192.168.14.6 momo488m" >>/etc/hosts
echo "192.168.14.7 momo488s1" >>/etc/hosts
echo "192.168.14.8 momo488s2" >>/etc/hosts
echo -e "\n" >>/etc/hosts
[root@momo488m ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.14.6 momo488m
192.168.14.7 momo488s1
192.168.14.8 momo488s2
3.关闭防火墙和关闭SELINUX
3.1所有虚拟机停止firewall，禁止firewall开机启动
systemctl stop firewalld.service
systemctl disable firewalld.service
3.2所有虚拟机关闭selinux
sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config
setenforce 0
4.设置字符集参数，设置时区和时间
4.1所有虚拟机设置字符集
cat>> /etc/profile<<EOF
export LANG=en_US.UTF-8
EOF
4.2所有虚拟机设置时区和时间
timedatectl set-timezone Asia/Shanghai
ntpdate ntp1.aliyun.com
5.关闭swap交换内存
5.1临时关闭swap
swapoff -a
5.2修改分区表文件，删除swap mount信息
cp /etc/fstab /etc/fstab.bak
sed -i '/swap/s/^/#/' /etc/fstab
cat /etc/fstab|grep -v ^#|grep -v '^$'
6.设置root用户远程登录
6.1配置SSH服务(允许root登录，关闭Banner)
sed -i '/Banner/s/^/#/' /etc/ssh/sshd_config
sed -i '/PermitRootLogin/s/^/#/' /etc/ssh/sshd_config
echo -e "\n" >> /etc/ssh/sshd_config
echo "Banner none " >> /etc/ssh/sshd_config
echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
6.2重启服务使设置生效
systemctl restart sshd.service
7.配置操作系统参数
7.1所有虚拟机修改内核参数，并使其生效，注意可能会输出很多文件或目录不存在，可以忽略，重启后正常
cat >> /etc/sysctl.conf << EOF
net.ipv4.tcp_max_tw_buckets = 10000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_keepalive_time = 30
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_retries1 = 5
net.ipv4.tcp_syn_retries = 5
net.ipv4.tcp_synack_retries = 5
net.sctp.path_max_retrans = 10
net.sctp.max_init_retransmits = 10
net.sctp.association_max_retrans = 10
net.sctp.hb_interval = 30000
net.ipv4.tcp_retries2 = 12
vm.overcommit_memory = 0
net.sctp.sndbuf_policy = 0
net.sctp.rcvbuf_policy = 0
net.sctp.sctp_mem = 94500000 915000000 927000000
net.sctp.sctp_rmem = 8192 250000 16777216
net.sctp.sctp_wmem = 8192 250000 16777216
net.ipv4.tcp_rmem = 8192 250000 16777216
net.ipv4.tcp_wmem = 8192 250000 16777216
net.core.wmem_max = 21299200
net.core.rmem_max = 21299200
net.core.wmem_default = 21299200
net.core.rmem_default = 21299200
net.ipv4.ip_local_port_range = 26000 65535
kernel.sem = 250 6400000 1000 25600
vm.min_free_kbytes = 419430 ##suggest to set as physical memory * 5%
net.core.somaxconn = 65535
net.ipv4.tcp_syncookies = 1
net.sctp.addip_enable = 0
net.core.netdev_max_backlog = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.tcp_fin_timeout = 60
kernel.shmall = 1152921504606846720
kernel.shmmax = 18446744073709551615
net.ipv4.tcp_sack = 1
net.ipv4.tcp_timestamps = 1
vm.extfrag_threshold = 500
vm.overcommit_ratio = 90
net.ipv4.ip_local_reserved_ports = 20050-20057,26000-26007
net.sctp.sctp_mem = 94500000 915000000 927000000
net.sctp.sctp_rmem = 8192 250000 16777216
net.sctp.sctp_wmem = 8192 250000 16777216
EOF
7.2执行sysctl -p
sysctl -p
8.修改操作系统资源限制，关闭透明页
8.1所有虚拟机配置文件系统参数、文件句柄设置、系统支持的最大进程数设置、网卡参数配置
echo "* soft stack 3072" >> /etc/security/limits.conf
echo "* hard stack 3072" >> /etc/security/limits.conf
echo "* soft nofile 1000000" >> /etc/security/limits.conf
echo "* hard nofile 1000000" >> /etc/security/limits.conf
echo "* soft nproc unlimited" >> /etc/security/limits.d/90-nproc.conf
tail -n 4 /etc/security/limits.conf
tail -n 1 /etc/security/limits.d/90-nproc.conf
8.2临时关闭透明页
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
8.3编译grub，内核层面关闭透明页
sed -i '/^GRUB_CMDLINE_LINUX/d' /etc/default/grub
echo "GRUB_CMDLINE_LINUX=\"rhgb quiet transparent_hugepage=never\"" >> /etc/default/grub
grub2-mkconfig -o /boot/grub2/grub.cfg
9.创建用户以及组（可选）和手工建立互信（可选）
9.1所有虚拟机创建omm用户以及dbgrp组
groupadd dbgrp
useradd -g dbgrp -m omm
echo MoMo488le | passwd --stdin omm
9.2生成密钥
mkdir ~/.ssh
chmod 700 ~/.ssh
ssh-keygen -t rsa
ssh-keygen -t dsa
9.3生成本机授权文件
cat ~/.ssh/id_rsa.pub >> ./.ssh/authorized_keys
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
9.4收集公钥互信文件分发
ssh momo488m cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh momo488m cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys momo488m:~/.ssh/authorized_keys
ssh momo488s1 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh momo488s1 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys momo488s1:~/.ssh/authorized_keys
ssh momo488s2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh momo488s2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys momo488s2:~/.ssh/authorized_keys
exec /usr/bin/ssh-agent $SHELL
/usr/bin/ssh-add
9.5测试互信
ssh momo488m date
ssh momo488s1 date
ssh momo488s2 date
10.安装相关软件包和python3.6.x
10.1所有虚拟机配置华为YUM源
mkdir /etc/yum.repos.d/bak
mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/bak/
wget -O /etc/yum.repos.d/CentOS-Base.repo \
https://repo.huaweicloud.com/repository/conf/CentOS-7-reg.repo
yum clean all
10.2所有虚拟机使用华为YUM源安装依赖的软件包，注意官网文档里要求的依赖不全，最好安装下面所有的软件包
yum install -y bzip2 python3 python3-devel lksctp* java-1.8.0-openjdk* readline-devel psmisc \
glibc-devel patch redhat-lsb-core expect libaio-devel flex bison ncurses-devel
11.准备安装介质
11.1在momo488m上创建安装包的存放目录
mkdir -p /opt/software/openGauss
chmod 755 -R /opt/software
11.2下载安装文件
cd /opt/software/openGauss
wget https://opengauss.obs.cn-south-1.myhuaweicloud.com/5.0.0/x86/openGauss-5.0.0-CentOS-64bit-all.tar.gz
11.3解压安装文件
tar -zxvf openGauss-5.0.0-CentOS-64bit-all.tar.gz
tar -zxf openGauss-5.0.0-CentOS-64bit-om.tar.gz
tar -zxf openGauss-5.0.0-CentOS-64bit-cm.tar.gz
12.创建XML文件
下面是一主一备一级联配置的xml
cat >> /opt/software/openGauss/cluster_config.xml << EOF
<?xml version="1.0" encoding="UTF-8"?>
<ROOT>
<!-- openGauss整体信息 -->
<CLUSTER>
<PARAM name="clusterName" value="Cluster_template" />
<PARAM name="nodeNames" value="momo488m,momo488s1,momo488s2" />
<PARAM name="gaussdbAppPath" value="/data/openGauss/install/app" />
<PARAM name="gaussdbLogPath" value="/var/log/omm" />
<PARAM name="tmpMppdbPath" value="/opt/huawei/tmp"/>
<PARAM name="gaussdbToolPath" value="/data/openGauss/install/om" />
<PARAM name="corePath" value="/opt/huawei/corefile"/>
<PARAM name="backIp1s" value="192.168.14.6,192.168.14.7,192.168.14.8"/>
</CLUSTER>
<!-- 每台服务器上的节点部署信息 -->
<DEVICELIST>
<!-- momo488m上的节点部署信息 -->
<DEVICE sn="momo488m">
<PARAM name="name" value="momo488m"/>
<PARAM name="azName" value="AZ1"/>
<PARAM name="azPriority" value="1"/>
<!-- 如果服务器只有一个网卡可用，将backIP1和sshIP1配置成同一个IP -->
<PARAM name="backIp1" value="192.168.14.6"/>
<PARAM name="sshIp1" value="192.168.14.6"/>
<!-- cm主 -->
<PARAM name="cmsNum" value="1"/>
<PARAM name="cmDir" value="/data/openGauss/install/cm"/>
<PARAM name="cmServerPortBase" value="15300"/>
<PARAM name="cmServerListenIp1" value="192.168.14.6,192.168.14.7,192.168.14.8"/>
<PARAM name="cmServerHaIp1" value="192.168.14.6,192.168.14.7,192.168.14.8"/>
<!-- cmServerlevel目前只支持1 -->
<PARAM name="cmServerlevel" value="1"/>
<!-- cms主及所有备的hostname -->
<PARAM name="cmServerRelation" value="momo488m,momo488s1,momo488s2"/>
<!--dn-->
<PARAM name="dataNum" value="1"/>
<PARAM name="dataPortBase" value="26000"/>
<PARAM name="datamomo488m" value="/data/openGauss/install/data/dn,momo488s1,/data/openGauss/install/data/dn,momo488s2,/data/openGauss/install/data/dn"/>
<PARAM name="datamomo488m_syncNum" value="0"/>
</DEVICE>
<!-- momo488s1上的节点部署信息，其中“name”的值配置为主机名称 -->
<DEVICE sn="momo488s1">
<PARAM name="name" value="momo488s1"/>
<PARAM name="azName" value="AZ1"/>
<PARAM name="azPriority" value="1"/>
<!-- 如果服务器只有一个网卡可用，将backIP1和sshIP1配置成同一个IP -->
<PARAM name="backIp1" value="192.168.14.7"/>
<PARAM name="sshIp1" value="192.168.14.7"/>
<!-- cm -->
<PARAM name="cmDir" value="/data/openGauss/install/cm"/>
<PARAM name="cmServerPortStandby" value="15300"/>
</DEVICE>
<!-- momo488s2上的节点部署信息，其中“name”的值配置为主机名称 -->
<DEVICE sn="momo488s2">
<PARAM name="name" value="momo488s2"/>
<PARAM name="azName" value="AZ1"/>
<PARAM name="azPriority" value="1"/>
<!-- 如果服务器只有一个网卡可用，将backIP1和sshIP1配置成同一个IP -->
<PARAM name="backIp1" value="192.168.14.8"/>
<PARAM name="sshIp1" value="192.168.14.8"/>
<PARAM name="cascadeRole" value="on"/>
<!-- cm -->
<PARAM name="cmDir" value="/data/openGauss/install/cm"/>
<PARAM name="cmServerPortStandby" value="15300"/>
</DEVICE>
</DEVICELIST>
</ROOT>
EOF
授予可执行权限
chmod +x /opt/software/openGauss/cluster_config.xml
13.配置环境变量和重启所有虚拟机
13.1设置环境变量
cat>> /etc/profile << EOF
export LD_LIBRARY_PATH=/opt/software/openGauss/script/gspylib/clib:$LD_LIBRARY_PATH
EOF
13.2重启所有虚拟机
init 6
14.执行交互模式前置脚本
在momo488m虚拟机执行过程中自动创建操作系统root用户互信和omm用户互信
cd /opt/software/openGauss/script/
./gs_preinstall -U omm -G dbgrp -X /opt/software/openGauss/cluster_config.xml
[root@momo488m script]# ./gs_preinstall -U omm -G dbgrp -X /opt/software/openGauss/cluster_config.xml
输入密码：MoMo488le
15.执行安装
15.1使用root给安装目录授权
cd /opt/software/openGauss/script
chmod -R 775 /opt/software/openGauss/script
chown -R omm:dbgrp /opt/software/openGauss/script
15.2使用omm自定义内存参数安装
su - omm
cd /opt/software/openGauss/script/
./gs_install -X /opt/software/openGauss/cluster_config.xml \
--gsinit-parameter="--encoding=UTF8" \
--dn-guc="max_connections=2000" \
--dn-guc="max_process_memory=2GB" \
--dn-guc="shared_buffers=128MB" \
--dn-guc="bulk_write_ring_size=128MB" \
--dn-guc="cstore_buffers=16MB"
16.安装验证
16.1以omm用户身份登录，命令检查CMServer状态、Cluster集群状态、数据库状态
[root@momo488m ~]# su - omm
Last login: Sat Apr 1 22:15:00 CST 2023 on pts/0
[omm@momo488m ~]$ gs_om -t status --detail
[ CMServer State ]
node node_ip instance state
----------------------------------------------------------------------
1 momo488m 192.168.14.6 1 /data/openGauss/install/cm/cm_server Primary
2 momo488s1 192.168.14.7 2 /data/openGauss/install/cm/cm_server Standby
3 momo488s2 192.168.14.8 3 /data/openGauss/install/cm/cm_server Standby
[ Cluster State ]
cluster_state : Normal
redistributing : No
balanced : Yes
current_az : AZ_ALL
[ Datanode State ]
node node_ip instance state
---------------------------------------------------------------------------
1 momo488m 192.168.14.6 6001 /data/openGauss/install/data/dn P Primary Normal
2 momo488s1 192.168.14.7 6002 /data/openGauss/install/data/dn S Standby Normal
3 momo488s2 192.168.14.8 6003 /data/openGauss/install/data/dn C Cascade Standby Normal
16.2使用gsql工具登录
[omm@momo488m ~]$ gsql -d postgres -p 26000 -r
gsql ((openGauss 5.0.0 build a07d57c3) compiled at 2023-03-29 03:07:56 commit 0 last mr )
Non-SSL connection (SSL connection is recommended when requiring high-security)
Type "help" for help.
openGauss=# select version();
version
----------------------------------------------------------------------------------------------------------------------------------------
(openGauss 5.0.0 build ...... compiled by g++ (GCC) 7.3.0, 64-bit
(1 row)
17.开启所有外部IP访问
17.1查询postgresql.conf参数listen_addresses
gs_guc check -I all -c "listen_addresses"
[omm@momo488m ~]$ gs_guc check -I all -c "listen_addresses"
The gs_guc run with the following arguments: [gs_guc -I all -c listen_addresses check ].
expected guc information: momo488m: listen_addresses=NULL: [/data/openGauss/install/data/dn/postgresql.conf]
gs_guc check: momo488m: listen_addresses='localhost,192.168.14.6': [/data/openGauss/install/data/dn/postgresql.conf]
Total GUC values: 1. Failed GUC values: 0.
The value of parameter listen_addresses is same on all instances.
listen_addresses='localhost,192.168.14.6'
17.2设置listen_addresses
gs_guc set -I all -c "listen_addresses='*'"
[omm@momo488m ~]$ gs_guc set -I all -c "listen_addresses='*'"
The gs_guc run with the following arguments: [gs_guc -I all -c listen_addresses='*' set ].
expected instance path: [/data/openGauss/install/data/dn/postgresql.conf]
gs_guc set: listen_addresses='*': [/data/openGauss/install/data/dn/postgresql.conf]
Total instances: 1. Failed instances: 0.
Success to perform gs_guc!
17.3修改pg_hba.conf访问设置
gs_guc reload -N all -I all -h "host all all 0.0.0.0/0 sha256"
[omm@momo488m ~]$ gs_guc reload -N all -I all -h "host all all 0.0.0.0/0 sha256"
The gs_guc run with the following arguments: [gs_guc -N all -I all -h host all all 0.0.0.0/0 sha256 reload ].
Begin to perform the total nodes: 3.
Popen count is 3, Popen success count is 3, Popen failure count is 0.
Begin to perform gs_guc for datanodes.
Command count is 3, Command success count is 3, Command failure count is 0.
Total instances: 3. Failed instances: 0.
ALL: Success to perform gs_guc!
17.4重启集群
gs_om -t stop && gs_om -t start
[omm@momo488m ~]$ gs_om -t stop && gs_om -t start
Stopping cluster.
=========================================
Successfully stopped cluster.
=========================================
End stop cluster.
Starting cluster.
======================================================================
Successfully started primary instance. Wait for standby instance.
======================================================================
.
Successfully started cluster.
======================================================================
cluster_state : Normal
redistributing : No
node_count : 3
Datanode State
primary : 1
standby : 1
secondary : 0
cascade_standby : 1
building : 0
abnormal : 0
down : 0
Successfully started cluster.
本文参考官方文档
https://docs.opengauss.org/zh/
https://docs.vastdata.com.cn/zh/