Oracle RAC部署前期配置-526互联

1.挂载image file镜像文件

2.运行下边的语法配置yum源

> cd /etc/yum.repos.d/
> vi /etc/yum.repos.d/yum.repo
[yum]
name=yum
baseurl=file:///mnt
enabled=1
gpgcheck=0

3.挂载sr0镜像

mount /dev/sr0 /mnt

4.安装rpm包,在mini系统安装后，需要安装一些基础包

> yum groupinstall base

当存储系统连接上，安装下边的包并执行rescan-scsi-bus.sh 脚本，来扫描多路径软件所需要的映射信息

> yum install sg3_utils*

要用ORAchk，安装以下RPM包

> yum install sg3_utils*

安装以下包，实现时间同步

> yum install ntp*

安装以下包来提供图形界面安装 Oracle Grid 和 Oracle Database

yum install java*
yum install xdpy*

在安装 Oracle Grid，会提示安装以下包

yum install compat*
yum install libaio*
yum install ksh*
yum install gcc*
yum install libstdc*
yum install libcap*
yum install nfs-utils*

如果安装数据库的OS是RHEL7，会有报警提示要安装 compat-libstdc++-33-3.2.3

rpm -ivh compat-libstdc++-33-3.2.3-72.el7.x86_64.rpm

5.配置host文件

IP地址类型	域名	IP地址
Public IP	testdb01 testdb02	10.36.21.248 10.36.21.249
Private IP	tsdb01-pri tsdb02-pri	100.100.100.30 100.100.100.31
Virtual IP	tsdb01-vip tsdb02-vip	10.36.21.225 10.36.21.226
SCAN IP	tsdb-scan	10.36.21.227

[root@dbn01 ~]# vi /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
127.0.0.1 localhost.localdomain localhost
#Public
10.36.21.248 testdb01
10.36.21.249 testdb02
#Private
100.100.100.30 tsdb01-pri
100.100.100.31 tsdb02-pri
#Virtual
10.36.21.225 tsdb01-vip
10.36.21.226 tsdb02-vip
#scan
10.36.21.227 tsdb-scan

6.修改hostname

注意：hostname不能包含(_) 和 (@)字符

如果是RHEL 6，配置以下内容

 [root@dbn01 ~]# vim /etc/sysconfig/network <br>
 NETWORKING=yes
 HOSTNAME=dbn01

如果是RHEL 7，配置以下内容

[root@dbn01~ ]# hostnamectl set-hostname dbn01

7.关闭防火墙

如果是RHEL 6，执行以下命令关闭

[root@dbn01 ~]# service iptables stop
[root@dbn01 ~]# chkconfig iptables off
[root@dbn01 ~]# service iptables status
iptables: Firewall is not running.

如果是RHEL 7，执行以下命令关闭

[root@dbn01 ~]# systemctl stop firewalld
[root@dbn01 ~]# systemctl disable firewalld

8.关闭Transparent Huge Pages

1）开通透明大页， I/O 读写性能降低 30%，2）防止实例或节点被集群移除

（此处用tidb “环境与系统配置”-部署文档配置）
原文配置参考：https://support.huawei.com/enterprise/en/doc/EDOC1100053122/90aabdb0/disabling-thp

[root@dbn01~]# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never

>> 如果"always"显示在输出界面，则表示启动了透明大页

>> 执行 grubby 命令查看默认内核版本
[root@testdb01 ~]# grubby --default-kernel
/boot/vmlinuz-4.1.12-61.1.28.el6uek.x86_64

>> 执行 grubby --update-kernel 命令修改内核配置
[root@testdb01 ~]# grubby --args="transparent_hugepage=never" --update-kernel /boot/vmlinuz-4.1.12-61.1.28.el6uek.x86_64

>> 执行 grubby --info 命令查看修改后的默认内核配置
[root@testdb01 ~]# grubby --info /boot/vmlinuz-4.1.12-61.1.28.el6uek.x86_64
boot=/dev/sda
index=0
kernel=/vmlinuz-4.1.12-61.1.28.el6uek.x86_64
args="ro rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=system/swap  KEYBOARDTYPE=pc KEYTABLE=us rd_LVM_LV=system/home rd_NO_DM rhgb quiet transparent_hugepage=never"
root=/dev/mapper/system-home
initrd=/boot/initramfs-4.1.12-61.1.28.el6uek.x86_64.img

>> 修改当前的内核配置立即关闭透明大页。
[root@testdb01 ~]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@testdb01 ~]# echo never > /sys/kernel/mm/transparent_hugepage/defrag

9.手动创建用户

1) 创建用户和用户组

/usr/sbin/groupadd -g 1000 oinstall
/usr/sbin/groupadd -g 1001 dba
/usr/sbin/groupadd -g 1002 oper
/usr/sbin/groupadd -g 1010 asmadmin
/usr/sbin/groupadd -g 1011 asmdba
/usr/sbin/groupadd -g 1012 asmoper
/usr/sbin/useradd -u 1000 -g oinstall -G dba,asmdba,oper -m -d /home/oracle oracle
/usr/sbin/useradd -u 1001 -g oinstall -G asmadmin,asmdba,asmoper,oper,dba -m -d /home/grid grid

2）设置用户密码，所有节点都设置一样

[root@dbn01 ~]# passwd oracle
Changing password for user oracle.
New password:
BAD PASSWORD: The password is shorter than 8 characters
Retype new password:
passwd: all authentication tokens updated successfully.

[root@dbn01 ~]# passwd grid
Changing password for user grid.
New password:
BAD PASSWORD: The password is shorter than 8 characters
Retype new password:
passwd: all authentication tokens updated successfully.

10.配置SSH互信

分别登录Oracle、grid，执行以下命令配置互信

1）登录rac的节点1：
dbn01: # ssh-keygen -t rsa
2）在节点2执行：
dbn01: # ssh-keygen -t rsa

3）在RAC-node1执行：
ssh dbn01 "echo $(cat /home/oracle/.ssh/id_rsa.pub) >> /home/oracle/.ssh/authorized_keys"
ssh dbn02 "echo $(cat /home/oracle/.ssh/id_rsa.pub) >> /home/oracle/.ssh/authorized_keys"

4）在RAC-node2执行：
ssh dbn01 "echo $(cat /home/oracle/.ssh/id_rsa.pub) >> /home/oracle/.ssh/authorized_keys"
ssh dbn02 "echo $(cat /home/oracle/.ssh/id_rsa.pub) >> /home/oracle/.ssh/authorized_keys"

5）确认下配置结果

11.配置环境变量

为数据库配置ORACLE_UNQNAME，配置唯一的ORACLE_SID，以下分别为oracle、grid、root用户设置环境变量
，打开.bash_profile文件

节点1：
[root@dbn01~]# su - oracle
[oracle@dbn01 ~]# vi .bash_profile
export PATH
export TMPDIR=$TMP
export ORACLE_HOSTNAME=dbn01
export ORACLE_UNQNAME=db0
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db
export ORACLE_SID=dbn01
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

[root@dbn01~]# su - grid
[grid@dbn01 ~]# vi .bash_profile
export PATH
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_HOSTNAME=dbn01
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/11.2.0/grid
export ORACLE_SID=+ASM1
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

[root@dbn01 ~]# vi .bash_profile
export GRID_HOME=/u01/app/11.2.0/grid
export ORACLE_HOME=/u01/app/oracle/product/11.2.0/db
export PATH=$GRID_HOME/bin:$GRID_HOME/OPatch:/sbin:/bin:/usr/sbin:/usr/bin

> 节点2，复制节点1，并进行部分修改
[root@dbn01~]# su - oracle
[oracle@rac2 ~]# vi /home/oracle/.bash_profile
修改     export ORACLE_HOSTNAME=dbn02
修改     export ORACLE_SID=dbn02

[root@dbn01~]# su - grid
[grid@rac2 ~]# vi /home/grid/.bash_profile
修改      export ORACLE_SID=+ASM2
修改      export ORACLE_HOSTNAME=dbn02

[root@dbn02 ~]# vi .bash_profile
export GRID_HOME=/u01/app/11.2.0/grid
export ORACLE_HOME=/u01/app/oracle/product/11.2.0/db
export PATH=$GRID_HOME/bin:$GRID_HOME/OPatch:/sbin:/bin:/usr/sbin:/usr/bin

12.设置系统参数

1）/etc/selinux/config，SELINUX 为 disabled

vi /etc/selinux/config
SELINUX=disabled

2）/etc/security/limits.conf，SELINUX 为 disabled，注意计算，有参考（待写）

>> 如果OS是 RHEL 6，加如下配置
[root@dbn01 ~]# vi /etc/security/limits.conf
#ORACLE SETTING
grid soft nproc 65536
grid hard nproc 65536
grid soft nofile 65536
grid hard nofile 65536
oracle soft nproc 65536
oracle hard nproc 65536
oracle soft nofile 65536
oracle hard nofile 65536
oracle soft memlock 1425011166
oracle hard memlock 1425011166

>> 如果OS是 RHEL 7，加如下配置
[root@dbn01 ~]# vi /etc/security/limits.conf
#ORACLE SETTING
grid soft nproc 65536
grid hard nproc 65536
grid soft nofile 65536
grid hard nofile 65536
oracle soft nproc 65536
oracle hard nproc 65536
oracle soft nofile 65536
oracle hard nofile 65536
grid soft stack 65536
oracle soft stack 65536
oracle soft memlock 1425011166
oracle hard memlock 1425011166

3）/etc/sysconfig/network

[root@dbn01~]# vi /etc/sysconfig/network
NOZEROCONF=yes

4）配置/etc/sysctl.conf文件，注意计算，有参考（待写）

[root@dbn01~]# vi /etc/sysctl.conf
#ORACLE SETTING
kernel.shmall = 4294967296
kernel.shmmax = 384829069722
kernel.shmmni = 4096
kernel.sem = 12000 1536000 100 128
fs.file-max = 6815744
fs.aio-max-nr = 3145728
net.ipv4.ip_local_port_range = 9000 65500
net.ipv4.ipfrag_high_thresh = 16777216
net.ipv4.ipfrag_low_thresh = 15728640
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
vm.min_free_kbytes= 1048576

5）用root用户执行以下命令，使新配置的参数生效

[root@dbn01~]# sysctl -p

13.配置大页

1）检查大页大小是否是2048 KB（2M）

[root@dbn01~]# cat /proc/meminfo |grep Hugepagesize
Hugepagesize: 2048 kB

2）计算 vm.nr_hugepages的值

计算公式
vm.nr_hugepages = (node physical memory x 0.65 x 0.75 + 2) x 1024 //Huge page memory size
假设物理内存是512 GB，则计算方式如下：
vm.nr_hugepages = (512 GB x 0.65 x 0.75 + 2) x 1024/2 MB = 128819.2 (pages)

3）配置/etc/sysctl.conf 文件

[root@dbn01~]# vi /etc/sysctl.conf
vm.nr_hugepages= 128819

4）执行以下命令，使设置参数生效，如果设置不生效，请重启OS

[root@dbn01~]# sysctl -p
[root@dbn01~]# sysctl -a|grep nr_hugepages
vm.nr_hugepages = 128819
vm.nr_hugepages_mempolicy = 128819

13.创建RAC安装目录，用root账号在每个节点操作

mkdir -p /u01/app/11.2.0/grid
mkdir -p /u01/app/grid
mkdir -p /u01/app/oracle/product/11.2.0/db
chown -R grid:oinstall /u01/app
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/

14.配置时间同步

需配置NTP时间同步而不用Oracle集群时间同步服务。忽略在安装Oracle Grid组件时的告警，不能装chrony软件因为它会报错，可用 rpm -e chrony-2.1.1-3.el7.x86_64删除chrony包

OS is RHEL 6

1) 设置开机启动

>> 启动ntpd和配置chkconfig ntpd on开机启动
[root@dbn01 ~]# service ntpd start
[root@dbn01 ~]# chkconfig ntpd on

2) 添加IP到配置文件

[root@dbn01 ~]# vi /etc/ntp.conf
server 192.168.35.31

3) 重启NTP服务

[root@dbn01 ~]# service ntpd restart

4) 添加ntpq -p命令检查NTP服务状态

[root@dbn01 ~]# ntpq -p
remote refid st t when poll reach delay offset jitter
========================================================
192.168.35.31 .LOCL. 1 u 2 64 1 0.417 5.373 0.000

OS is RHEL 7

1) 设置开机启动

>> 启动ntpd和配置chkconfig ntpd on开机启动
[root@dbn01~]# systemctl enable ntpd.service

2) 添加IP到配置文件

[root@dbn01 ~]# vi /etc/ntp.conf
server 192.168.35.31

3) 重启NTP服务

[root@dbn01~]# systemctl restart ntpd.service

4) 添加ntpq -p命令检查NTP服务状态

[root@dbn01~]# ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================
192.168.35.31 .LOCL. 1 u 2 64 1 0.417 5.373 0.000

15.配置ASM磁盘

1) 添加raw裸磁盘到服务器

2) 查看磁盘名称

[root@localhost rules.d]# fdisk -l
/dev/sdg
/dev/sde

3) 查看每个涉及的盘的ID_Serial，如果我们用磁盘名称，名称可能在系统reboot后发生改变

udevadm info --query=property /dev/sdg
udevadm info --query=property /dev/sde

>> 正确示例：
udevadm info --query=property --name=/dev/sdc |grep 'ID_SERIAL='
udevadm info --query=property --name=/dev/sdd |grep 'ID_SERIAL='

4) 可以在/etc/udev/rules.d/60-raw.rules文件中，编写UDEV规则，可用以下“规则文件1”、“规则文件2”，操作环境为Redhat 6.

规则文件1：
ACTION=="add|change", SUBSYSTEM=="block", ENV{ID_SERIAL}=="36000c297a90d6a9681b1d2a5a1b88568", GROUP="grid", SYMLINK+="oraasm01",OWNER="oinstall", MODE="0660"
ACTION=="add|change", SUBSYSTEM=="block", ENV{ID_SERIAL}=="36000c29906aa418e9822a0947ce9ad22", GROUP="grid", SYMLINK+="oraasm02",OWNER="oinstall", MODE="0660"
ACTION=="add|change", SUBSYSTEM=="block", ENV{ID_SERIAL}=="36000c291823fef46b990dbc1fe7bebdf", GROUP="grid", SYMLINK+="oraasm03",OWNER="oinstall", MODE="0660"
ACTION=="add|change", SUBSYSTEM=="block", ENV{ID_SERIAL}=="36000c2944397980b1b7f91a3d0c99e56", GROUP="grid", SYMLINK+="oraasm04",OWNER="oinstall", MODE="0660"
ACTION=="add|change", SUBSYSTEM=="block", ENV{ID_SERIAL}=="36000c2954d200a2927a1e815945e603d", GROUP="grid", SYMLINK+="oraasm05",OWNER="oinstall", MODE="0660"
** 注意：group和owner设置，在10g为oracle、oinstall，11g为grid、oinstall


规则文件2：
#OCR
ACTION=="add|change", SUBSYSTEM=="block" , ENV{ID_SERIAL}=="36000c29bb1bb3462e05be7fdb10b02a2", RUN+="/bin/raw /dev/raw/raw1 %N"
ACTION=="add|change", SUBSYSTEM=="block" , ENV{ID_SERIAL}=="36000c29bb1bb3462e05be7fdb10b02a2", RUN+="/bin/raw /dev/raw/raw2 %N"
ACTION=="add|change", SUBSYSTEM=="block" , ENV{ID_SERIAL}=="36000c29bb1bb3462e05be7fdb10b02a2", RUN+="/bin/raw /dev/raw/raw3 %N"
#
###DATA
ACTION=="add|change", SUBSYSTEM=="block" , ENV{ID_SERIAL}=="36000c29bb1bb3462e05be7fdb10b02a2", RUN+="/bin/raw /dev/raw/raw4 %N"
ACTION=="add|change", SUBSYSTEM=="block" , ENV{ID_SERIAL}=="36000c29bb1bb3462e05be7fdb10b02a2", RUN+="/bin/raw /dev/raw/raw5 %N"
ACTION=="add|change", SUBSYSTEM=="block" , ENV{ID_SERIAL}=="36000c29bb1bb3462e05be7fdb10b02a2", RUN+="/bin/raw /dev/raw/raw6 %N"
#
###FRA archivlog
ACTION=="add|change", SUBSYSTEM=="block" , ENV{ID_SERIAL}=="36000c29bb1bb3462e05be7fdb10b02a2", RUN+="/bin/raw /dev/raw/raw7 %N"
ACTION=="add|change", SUBSYSTEM=="block" , ENV{ID_SERIAL}=="36000c29bb1bb3462e05be7fdb10b02a2", RUN+="/bin/raw /dev/raw/raw8 %N"
ACTION=="add|change", SUBSYSTEM=="block" , ENV{ID_SERIAL}=="36000c29bb1bb3462e05be7fdb10b02a2", RUN+="/bin/raw /dev/raw/raw9 %N"
#
##
ACTION=="add",KERNEL=="raw[1-9]",OWNER=="grid",GROUP=="oinstall",MODE=="0660"
ACTION=="add",KERNEL=="raw1[0-9]",OWNER=="grid",GROUP=="oinstall",MODE=="0660"

5) 重新加载UDEV规则，让其识别新磁盘

udevadm control --reload-rules
udevadm trigger --type=devices --action=change

6) 查看新识别的磁盘

ls -l /dev/oraasm05
ls -l /dev/oraasm03
ls -l /dev/oraasm04
ls -l /dev/oraasm01
ls -l /dev/oraasm02
或
# ll /dev/raw/*

16.在RHEL 7，avahi-daemon服务是默认开启，需关闭

[root@dbn01 ]# systemctl status avahi-daemon
[root@dbn01 ]# systemctl stop avahi-daemon
[root@dbn01 ]# systemctl disable avahi-daemon