Hive3.1.3集群安装

发布时间 2023-06-27 11:40:29作者: KwFruit

一、安装mysql

Hive默认的Derby数据库一次只能打开一个会话,不支持多用户并发访问,需要替换Hive的存储引擎为mysql

1. 下载mysql安装包

wget http://dev.mysql.com/get/mysql57-community-release-el7-7.noarch.rpm

2. 安装mysql源

yum localinstall -y mysql57-community-release-el7-7.noarch.rpm

ll /etc/yum.repos.d/

3. 安装mysql

yum install mysql-community-server -y

此时安装报错

Public key for mysql-community-server-5.7.41-1.el6.x86_64.rpm is not installed

解决方案:

rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022
yum install mysql-community-server -y

 

4. 启动mysql和设置开机启动

# 启动
systemctl start mysqld
# 重启
systemctl restart mysqld
# 查看状态
systemctl status mysqld
# 设置开机启动
systemctl enable mysqld
systemctl daemon-reload

修改/etc/my.cnf配置文件,配置默认编码为utf-8,
(注意其他目录不要动)

character_set_server=utf8

重启服务:

systemctl restart mysqld

5.查询默认密码并修改密码

查询默认密码:

grep 'temporary password' /var/log/mysqld.log

 使用临时密码登录mysql,密码有安全插件检测(大小写字母+数字+特殊字符,大于7位)

mysql -uroot -p
alter user 'root'@'localhost' identified by 'xxxxxx@123A';

6.创建hive数据库用户

create user 'hive'@'%' identified by 'xxxxxx@123A';
grant all privileges on *.* to 'root'@'%' identified by 'xxxxxx@123A';
grant all privileges on *.* to 'hive'@'%' identified by 'xxxxxx@123A';
flush privileges;

二、Hive安装

1.修改hive-config.sh配置

vim /usr/hive/apache-hive-3.1.3-bin/bin/hive-config.sh

文件后面添加:

export JAVA_HOME=/usr/java/jdk1.8.0_181
export HADOOP_HOME=/usr/hadoop/hadoop-3.1.3
export HIVE_HOME=/usr/hive/apache-hive-3.1.3-bin

2.下载mysql驱动

https://downloads.mysql.com/archives/c-j/
选择版本下载:

上传驱动包至hadoop160服务器的/usr/mysql目录解压

tar -zxvf mysql-connector-java-5.1.49.tar.gz

复制驱动包至$HIVE_HOME/lib目录下

cp mysql-connector-java-5.1.49-bin.jar /usr/hive/apache-hive-3.1.3-bin/lib

3.Hive配置修改

创建临时目录

mkdir /usr/hive/apache-hive-3.1.3-bin/tmp

修改hive-site.xml,更换Derby数据库的连接信息为mysql的,
vim hive-site.xml 命令模式下使用/向后查找关键字Connection,使用?向前查找,更改ConnectionPassword、ConnectionURL、ConnectionDriverName、ConnectionUserName四项配置:
如果没有 hive-site.xml   将 conf 下的 hive-default.xml.template 复制改名成hive-site.xml

<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>xxxxxx@123A</value>
</property>
  
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://hadoop0:3306/hive?createDatabaseIfNotExist=true&amp;useUnicode=true&amp;serverTimezone=GMT%2B8&amp;characterEncoding=UTF-8&amp;zeroDateTimeBehavior=convertToNull&amp;allowMultiQueries=true&amp;autoReconnect=true&amp;rewriteBatchedStatements=true&amp;useSSL=false</value>
    </property>

<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
   </property>
  
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
  </property>

替换系统临时目录和用户名:

替换hive-site.xml文件中:
${system:java.io.tmpdir} 替换为 ${HIVE_HOME}/tmp
${system:user.name} 替换为 root

全局替换

4.数据库初始化和验证

初始化hive数据库:主要是在bin目录

./schematool -dbType mysql -initSchema

此时报错:

-bash: schematool: command not found

yum -y install schematool

如果此时还是报错:

Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.ba

. 解决

  hadoop和hive的两个guava.jar版本不一致
  两个位置分别位于下面两个目录:
  - /usr/local/hive/lib/
  - /usr/local/hadoop/share/hadoop/common/lib/

 重新初始化此时还是报错:

./schematool -dbType mysql -initSchema

 解决方案:

打开hive-site.xml  搜索  Ensures commands with OVERWRITE (such as IN  删除这个 description 标签

 重新初始化此时还是报错: 

 请检查 hive-site.xml  中 你配的mysql 账号密码是否正确

重新执行

./schematool -dbType mysql -initSchema

 成功安装!!!!!SUCCESS