Hive安装配置，

和通数据库htsjk.Com2019-12-01 22:05 来源:未知阅读:1194 评论 19 热度2

标签：

Hive安装配置，

环境介绍：hadoop-2.9.1、hive-1.2.0

节点规划：
mini0 hadoop hive(server) metastore
mini1 hadoop hive(client) mysql
mini2 hadoop hive(client)

1、安装MYSQL
CentOS7 64位下MySQL5.7安装与配置:http://www.linuxidc.com/Linux/2016-09/135288.htm
mysql-connector-java.jar下载：https://www.cnblogs.com/jizuiku/p/7637125.html

2、配置MYSQL Service and Connector
说明：因为使用MySQL作为存储元数据的数据库，所以需要把连接MySQL的jar包放入或链接到$HIVE_HOME/lib目录下。

3、
3.1、mysql配置
启动MySQL服务：systemctl start mysqld
查看MySQL状态：systemctl status mysqld
开机启动：systemctl enable mysqld;systemctl daemon-reload
重启Mysql服务：service mysqld restart 或者 systemctl restart mysqld.service
默认只允许root帐户在本地登录，如果要在其它机器上连接mysql，必须修改root允许远程连接；或者添加一个允许远程连接的帐户，为了安全起见，我添加一个新的帐户：

        创建数据库
        drop database metastore;
        CREATE DATABASE metastore;
        USE metastore;
        SOURCE /home/hadoop/apps/hive/scripts/metastore/upgrade/mysql/hive-schema-0.9.0.mysql.sql;

        创建用户
        CREATE USER 'hive'@'mini0' IDENTIFIED BY 'zys123';
        REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hive'@'mini0';
        GRANT ALL ON metastore.* TO 'hive'@'mini0' IDENTIFIED BY 'zys123';
        GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY 'zys123';
        FLUSH PRIVILEGES;
        ALTER DATABASE metastore CHARACTER SET latin1;
        select user,host from mysql.user;
        flush privileges;
        quit;       

        删除用户
        select user,host from mysql.user;
        delete from mysql.user where user='hive' and host='%';
        delete from mysql.user where user='hive' and host='mini0';
        drop user 'hive'@'%';
        drop user 'hive'@'mini0';
        flush privileges;

4.配置hive
4.1、HDFS存储位置配置（Hive配置文件里要用到HDFS的一些路径，需要先手动创建。）

        hdfs dfs -mkdir -p /usr/hive/warehouse
        hdfs dfs -mkdir -p /usr/hive/tmp
        hdfs dfs -mkdir -p /usr/hive/log
        hdfs dfs -chmod 777 /usr/hive/warehouse
        hdfs dfs -chmod 777 /usr/hive/tmp 
        hdfs dfs -chmod 777 /usr/hive/log

上述语句涉及hive-site.xml hive.metastore.warehouse.dir等，表示数据在hdfs中的存储位置
4.2 配置hive-env.sh

        export HIVE_CONF_DIR=/home/hadoop/apps/hive/conf
        export HADOOP_HOME=/home/hadoop/apps/hadoop-2.9.1
        export HIVE_HOME=/home/hadoop/apps/hive

    然后配置hive-log4j.properties

        hive.log.dir=/usr/lib/hive/logs

4.3、服务端hive-site.xml（mini0）
（服务端指的是Metastore服务所在的机器，即安装metastore的机器）

<configuration>
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://mini1:3306/metastore?createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
        <description>Driver class name for a JDBC metastore</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>hive</value>
        <description>username to use against metastore database</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>zys123</value>
        <description>password to use against metastore database</description>
    </property>
    <property>
        <name>hive.metastore.warehouse.dir</name>
        <value>/usr/hive/warehouse</value>
    </property>
    <property>
        <name>hive.exec.scratchdir</name>
        <value>/usr/hive/tmp</value>
    </property>
    <property>
        <name>hive.querylog.location</name>
        <value>/usr/hive/log</value>
    </property>
</configuration>

1、客户端hive-site.xml（mini1、mini2）

<configuration> 
    <property>
        <name>hive.metastore.uris</name>
        <value>thrift://mini0:9083</value>
        <description>IP address (or fully-qualified domain name) and port of the metastore host</description>
    </property>
    <property>
        <name>hive.metastore.warehouse.dir</name>
        <value>/usr/hive/warehouse</value>
    </property>
    <property>
        <name>hive.exec.scratchdir</name>
        <value>/usr/hive/tmp</value>
    </property>
    <property>
        <name>hive.querylog.location</name>
        <value>/usr/hive/log</value>
    </property>
</configuration>

6、Jline包版本不一致的问题，需要拷贝hive的lib目录中jline.2.12.jar的jar包替换掉hadoop中的
/home/hadoop/app/hadoop-2.9.1/share/hadoop/yarn/lib/jline-0.9.94.jar

7、metastore初始化（必须要初始化，否则后边hiveserver2无法启动）：

schematool -dbType mysql -initSchema

8、启动Hive
1、**本地方式启动**hive:hive
2、
服务端启动：通过Hive thrift服务（远程）
启动为前台：hiveserver2
启动为后台：nohup hiveserver2 &
客户端启动：
方式一：beeline回车,进入beeline的命令界面
输入命令连接：!connect jdbc:hive2//mini0:10000 -n hadoop
（mini0是hiveserver2所启动的那台主机名，端口默认是10000）
方式二：直接启动：beeline -u jdbc:hive2://mini0:10000 -n hadoop

查看hiveserver2服务端状态：netstat -nltp | grep 10000

Hive基本操作：
1、hive命令：
清空表：truncate table person;
创建数据库：create database test_db
删除数据库：drop database test_db;
删除表：drop table ****;

6.建表(默认是内部表)

create table person(name string,id int) row format delimited fields terminated by '\t';

建分区表

create table td_part(id bigint, account string, income double, expenses double, time string) partitioned by (logdate string) row format delimited fields terminated by '\t';

建外部表

create external table td_ext(id bigint, account string, income double, expenses double, time string) row format delimited fields terminated by '\t' location '/td_ext';

参考网站： Hive安装配置指北（含Hive
Metastore详解）：https://www.cnblogs.com/linbingdong/p/5829369.html