欢迎投稿

今日深度：

> > 正文

Hive安装，

来源:未知 阅读:14043 评论 388 热度4

标签：

Hive安装，


安装环境

hadoop2.7.3集群
centOS 6.5—-master
centOS 6.5—-slave1
centOS 6.5—-slave2

安装版本

MySQL-5.6.22
Hive-2.3.0

为方便起见本文档相关路径均为作者本机的路径，参考请看准路径

安装mysql

Hive 的元数据是一般是存储在关系型数据库上的，我们这里选择MySQL，Hive和MySQL之间通过MetaStore服务交互，因此首先安装MySQL数据库以作为Hive的元数据仓库

    //服务端依赖库
     yum -y install libaio.so.1
     yum update libstdc++-4.4.7-4.el6.x86_64
     yum -y install libstdc++.so.6
     yum -y install libgcc_s.so.1
     //客户端依赖库
     yum -y install libncurses.so.5
     yum -y install libtinfo.so.5

使用rpm命令安装mysql

    //安装服务端
    rpm -ivh MySQL-server-xxxxxxx.rpm
    //安装客户端
    rpm -ivh MySQL-client-xxxx.rpm
    //安装库和包含文件
    rpm -ivh MySQL-devel-xxxx.rpm

3.配置MySQL。mysql安装完成后就可以启动mysql服务登录后进行相关配置了。

    //启动mysql
    service mysql start
    //将mysql加入到系统服务中
    chkconfig --add mysql
    //设置开机自启动
    chkconfig mysql on
    //mysql安装好后会生成一个临时随机密码,存储位置在 /root/.mysql_secret,查看随机临时密码
    cat /root/.mysql_secret
    //登录mysql
    mysql -uroot -p随机密码
    //登陆成功后修改密码
    set password = password('新密码');
    //默认情况下mysql为安全起见，不支持远程登录mysql，所以需要设置开启远程登录mysql的权限,重新登录mysql
    grant all privileges on *.*to 'root' @'%' identified by 'root';
    //刷新是权限生效
    flush privileges;
    //开放Linux的对外访问的端口3306
    /sbin/iptables -I INPUT -p tcp --dport 3306 -j ACCEPT
    //保存设置
    /etc/rc.d/init.d/iptables save

rpm包安装的MySQL是不会安装/etc/my.cnf文件的，我们可以手动复制文件到/etc下cp /usr/share/mysql/my-default.cnf /etc/my.cnf（5.5版本可选择my-huge.cnf），设置etc/my.cnf文件，使binlog_format=mixed vi etc/my.cnf,添加binlog_format=mixed 重启mysql service mysql restart

注：5.6版本的mysql在安装完成后会产生随机临时密码以供root用户进行初次登陆，而在5.5版本的mysql安装完成后并不会产生随机密码而是会创建一个密码为空的临时用户，此时可以使用mysql提供的初始化程序快速设置mysql/usr/bin/mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL

SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!

In order to log into MySQL to secure it, we'll need the current

password for the root user. If you've just installed MySQL, and

you haven't set the root password yet, the password will be blank,

so you should just press enter here.

Enter current password for root (enter for none):<–初次运行直接回车

OK, successfully used password, moving on…

Setting the root password ensures that nobody can log into the MySQL

root user without the proper authorisation.

Set root password? [Y/n] <– 是否设置root用户密码，输入y并回车或直接回车

New password: <– 设置root用户的密码

Re-enter new password: <– 再输入一次你设置的密码

Password updated successfully!

Reloading privilege tables..

… Success!

By default, a MySQL installation has an anonymous user, allowing anyone

to log into MySQL without having to have a user account created for

them. This is intended only for testing, and to make the installation

go a bit smoother. You should remove them before moving into a

production environment.

Remove anonymous users? [Y/n] <– 是否删除匿名用户,生产环境建议删除，所以直接回车

… Success!

Normally, root should only be allowed to connect from 'localhost'. This

ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] <–是否禁止root远程登录,根据自己的需求选择Y/n并回车

… Success!

By default, MySQL comes with a database named 'test' that anyone can

access. This is also intended only for testing, and should be removed

before moving into a production environment.

Remove test database and access to it? [Y/n] <– 是否删除test数据库,直接回车

- Dropping test database…

… Success!

- Removing privileges on test database…

… Success!

Reloading the privilege tables will ensure that all changes made so far

will take effect immediately.

Reload privilege tables now? [Y/n] <– 是否重新加载权限表，直接回车

… Success!

Cleaning up…

All done! If you've completed all of the above steps, your MySQL

installation should now be secure.

Thanks for using MySQL!

安装Hive

1.将Hive安装在hadoop的namenode即master上，

  • 拷贝安装文件到linux中/usr/tools/apache-hive-2.0.0-bin.tar.gz
  • 解压：tar –zxvf apache-hive-2.0.0-bin.tar.gz
  • 配置环境变量：vi /etc/profile
        #hive
        export HIVE_HOME=/usr/tools/apache-hive-2.0.0-bin
        export PATH=$PATH:$HIVE_HOME/bin

保存后使其生效：source /etc/profile

2.配置Hive
- 在hdfs中新建目录

hdfs dfs –mkdir /tmp
hdfs dfs –mkdir /user
hdfs dfs –mkdir /user/hive
hdfs dfs –mkdir /user/hive/warehouse
  • 修改文件夹权限
hadoop fs -chmod g+w /tmp
hadoop fs -chmod g+w /user/hive/warehouse

  • 将mysql的驱动jar包mysql-connector-java-5.1.7-bin.jar拷入hive的lib目录下面

  • 进入hive的conf目录下面复制一下hive-default.xml.template名子命名为：hive-site.xml
    cp hive-default.xml.template hive-site.xml，修改文件内的参数如下：

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://127.0.0.1:3306/hive?createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>Username to use against metastore database</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>sa</value>
    <description>password to use against metastore database</description>
  </property>

  <property>
    <name>hive.exec.local.scratchdir</name>
    <value>/usr/tools/apache-hive-2.0.0-bin/tmp</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/usr/tools/apache-hive-2.0.0-bin/tmp/resources</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>
  <property>
    <name>hive.querylog.location</name>
    <value>/usr/tools/apache-hive-2.0.0-bin/tmp</value>
    <description>Location of Hive run time structured log file</description>
  </property>
  <property>
    <name>hive.server2.logging.operation.log.location</name>
    <value>/usr/tools/apache-hive-2.0.0-bin/tmp/operation_logs</value>
    <description>Top level directory where operation logs are stored if logging functionality is enabled</description>
  </property>


  • 初始化metastore
    schematool -initSchema -dbType mysql

如果显示初始化失败请检查mysql下的用户的权限是否拥有远程连接的全部权限、数据库连接驱动是否正常、hadoop集群运行状况是否正常、集群时间是否同步等。
3.运行hive
在命令行中敲入hive进入hive命令行，可以正确执行hive命令，即安装配置成功
一般情况下关系数据库作为hive的metastroe需要摄者编码为拉丁字母，在mysql中运行以下两句命令alter database hive character set latin1;
set character_set_client=latin1;

www.htsjk.Com true http://www.htsjk.com/hive/40896.html NewsArticle Hive安装， 安装环境 hadoop2.7.3集群 centOS 6.5—-master centOS 6.5—-slave1 centOS 6.5—-slave2 安装版本 MySQL-5.6.22 Hive-2.3.0 为方便起见本文档相关路径均为作者本机的路径，参考请看准路径 安装m...
本站文章为和通数据库网友分享或者投稿，欢迎任何形式的转载，但请务必注明出处.
同时文章内容如有侵犯了您的权益，请联系QQ：970679559，我们会在尽快处理。
返回首页
相关文章
    暂无相关文章
评论暂时关闭