hbase笔记,
1.用户问题
最好不用root吧,如下新建一个用户,新建一个目录给这个用户,作为安装hbase吧
$ useradd hbase
$ mkdir /usr/local/hbase
$ chown hbase:hbase /usr/local/hbase/
2.版本升级问题
可以使用快捷方式,链接到hbase_home目录,对以后升级版本时,就修改链接就行。
$ ln -s /usr/local/hbase/hbase-0.90.4 /usr/local/hbase/current
原文:
We installed HBase 0.92.1 on a single server. We have used a symbolic link named current
for it, so that version upgrading in the future is easy to do.
3. $ list
hbase command, list all the tables.
4.关于多个系统同步
hadoop@master1$ for i in 1 2 3
do rsync -avz /usr/local/hadoop/ slave$i:/usr/local/hadoop/
sleep 1
done
5. 关于hadoop启动,先要格式化
$ HADOOP_HOME/bin/hadoop namenode -format
6.关于hadoop的查看
hdfs admin page:
http://master1:50070
MapReduce admin page:
http://master1:50030
7.关于跑多个zookeeper
实际时,至少三个,总是要单数个。
but in production it is recommended that you run a
ZooKeeper ensemble of at least three nodes. Also, make sure to run an odd number of nodes
8. zookeeper 配置 zoo.cfg
可以复制zoo_sample.cfg的。
hadoop@master1$ vi $ZK_HOME/conf/zoo.cfg
dataDir=/usr/local/ZooKeeper/var/data
dataLogDir=/usr/local/ZooKeeper/var/datalog
9.zookeeper的同步时,不要同步datadir和datalog目录
10. 配置多个zookeeper
server.1中的1是myid。
hadoop@node{1,2,3}$ vi $ZK_HOME/conf/zoo.cfg
server.1=node1:2888:3888
server.2=node2:2888:3888
server.3=node3:2888:3888
Also, you need to put a myid file under ${dataDir}. The myid file consists of a single line
containing only the node ID. So myid of node1 would contain the text 1 and nothing else.
11. 连接到zookeeper集群
三个server时,只能一个挂掉。
$ zkCli.sh -server node1,node2,node3
ZooKeeper will function as long as more than half of the nodes in the ZooKeeper cluster are
alive. This means, in a three node cluster, only one server can die.
12. 关于同时打开文件数限制
最大文件打开数,和一个进程最大文件打开数,都要设置为最大,使用nproc命令,否则有OutOfMemoryError。修改的必须对启动hadoop和hbase的用户。要注释才生效。
查看最大文件打开数:
hadoop$ ulimit -n
1024
查看1个进程的最大打开数:
hadoop$ ulimit -u
unlimited
或者直接修改
root# vi /etc/security/limits.conf
hadoop soft nofile 65535
hadoop hard nofile 65535
hadoop soft nproc 32000
hadoop hard nproc 32000