HBase Tools and Utilities，hbaseutilities

HBase Tools and Utilities，hbaseutilities

hadoop jar /usr/local/fqlhadoop/hbase-1.1.2/lib/hbase-server-1.1.2.jar与

hbase org.apache.hadoop.hbase.mapreduce.Driver等效。

hadoop /usr/local/fqlhadoop/hbase-1.1.2/lib/hbase-server-1.1.2.jar importtsv

hbase org.apache.hadoop.hbase.mapreduce.Driver importtsv

hbase org.apache.hadoop.hbase.mapreduce.ImportTsv

####129. HBase Tools and Utilities BEGIN####

#1
hbase org.apache.hadoop.hbase.mapreduce.Driver
##1.1
hbase org.apache.hadoop.hbase.mapreduce.Driver verifyrep与hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication等效


hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication --starttime=1475251200000 --endtime=1475424000000 1 hbase_test:fql_hbase_replication_ms


hbase org.apache.hadoop.hbase.mapreduce.Driver verifyrep --starttime=1475251200000 --endtime=1475424000000 1 hbase_test:fql_hbase_replication_ms


#1.2
hbase org.apache.hadoop.hbase.mapreduce.Driver importtsv
hbase org.apache.hadoop.hbase.mapreduce.ImportTsv


#2 hbase wal
wal             Write-ahead-log analyzer


WAL Tools
hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog


##get a textual dump of a WAL file
hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump /hbase/WALs/11.hadoop.com,16020,1474628670626/11.hadoop.com%2C16020%2C1474628670626.default.1475741264910


##force a split of a log file directory
hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --split /hbase/WALs/5.hadoop.com,16020,1475304063995


##print the contents of a WAL
hbase wal /hbase/WALs/11.hadoop.com,16020,1474628670626/11.hadoop.com%2C16020%2C1474628670626.default.1475741613348
hbase wal --printvals /hbase/WALs/11.hadoop.com,16020,1474628670626/11.hadoop.com%2C16020%2C1474628670626.default.1475741613348
hbase wal --json --printvals /hbase/WALs/11.hadoop.com,16020,1474628670626/11.hadoop.com%2C16020%2C1474628670626.default.1475742164406


hbase org.apache.hadoop.hbase.wal.WALPrettyPrinter
 
旧版
hbase hlog 
hbase org.apache.hadoop.hbase.regionserver.wal.HLogPrettyPrinter


#3 hbase hbck
hbck            Run the hbase 'fsck' tool


hbase hbck
hbase hbck hbase org.apache.hadoop.hbase.util.HBaseFsck


hbase hbck -details
hbase hbck -metaonly
hbase hbck -fixAssignments
hbase hbck -fix   
hbase hbck -repairHoles


#4 verify the compressor is available
hbase org.apache.hadoop.hbase.util.CompressionTest /compressionTest snappy
hbase org.apache.hadoop.hbase.util.CompressionTest /compressionTest lzo
hbase org.apache.hadoop.hbase.util.CompressionTest /compressionTest gz


#5 WALPlayer
hbase org.apache.hadoop.hbase.mapreduce.WALPlayer


#6.1 RowCounter 
hbase org.apache.hadoop.hbase.mapreduce.RowCounter -Dhbase.client.scanner.caching=100 PH_MONITOR.T_MONITOR_INTERFACE_LOG


#6.2 RowCounter 
hbase org.apache.hadoop.hbase.mapreduce.CellCounter PH_MONITOR.T_MONITOR_FSOF_PROVIDER_SERVICE /test53 |


<property>
  <name>mapreduce.job.counters.limit</name>
  <!--<value>120</value>-->
  <value>20000</value>
  <description>Limit on the number of counters allowed per job. The default value is 200.</description>
</property>


#7
hbase org.apache.hadoop.hbase.regionserver.CompactionTool


#8
hbase clean


#9 hbase mapredcp
hbase org.apache.hadoop.hbase.util.MapreduceDependencyClasspathTool


#10 hbase pe
hbase org.apache.hadoop.hbase.PerformanceEvaluation


#11 hbase ltt
hbase ltt
hbase org.apache.hadoop.hbase.util.LoadTestTool


其他
hbase org.apache.flume.tools.GetJavaProperty
hbase org.apache.hadoop.hbase.util.HBaseConfTool hbase.rootdir


####129. HBase Tools and Utilities END####




130. Region Management
130.2. Merge
hbase org.apache.hadoop.hbase.util.Merge <tablename> <region1> <region2>






69.7.4. StoreFile (HFile)
hbase hfile
hbase org.apache.hadoop.hbase.io.hfile.HFile


hfile           Store file analyzer


HBase中表ns_monitor:t_monitor_memory的Region分区与HBase在HDFS中的路径/hbase/data/ns_monitor/t_monitor_memory/相对应。


#to view the content of the file by HDFS file
hbase hfile -v -f /hbase/data/ns_monitor/t_monitor_memory/e15236a1387c7148dfde23022884cde3/monitor_memory/e1f274ed5c6c44f0984ac95de30fa880
hbase hfile -b -v -f /hbase/data/ns_monitor/t_monitor_memory/e15236a1387c7148dfde23022884cde3/monitor_memory/e1f274ed5c6c44f0984ac95de30fa880


hbase hfile -m -v -f /hbase/data/ns_monitor/t_monitor_memory/e15236a1387c7148dfde23022884cde3/monitor_memory/e1f274ed5c6c44f0984ac95de30fa880
hbase hfile -p -v -f /hbase/data/ns_monitor/t_monitor_memory/e15236a1387c7148dfde23022884cde3/monitor_memory/e1f274ed5c6c44f0984ac95de30fa880


#to view the content of the file by HBase region name
hbase hfile -p -v -r ns_monitor:t_monitor_memory,,1463673066515.07ded346fc2ee8ee07fdcd04eb644243.
hbase hfile -s -v -r ns_monitor:t_monitor_memory,,1463673066515.07ded346fc2ee8ee07fdcd04eb644243.


#to view the content by the RowKey of a record
hbase hfile -p -v -w "2001-01-01|192.168.16.123|mem_cache"
输出Scanned kv count -> 0


-b,--printblocks         Print block index meta data
-e,--printkey            Print keys


-f <arg>                 File to scan. Pass full-path
-h,--printblockheaders   Print block headers for each block.


(useful)-m,--printmeta           Print meta data of file
(useful)-p,--printkv             Print key/value pairs
 
-r,--region <arg>        Region to scan. Pass region name
-s,--stats               Print statistics


-v,--verbose             Verbose output
-w <arg>                 Seek to this row and print all the kvs for this row only
 


69.7.7. Compaction
A StoreFile is a facade of HFile. In terms of compaction, use of StoreFile seems to have prevailed in the past.
A Store is the same thing as a ColumnFamily. StoreFiles are related to a Store, or ColumnFamily.