hadoop 编译native包，hadoop编译native
最近公司让我研究一下hadoop hbase native包
先附上参考地址 http://hadoop.apache.org/docs/r2.5.2/hadoop-project-dist/hadoop-common/NativeLibraries.html
Usage
It is fairly easy to use the native hadoop library:
- Review the components.
- Review the supported platforms.
- Either download a hadoop release, which will include a pre-built version of the native hadoop library, or build your own version of the native hadoop library. Whether you download or build, the name for the library is the same: libhadoop.so
- Install the compression codec development packages (>zlib-1.2, >gzip-1.2): + If you download the library, install one or more development packages - whichever compression codecs you want to use with your deployment. + If you build the library, it is mandatory to install both development packages.
- Check the runtime log files.
上图是官网对native包的作用描述
Build
The native hadoop library is written in ANSI C and is built using the GNU autotools-chain (autoconf, autoheader, automake, autoscan, libtool). This means it should be straight-forward to build the library on any platform with a standards-compliant C compiler and the GNU autotools-chain (see the supported platforms).
The packages you need to install on the target platform are:
- C compiler (e.g. GNU C Compiler)
- GNU Autools Chain: autoconf, automake, libtool
- zlib-development package (stable version >= 1.2.0)
Once you installed the prerequisite packages use the standard hadoop pom.xml file and pass along the native flag to build the native hadoop library:
$ mvn package -Pdist,native -Dskiptests -Dtar
You should see the newly-built library in:
$ hadoop-dist/target/hadoop-2.5.2/lib/native
Please note the following:
- It is mandatory to install both the zlib and gzip development packages on the target platform in order to build the native hadoop library; however, for deployment it is sufficient to install just one package if you wish to use only one codec.
- It is necessary to have the correct 32/64 libraries for zlib, depending on the 32/64 bit jvm for the target platform, in order to build and deploy the native hadoop library.
上图是官网关于编译本地包的步骤
首先就是os了,支持
- RHEL4/Fedora
- Ubuntu
- Gentoo
这几个系统做编译.然后就是环境的设置了,安装jdk1.7 安装maven
还需要安装gcc autoconf,automake,libtool,zlib-devel不然会出现莫名其妙的错误.
之后就是执行命令
mvn package -Pdist,native -Dskiptests -Dtar
或者
mvn package -Pdist,native -Dmaven.test.skip=true -Dtar或者mvn package -Pdist,native -DeskipTests -Dtar -Dmaven.test.failure.ignore=true
这两者区别是 后面一个不进行junit测试预祝大家编译成功.