官网下载的2.0版本部署后报错:
hadoop fs -ls
14/09/09 11:33:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
SYSTEM:Centos6.2
1.下载官网hadoop2.5.2 源码
2.cengos系统准备编译所需安装包(Ant,Maven,ProtocolBuffer,findbugs,CMake)
3.替换之前的$HADOOP_HOME/lib/native为新编译的native
4.修改hadoop-env.sh 配置文件
(1) 安装编译工具
yum -y install lzo-devel zlib-devel gcc autoconf automake libtool ant
[root@hdn03 src]# tar zxf apache-maven-3.1.1-bin.tar.gz
[root@hdn03 src]# unzip findbugs-3.0.1.zip
[root@hdn03 src]# tar zxf protobuf-2.5.0.tar.gz
[root@hdn03 protobuf-2.5.0]# ./configure
[root@hdn03 protobuf-2.5.0]# make ; make check ; make install
域名解析问题(看情况设置):
[root@hdn03 hadoop-2.5.2-src]# tail /etc/hosts
23.235.47.215 repo.maven.apache.org
192.87.106.229 archive.apache.org
添加环境变量:
vim /etc/profile
export MAVEN_HOME=/usr/local/src/apache-maven-3.1.1
export FINDBUGS_HOME=/usr/local/src/findbugs-3.0.1
export PATH=$FINDBUGS_HOME/bin:$MAVEN_HOME/bin:$PATH
[root@hdn03 src]# source /etc/profile
(2)解压源码包
[root@hdn03 src]# tar zxf hadoop-2.5.2-src.tar.gz
[root@hdn03 src]# cd hadoop-2.5.2-src
[root@hdn03 hadoop-2.5.2-src]# mvn package -DskipTests -Pdist,native
完成后,替换native文件夹
/usr/local/src/hadoop-2.5.2-src/hadoop-dist/target/hadoop-2.5.2/lib/native/
将以上编译好的文件夹内的所有内容替换掉你原来的32位的hadoop的native文件
现在你可以安心舒服的进行文件系统操作了,再也不会看见讨厌的:
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
修改 hadoop-env.sh 配置文件:
vim hadoop-env.sh
export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=/usr/local/hadoop/lib/native"
export HADOOP_COMMON_LIB_NATIVE_DIR="/usr/local/hadoop/lib/native/"
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/local/hadoop/etc/hadoop"}
无需重启服务,直接在再试,报错消失了 !
[root@hdn03 hadoop-2.5.2-src]# hadoop fs -ls /
Found 1 items
drwxr-xr-x - root supergroup 0 2015-03-09 14:06 /home
阅读(1701) | 评论(0) | 转发(0) |