Chinaunix首页 | 论坛 | 博客
  • 博客访问: 269658
  • 博文数量: 41
  • 博客积分: 2560
  • 博客等级: 少校
  • 技术积分: 465
  • 用 户 组: 普通用户
  • 注册时间: 2010-07-01 15:56
文章分类

全部博文(41)

文章存档

2011年(10)

2010年(31)

我的朋友

分类: 服务器与存储

2010-09-27 13:23:52

hadoop安装配置
系统环境
redhat 5.5-x64
所需软件
hadoop-0.21.0.tar.gz
jdk-6u21-linux-x64.bin

部署环境:
namenode:192.168.10.20(hadoop1)
datanode:192.168.10.21(hadoop2)
               192.168.10.22(hadoop3)
               192.168.10.23(hadoop4)
一、安装
1、在namenode:(在datanode上操作和以下相同)
[hadoop@hadoop1 ~]# vi /etc/hosts
192.168.10.20  hadoop1
192.168.10.21  hadoop2
192.168.10.22  hadoop3
192.168.10.23  hadoop4

2、用root登陆,建立hadoop用户
[hadoop@hadoop1 ~]# useradd hadoop
[hadoop@hadoop1 ~]# passwd hadoop
输入******做为密码

3、su - hadoop       #进入hadoop用户目录
[hadoop@hadoop1 ~]# su - hadoop
[hadoop@hadoop1 ~]$ ssh-keygen -t rsa #建立ssh目录,敲回车到底
[hadoop@hadoop1 ~]$ cd .ssh/
[hadoop@hadoop1 .ssh]$ ll
total 20
-rw------- 1 hadoop hadoop 1675 Sep 23 16:19 id_rsa
-rw-r--r-- 1 hadoop hadoop  403 Sep 23 16:19 id_rsa.pub
-rw-r--r-- 1 hadoop hadoop 3136 Sep 24 15:23 known_hosts
[hadoop@hadoop1 .ssh]$ scp -r id_rsa.pub hadoop@hadoop2:/home/hadoop/.ssh/
[hadoop@hadoop1 .ssh]$ ssh hadoop2
[hadoop@hadoop2 .ssh]$ cat id_rsa.pub >>authorized_keys
[hadoop@hadoop2 .ssh]$ chmod 644 authorized_keys
其它datanode机器操作一样

4、jdk-6u21-linux-x64.bin安装
[root@master src]# ./jdk-6u21-linux-x64.bin
[root@master src]# mv jdk1.6.0_21 /usr/local/
[root@master local]# ln -s jdk1.6.0_21 java
5、hadoop-0.21.0.tar.gz安装
[root@master src]# tar -zxvf hadoop-0.21.0.tar.gz
[root@master src]# mv hadoop-0.21.0 /usr/local/
[root@master local]# ln -s hadoop-0.21.0 hadoop
6、修改环境变量
[root@master src]# vi /etc/profile
export JAVA_HOME=/usr/local/java
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=$JAVA_HOME/lib:$JAVA_HOME/jre/bin:$PATH:$HOME/bin
export HADOOP_HOME=/usr/local/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
[root@master src]# java -version
java version "1.6.0_21"
Java(TM) SE Runtime Environment (build 1.6.0_21-b06)
Java HotSpot(TM) 64-Bit Server VM (build 17.0-b16, mixed mode)
[root@master src]# hadoop version
Hadoop 0.21.0
Subversion -r 985326
Compiled by tomwhite on Tue Aug 17 01:02:28 EDT 2010
From source with checksum a1aeb15b4854808d152989ba76f90fac

二、配置
配置文件
1、hadoop-env.sh,core-site.xml,hdfs-site.xml,mapred-site.xml,hadoop-env.sh,masters,slaves
namenode配置
[hadoop@hadoop1 ~]$ cd /usr/local/hadoop/conf/
[hadoop@hadoop1 ~]$ vi hadoop-env.sh (修改java环境变量)
export JAVA_HOME=/usr/local/java
[hadoop@hadoop1 ~]$ vi core-site.xml(hdfs和mapreduce中很普通的I/O设置)




  
     fs.default.name
     hdfs://hadoop1:9000
  


[hadoop@hadoop1 ~]$ vi hdfs-site.xml(HDFS后台程序设置的配置:名称节点,第二名称节点和数据节点)




   
       dfs.replication
       3
   


   
       dfs.name.dir
       /usr/local/hadoop/namenode/
   


   
       hadoop.tmp.dir
       /usr/local/hadoop/tmp/
   



[hadoop@hadoop1 ~]$ vi mapred-site.xml




   
        mapred.job.tracker
        hadoop1:9001
   


   
       mapred.tasktracker.map.tasks.maximum
       4
   


   
       mapred.tasktracker.reduce.tasks.maximum
       4
   



datanode配置 (只需修改hdfs-site.xml)
[hadoop@hadoop2 ~]$ vi hdfs-site.xml




   
       dfs.replication
       3
   


   
       dfs.data.dir
       /home/hadoop/data
   


   
       hadoop.tmp.dir
       /usr/local/hadoop/tmp/
   


[hadoop@hadoop1 conf]$ vi masters
hadoop1
[hadoop@hadoop1 conf]$ vi slaves
hadoop2
hadoop3
hadoop4
[hadoop@hadoop1 ~]$ start-all.sh
[hadoop@hadoop1 ~]$ stop-all.sh








阅读(5656) | 评论(0) | 转发(0) |
0

上一篇:hadoop介绍

下一篇:wget中文乱码

给主人留下些什么吧!~~