Hadoop-Fedora24搭建Hadoop

1. 安装Java环境并配置好环境变量 /etc/profile末尾追加
    export JAVA_HOME=/opt/java
    export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
    export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOME/bin
2. 新建用户组及用户名 （用于隔离环境）
    groupadd hadoop
    useradd hduser -g hadoop
    (userdel test 删除用户)
3. 为hduser配置ssh
user@ubuntu:~$ su - hduser
hduser@ubuntu:~$ ssh-keygen -t rsa -P ""
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hduser/.ssh/id_rsa):
Created directory '/home/hduser/.ssh'.
Your identification has been saved in /home/hduser/.ssh/id_rsa.
Your public key has been saved in /home/hduser/.ssh/id_rsa.pub.
The key fingerprint is: 9b:82:ea:58:b4:e0:35:d7:ff:19:66:a6:ef:ae:0e:d2
hduser@ubuntu The key's randomart image is: [...snipp...]
hduser@ubuntu:~$

Second, you have to enable SSH access to your local machine with this newly created key.
hduser@ubuntu:~$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
ssh localhost

chmod 700 ~/.ssh
chmod 600 ~/.ssh/*

4. Disabling ipv6(没有配成功）
5. 安装Hadoop
从hadoop官网上下载
$ cd /opt
$ sudo tar xzvf hadoop-1.2.1.tar.gz
$ sudo mv hadoop-1.2.1 hadoop
$ sudo chown -R hduser:hadoop hadoop
6. 修改$HOME/.bashrc
# Set Hadoop-related environment variables
export HADOOP_HOME=/opt/hadoop

# Set JAVA_HOME (we will also configure JAVA_HOME directly for Hadoop later on)
export JAVA_HOME=/opt/java

# Some convenient aliases and functions for running Hadoop-related commands
unalias fs &> /dev/null
alias fs="hadoop fs"
unalias hls &> /dev/null
alias hls="fs -ls"

# If you have LZO compression enabled in your Hadoop cluster and
# compress job outputs with LZOP (not covered in this tutorial):
# Conveniently inspect an LZOP compressed file from the command
# line; run via:
#
# $ lzohead /hdfs/path/to/lzop/compressed/file.lzo
#
# Requires installed 'lzop' command.
#
lzohead () {
    hadoop fs -cat $1 | lzop -dc | head -1000 | less
}

# Add Hadoop bin/ directory to PATH
export PATH=$PATH:$HADOOP_HOME/bin

7. HDFS配置
（1）/opt/hadoop/conf/hadoop-env.sh
# The java implementation to use. Required.
export JAVA_HOME=/opt/java
（2）hadoop.tmp.dir
$ sudo mkdir -p /app/hadoop/tmp
$ sudo chown hduser:hadoop /app/hadoop/tmp
# ...and if you want to tighten up security, chmod from 755 to 750...
$ sudo chmod 750 /app/hadoop/tmp
（3）conf/*-site.xml
配置详见
8.Formatting the HDFS filesystem via the NameNode
/opt/hadoop/bin/hadoop namenode -format
9.启动 start-all.sh
10.关闭 stop-all.sh
11.jps(Java Virtual Machine Process Status Tool) 查看当前运行的java进程
12. wordcount示例