Chinaunix首页 | 论坛 | 博客
  • 博客访问: 363143
  • 博文数量: 162
  • 博客积分: 0
  • 博客等级: 民兵
  • 技术积分: 1501
  • 用 户 组: 普通用户
  • 注册时间: 2016-10-21 19:45
文章分类
文章存档

2018年(1)

2017年(101)

2016年(60)

分类: HADOOP

2017-01-07 12:29:21

1.启动hadoop, start-all.sh 启动hadoop
2. copy local example data to hdfs
    bin/hadoop dfs -copyFromLocal /opt/tmp /user/hduser/gutenberg
3.开始运行
  bin/hadoop jar contrib/streaming/hadoop-*streaming*.jar -file /home/hduser/mapper.py -mapper /home/hduser/mapper.py  -file /home/hduser/reducer.py -reducer /home/hduser/reducer.py  -input /user/hduser/gutenberg/* -output /user/hduser/gutenberg-output
4. 查看输出内容
 bin/hadoop dfs -cat /user/hduser/gutenberg-output/part-00000

hadoop dfs -copyFromLocal tmp ~/tmp

hadoop jar $HADOOP_HOME/contrib/streaming/hadoop-*streaming*.jar -file ~/work/hadoop/mapper.py -mapper ~/work/hadoop/mapper.py -file ~/work/hadoop/reducer.py -reducer ~/work/hadoop/reducer.py -input ~/tmp -output ~/tmp-output

阅读(1713) | 评论(0) | 转发(0) |
给主人留下些什么吧!~~