Chinaunix首页 | 论坛 | 博客
  • 博客访问: 1808389
  • 博文数量: 276
  • 博客积分: 5998
  • 博客等级: 大校
  • 技术积分: 5175
  • 用 户 组: 普通用户
  • 注册时间: 2010-12-24 14:43
文章分类

全部博文(276)

文章存档

2014年(25)

2013年(11)

2012年(69)

2011年(167)

2010年(4)

分类: 大数据

2014-04-03 14:20:18

在Rhadoop上运行mapreduce时报以下错误

>  small.ints = 1:10
> small.ints = to.dfs(1:10)
14/04/03 10:12:17 INFO util.NativeCodeLoader: Loaded the native-hadoop library
14/04/03 10:12:17 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
14/04/03 10:12:17 INFO compress.CodecPool: Got brand-new compressor
> mapreduce(input = small.ints, map = function(k, v) cbind(v, v^2))
packageJobJar: [/tmp/RtmprPiI80/rmr-local-env742a26089c1, /tmp/RtmprPiI80/rmr-global-env742a70b9f272, /tmp/RtmprPiI80/rmr-streaming-map742a597ae933, /export/hadoop/tmp/hadoop-unjar6149670842998439024/] [] /tmp/streamjob99151515370312214.jar tmpDir=null
14/04/03 10:12:33 INFO mapred.FileInputFormat: Total input paths to process : 1
14/04/03 10:12:33 INFO streaming.StreamJob: getLocalDirs(): [/export/hadoop/tmp/mapred/local]
14/04/03 10:12:33 INFO streaming.StreamJob: Running job: job_201404021340_0002
14/04/03 10:12:33 INFO streaming.StreamJob: To kill this job, run:
14/04/03 10:12:33 INFO streaming.StreamJob: /export/hadoop/libexec/../bin/hadoop job  -Dmapred.job.tracker=http://10.14.2.201:9001 -kill job_201404021340_0002
14/04/03 10:12:33 INFO streaming.StreamJob: Tracking URL: http://master.hadoop:50030/jobdetails.jsp?jobid=job_201404021340_0002
14/04/03 10:12:34 INFO streaming.StreamJob:  map 0%  reduce 0%
14/04/03 10:13:03 INFO streaming.StreamJob:  map 100%  reduce 100%
14/04/03 10:13:03 INFO streaming.StreamJob: To kill this job, run:
14/04/03 10:13:03 INFO streaming.StreamJob: /export/hadoop/libexec/../bin/hadoop job  -Dmapred.job.tracker=http://10.14.2.201:9001 -kill job_201404021340_0002
14/04/03 10:13:03 INFO streaming.StreamJob: Tracking URL: http://master.hadoop:50030/jobdetails.jsp?jobid=job_201404021340_0002
14/04/03 10:13:03 ERROR streaming.StreamJob: Job not successful. Error: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201404021340_0002_m_000000
14/04/03 10:13:03 INFO streaming.StreamJob: killJob...
Streaming Command Failed!
Error in mr(map = map, reduce = reduce, combine = combine, vectorized.reduce,  : 
  hadoop streaming failed with error code 1

解决方法:
    出现这种情况有可能是因为只在namenode节点上安装了rhadoop,在所用所有节点上安装rhadoop解决上面的报错


阅读(5254) | 评论(0) | 转发(0) |
给主人留下些什么吧!~~