Databases
StackOverflow2013
Postsdbo
POHadoop Streaming : no such file found error (error in configuring object)
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHadoop Streaming : no such file found error (error in configuring object)
    text
    copied!<p>I run the following command for hadoop streaming</p> <pre><code>hadoop jar contrib/streaming/hadoop-streaming-1.0.3.jar -input test1/testregxml.xml -output testoutput2 -file ~/Programs/CMapper.php -mapper ~/Programs/CMapper.php -jobconf mapred.reduce.tasks=1 </code></pre> <p>The hadoop task failed with following output</p> <pre><code>12/10/26 15:36:56 WARN streaming.StreamJob: -jobconf option is deprecated, please use - D instead. packageJobJar: [/home/HadoopNaman1/Programs/CMapper.php, /tmp/hadoop-HadoopNaman1/hadoop-unjar8119734358986860064/] [] /tmp/streamjob3840254105417044886.jar tmpDir=null 12/10/26 15:36:58 INFO util.NativeCodeLoader: Loaded the native-hadoop library 12/10/26 15:36:58 WARN snappy.LoadSnappy: Snappy native library not loaded 12/10/26 15:36:58 INFO mapred.FileInputFormat: Total input paths to process : 1 12/10/26 15:36:58 INFO streaming.StreamJob: getLocalDirs(): [/tmp/hadoop-HadoopNaman1/mapred/local] 12/10/26 15:36:58 INFO streaming.StreamJob: Running job: job_201209282004_0015 12/10/26 15:36:58 INFO streaming.StreamJob: To kill this job, run: 12/10/26 15:36:58 INFO streaming.StreamJob: /home/HadoopNaman1/hadoop-1.0.3/libexec/../bin/hadoop job -Dmapred.job.tracker=saraswati.tallysolutions.com:9051 -kill job_201209282004_0015 12/10/26 15:36:58 INFO streaming.StreamJob: Tracking URL: http://192.168.1.17:50030/jobdetails.jsp?jobid=job_201209282004_0015 12/10/26 15:36:59 INFO streaming.StreamJob: map 0% reduce 0% 12/10/26 15:37:51 INFO streaming.StreamJob: map 100% reduce 100% 12/10/26 15:37:51 INFO streaming.StreamJob: To kill this job, run: 12/10/26 15:37:51 INFO streaming.StreamJob: /home/HadoopNaman1/hadoop-1.0.3/libexec/../bin/hadoop job -Dmapred.job.tracker=saraswati.tallysolutions.com:9051 -kill job_201209282004_0015 12/10/26 15:37:51 INFO streaming.StreamJob: Tracking URL: http://192.168.1.17:50030/jobdetails.jsp?jobid=job_201209282004_0015 12/10/26 15:37:51 ERROR streaming.StreamJob: Job not successful. Error: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201209282004_0015_m_000001 12/10/26 15:37:51 INFO streaming.StreamJob: killJob... Streaming Command Failed! </code></pre> <p>Here is my jobtracker log</p> <pre><code>2012-10-26 15:37:44,906 INFO org.apache.hadoop.mapred.TaskInProgress: TaskInProgress task_201209282004_0015_m_000001 has failed 4 times. 2012-10-26 15:37:44,906 INFO org.apache.hadoop.mapred.JobInProgress: Aborting job job_201209282004_0015 2012-10-26 15:37:44,906 INFO org.apache.hadoop.mapred.JobInProgress: Killing job 'job_201209282004_0015' 2012-10-26 15:37:44,907 INFO org.apache.hadoop.mapred.JobTracker: Adding task (JOB_CLEANUP) 'attempt_201209282004_0015_m_000002_0' to tip task_201209282004_0015_m_000002, for tracker 'tracker_192.168.1.86:localhost/127.0.0.1:45880' 2012-10-26 15:37:44,907 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201209282004_0015_m_000001_3' 2012-10-26 15:37:46,861 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201209282004_0015_m_000000_3: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:416) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 9 more Caused by: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) ... 14 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 17 more Caused by: java.lang.RuntimeException: configuration exception at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:230) at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66) ... 22 more Caused by: java.io.IOException: Cannot run program "/tmp/hadoop-HadoopNaman1/mapred/local/taskTracker/HadoopNaman1/jobcache/job_201209282004_0015/attempt_201209282004_0015_m_000000_3/work/./CMapper.php": java.io.IOException: error=2, No such file or directory at java.lang.ProcessBuilder.start(ProcessBuilder.java:475) at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:214) ... 23 more Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory at java.lang.UNIXProcess.&lt;init&gt;(UNIXProcess.java:164) at java.lang.ProcessImpl.start(ProcessImpl.java:81) at java.lang.ProcessBuilder.start(ProcessBuilder.java:468) ... 24 more 2012-10-26 15:37:49,866 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201209282004_0015_m_000000_3' 2012-10-26 15:37:50,913 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201209282004_0015_m_000002_0' has completed task_201209282004_0015_m_000002 successfully. 2012-10-26 15:37:50,914 INFO org.apache.hadoop.mapred.JobInProgress$JobSummary: jobId=job_201209282004_0015,submitTime=1351246018628,launchTime=1351246018806,firstMapTaskLaunchTime=1351246028873,firstJobSetupTaskLaunchTime=1351246019863,firstJobCleanupTaskLaunchTime=1351246064906,finishTime=1351246070914,numMaps=2,numSlotsPerMap=1,numReduces=1,numSlotsPerReduce=1,user=HadoopNaman1,queue=default,status=FAILED,mapSlotSeconds=60,reduceSlotsSeconds=0,clusterMapCapacity=12,clusterReduceCapacity=12,jobName=streamjob3840254105417044886.jar 2012-10-26 15:37:50,936 INFO org.apache.hadoop.mapred.JobHistory: Moving file:/home/HadoopNaman1/hadoop-1.0.3/logs/history/job_201209282004_0015_1351246018628_HadoopNaman1_streamjob3840254105417044886.jar to file:/home/HadoopNaman1/hadoop-1.0.3/logs/history/done/version-1/saraswati.tallysolutions.com_1348842869935_/2012/10/26/000000 2012-10-26 15:37:50,942 INFO org.apache.hadoop.mapred.JobHistory: Moving file:/home/HadoopNaman1/hadoop-1.0.3/logs/history/job_201209282004_0015_conf.xml to file:/home/HadoopNaman1/hadoop-1.0.3/logs/history/done/version-1/saraswati.tallysolutions.com_1348842869935_/2012/10/26/000000 2012-10-26 15:37:50,948 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201209282004_0015_m_000002_0' 2012-10-26 15:37:50,948 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201209282004_0015_m_000003_0' 2012-10-26 15:37:51,850 INFO org.apache.hadoop.mapred.JobTracker: Killing job job_201209282004_0015 </code></pre> <p>Please suggest something.The Mapper file is executable and is present in the directory with permissions set as 777.</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload