Note that there are some explanatory texts on larger screens.

plurals
  1. POWhat is wrong with my boto elastic mapreduce jar jobflow parameters?
    primarykey
    data
    text
    <p>I am using the boto library to create a job flow in Amazons Elastic MapReduce Webservice (EMR). The following code should create a step:</p> <pre><code>step2 = JarStep(name='Find similiar items', jar='s3n://recommendertest/mahout-core/mahout-core-0.5-SNAPSHOT.jar', main_class='org.apache.mahout.cf.taste.hadoop.similarity.item.ItemSimilarityJob', step_args=['s3n://bucket/output/' + run_id + '/aggregate_watched/', 's3n://bucket/output/' + run_id + '/similiar_items/', 'SIMILARITY_PEARSON_CORRELATION' ]) </code></pre> <p>When I run the job flow, it always fails throwing this error:</p> <pre><code>java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/JobContext at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at org.apache.hadoop.util.RunJar.main(RunJar.java:148) at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.JobContext </code></pre> <p>This is the line in the EMR logs invoking the java code:</p> <pre><code>2011-01-24T22:18:54.491Z INFO Executing /usr/lib/jvm/java-6-sun/bin/java \ -cp /home/hadoop/conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/home/hadoop:/home/hadoop \ /hadoop-0.18-core.jar:/home/hadoop/hadoop-0.18-tools.jar:/home/hadoop/lib/*:/home/hadoop/lib/jetty-ext/* \ -Xmx1000m \ -Dhadoop.log.dir=/mnt/var/log/hadoop/steps/3 \ -Dhadoop.log.file=syslog \ -Dhadoop.home.dir=/home/hadoop \ -Dhadoop.id.str=hadoop \ -Dhadoop.root.logger=INFO,DRFA \ -Djava.io.tmpdir=/mnt/var/lib/hadoop/steps/3/tmp \ -Djava.library.path=/home/hadoop/lib/native/Linux-i386-32 \ org.apache.hadoop.mapred.JobShell \ /mnt/var/lib/hadoop/steps/3/mahout-core-0.5-SNAPSHOT.jar \ org.apache.mahout.cf.taste.hadoop.similarity.item.ItemSimilarityJob \ s3n://..../output/job_2011-01-24_23:09:29/aggregate_watched/ \ s3n://..../output/job_2011-01-24_23:09:29/similiar_items/ \ SIMILARITY_PEARSON_CORRELATION </code></pre> <p>What is wrong with the parameters? The java class definition can be found here:</p> <p><a href="https://hudson.apache.org/hudson/job/Mahout-Quality/javadoc/org/apache/mahout/cf/taste/hadoop/similarity/item/ItemSimilarityJob.html" rel="nofollow">https://hudson.apache.org/hudson/job/Mahout-Quality/javadoc/org/apache/mahout/cf/taste/hadoop/similarity/item/ItemSimilarityJob.html</a></p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload