Databases
StackOverflow2013
Postsdbo
POhadoop reduce task running even after telling on command line as -D mapred.reduce.tasks=0
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POhadoop reduce task running even after telling on command line as -D mapred.reduce.tasks=0
    primarykey
    data
    text
    <p>I have a<code>MapReduce</code> program as </p> <pre><code>public static class MapClass extends MapReduceBase implements Mapper&lt;Text, Text, IntWritable, IntWritable&gt; { private final static IntWritable uno = new IntWritable(1); private IntWritable citationCount = new IntWritable(); public void map(Text key, Text value, OutputCollector&lt;IntWritable, IntWritable&gt; output, Reporter reporter) throws IOException { citationCount.set(Integer.parseInt(value.toString())); output.collect(citationCount, uno); } } public static class Reduce extends MapReduceBase implements Reducer&lt;IntWritable, IntWritable, IntWritable, IntWritable&gt; { public void reduce(IntWritable key, Iterator&lt;IntWritable&gt; values, OutputCollector&lt;IntWritable, IntWritable&gt; output, Reporter reporter) throws IOException { int count = 0; while (values.hasNext()) { count += values.next().get(); } output.collect(key, new IntWritable(count)); } } </code></pre> <p>I want to run <strong>only</strong> <code>map</code> task, where output should be of form <code>&lt;number, 1&gt;</code> When I run this from command-line I say </p> <pre><code>$ hadoop jar Hadoop-programs.jar com/hadoop/patent/CitationHistogram input output -Dmapred.reduce.tasks=0 </code></pre> <p>But this is what I see on command-line output</p> <pre><code>12/07/30 06:13:14 INFO mapred.JobClient: map 50% reduce 0% 12/07/30 06:13:23 INFO mapred.JobClient: map 58% reduce 0% 12/07/30 06:13:26 INFO mapred.JobClient: map 60% reduce 8% 12/07/30 06:13:29 INFO mapred.JobClient: map 68% reduce 8% 12/07/30 06:13:32 INFO mapred.JobClient: map 76% reduce 8% 12/07/30 06:13:35 INFO mapred.JobClient: map 85% reduce 16% 12/07/30 06:13:38 INFO mapred.JobClient: map 93% reduce 16% 12/07/30 06:13:41 INFO mapred.JobClient: map 98% reduce 16% 12/07/30 06:13:44 INFO mapred.JobClient: map 100% reduce 16% 12/07/30 06:13:55 INFO mapred.JobClient: map 100% reduce 69% 12/07/30 06:13:58 INFO mapred.JobClient: map 100% reduce 78% 12/07/30 06:14:01 INFO mapred.JobClient: map 100% reduce 94% 12/07/30 06:14:08 INFO mapred.JobClient: map 100% reduce 100% </code></pre> <p>When I see the output of job, I see entries like </p> <pre><code>1 2 13 2 24 1 29 1 31 2 42 3 6796 7 6799 1 6806 1 6815 1 6824 2 </code></pre> <p>Which means data is getting aggregated</p> <p>How can I not run reducer at all?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USdaydreamer
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload