Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Your mapper maps your inputs to the value that you want to take the average of. So let's say that your input is a text file formatted like</p> <pre><code>ProcessName Time process1 10 process2 20 . . . </code></pre> <p>Then you would need to take each line in your file, split it, grab the second column, and output the value of that column as an <code>IntWritable</code> (or some other <code>Writable</code> numeric type). Since you want to take the average of all times, not grouped by process name or anything, you will have a single fixed key. Thus, your mapper would look something like</p> <pre><code>private IntWritable one = new IntWritable(1); private IntWritable output = new IntWritable(); proctected void map(LongWritable key, Text value, Context context) { String[] fields = value.split("\t"); output.set(Integer.parseInt(fields[1])); context.write(one, output); } </code></pre> <p>Your reducer takes these values, and simply computes the average. This would look something like</p> <pre><code>IntWritable one = new IntWritable(1); DoubleWritable average = new DoubleWritable(); protected void reduce(IntWritable key, Iterable&lt;IntWrtiable&gt; values, Context context) { int sum = 0; int count = 0; for(IntWritable value : values) { sum += value.get(); count++; } average.set(sum / (double) count); context.Write(key, average); } </code></pre> <p>I'm making a lot of assumptions here, about your input format and what not, but they are reasonable assumptions and you should be able to adapt this to suit your exact needs.</p> <blockquote> <p>Will my output always be a text file or is it possible to directly store the average in some sort of a variable?</p> </blockquote> <p>You have a couple of options here. You can post-process the output of the job (written a single file), or, since you're computing a single value, you can store the result in a counter, for example.</p>
    singulars
    1. This table or related slice is empty.
    1. PONeed help in writing Map/Reduce job to find average
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USjason
    1. USjason
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COThanks Jason. Just one more thing. I've set up a hadoop cluster with 1 job tracker and 3 other task trackers. So do I need to have the input file in all task tracker or is it enough if I have it in the job tracker alone? And is there any way of finding out whether the job is distributed evenly to all the slave nodes?
      singulars
      1. PO
      1. USAnon
    2. COHDFS and Hadoop will handle that for you. You can monitor a job by opening port 50030 (that's the *default* port at least) on your Hadoop job tracker node and from there can access your task trackers. Note that since you have exactly one output key from your map job, your reduce task will only run on one node.
      singulars
      1. PO
      1. USjason
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload