StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>Actually these concepts are somewhat different and common names are misleading.</p> <p>In functional programming (where Python borrowed these functions):</p> <ul> <li><code>map</code> applies some function to all elements of a list and returns a new list</li> <li><code>reduce</code> applies some function to aggregate all values in some list to get a single value.</li> </ul> <p>In distributed computations MapReduce:</p> <ul> <li>we are always working with key-value pairs (well, just pairs)</li> <li><strong>mapper</strong> takes a list of pairs and produces another list of pairs ("key" of input loses its semantics in this context)</li> <li><strong>reducer</strong> gets a key and list of values corresponding to this key (from mapper output) and produces some list of keys and values (single place where "key" has key semantics is the reducer input/mapper output: values are grouped by key before passing to reducer)</li> <li>you may also have partitioner and combiner here :)</li> </ul> <p>Note that neither mapper always produce one output pair for each input pair nor reducer always reduces every (key, list of values) to exactly one output pair. Mapper and reducer can output whatever they want. For example mapper can be used to filter pairs - in this case it produces output pair for some input pairs and ignores other. It is not also uncommon to yield more than one pair for each mapper/reducer input pair (or for some of them).</p> <p>But in most cases MapReduce can work in similar or almost similar way as <code>reduce(reduce_function, map(map_function, list))</code> - mapper usually does some computation for each input, and reducer usually aggregates a list of values in some way. For any <code>map_function</code> and <code>reduce_function</code>, it is possible to express this in MapReduce, but not vice versa.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload