Databases
StackOverflow2013
Postsdbo
POHow to set output writer in MapReduce
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to set output writer in MapReduce
    text
    copied!<p>I'm trying out the mapreduce framework from (<a href="http://code.google.com/p/appengine-mapreduce/" rel="nofollow">http://code.google.com/p/appengine-mapreduce/</a>) and modified the demo application a bit (use the mapreduce.input_readers.DatastoreInputReader instead of mapreduce.input_readers.BlobstoreZipInputReader).</p> <p>I've set up 2 pipeline classes:</p> <pre><code>class IndexPipeline(base_handler.PipelineBase): def run(self): output = yield mapreduce_pipeline.MapreducePipeline( "index", "main.index_map", #added higher up in code "main.index_reduce", #added higher up in code "mapreduce.input_readers.DatastoreInputReader", mapper_params={ "entity_kind": "model.SearchRecords", }, shards=16) yield StoreOutput("Index", output) class StoreOutput(base_handler.PipelineBase): def run(self, mr_type, encoded_key): logging.info("output is %s %s" % (mr_type, str(encoded_key))) if encoded_key: key = db.Key(encoded=encoded_key) m = db.get(key) yield op.db.Put(m) </code></pre> <p>And run it with:</p> <pre><code>pipeline = IndexPipeline() pipeline.start() </code></pre> <p>But I keep getting this error: </p> <pre><code>Handler yielded two: ['a'] , but no output writer is set. </code></pre> <p>I've tried to find somewhere in the <a href="http://code.google.com/p/appengine-mapreduce/source/browse/#svn/trunk/python/src/mapreduce" rel="nofollow">source</a> where to set the output writer but with out success. Only thing I found is that one should set a <code>output_writer_class</code> somewhere.</p> <p>Does anyone know how to set this?</p> <p>On a side note, the <code>encoded_key</code> argument in <code>StoreOutput</code> always seems to be None.</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload