StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POHadoop Map output IOException when emitting subclass of a class defined in Configuration as an ouput
text
Body
copied!<p>I have 3 simple classes:</p> <pre><code>public abstract class Container implements WritableComparable<Container> {} //empty public class WeightedEdge extends Container { ... } public class NodeWeightContainer extends Container { ... } </code></pre> <p>The Map phase was configured as such</p> <pre><code>JobConf createGraphPConf = new JobConf(new Configuration()); Job job = new Job(createGraphPConf); ... createGraphPConf.setMapOutputValueClass(Container.class); </code></pre> <p>However I am receiving this error:</p> <pre><code>java.io.IOException: Type mismatch in value from map: expected org.hadoop.test.data.util.Container, recieved org.hadoop.test.data.WeightedEdge at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1018) at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:591) at org.hadoop.test.map.CreateGPMap.map(CreateGPMap.java:33) at org.hadoop.test.map.CreateGPMap.map(CreateGPMap.java:19) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:435) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210) </code></pre> <p>Why I can't return a subclass of a class that was defined in the configuration? Is there a way around it? The problem is that my Map phase has to emit two distinct object types.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload