StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>I found that I needed to add this to the sample app to get hadoop to know what jar my class files are in.</p> <pre><code>diff --git a/src/org/myorg/WordCount.java b/src/org/myorg/WordCount.java index 912311a..8cc1b93 100644 --- a/src/org/myorg/WordCount.java +++ b/src/org/myorg/WordCount.java @@ -43,7 +43,8 @@ public class WordCount { public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); Job job = new Job(conf, "wordcount"); + job.setJarByClass(WordCount.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(IntWritable.class); </code></pre> <p>I'm not sure if this is new for hadoop or not, but setJarByClass will tell hadoop to use an entire jar based on a single class that is contained in that jar file. The jar must still be in your classpath. This is the command that I ran:</p> <pre><code>hadoop jar wordcount.jar org.myorg.WordCount /usr/$USER/wordcount/input /usr/$USER/wordcount/output </code></pre> <p>And I'd previously copied the sample files into the HDFS using this command:</p> <pre><code>hadoop dfs -copyFromLocal input/file01 /usr/$USER/wordcount/input/file01 hadoop dfs -copyFromLocal input/file02 /usr/$USER/wordcount/input/file02 hadoop dfs -ls /usr/$USER/wordcount/input </code></pre> <p>where input/file01:</p> <pre><code>Hello World Bye World </code></pre> <p>and input/file02:</p> <pre><code>Hello Hadoop Goodbye Hadoop </code></pre> <p>I put up a <a href="https://github.com/tednaleid/hadoop-wordcount">github repo</a> with instructions on what I was able to get working.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload