Databases
StackOverflow2013
Postsdbo
PONot able to fetch data from HDFS using URL protocol handler
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PONot able to fetch data from HDFS using URL protocol handler
    text
    copied!<p>I have a problem with running the following piece of program in java(I am a beginner in java).Here the program uses the HDFS sepecific URLstreamhandlerfactory to use the appropriate protocol handler to access the HDFS. In eclipse , it does not show any error. I have put hadoop-common-2.2.0.jar in build path.</p> <pre><code>package org.hdfs.prog; //cc URLCat Displays files from a Hadoop filesystem on standard output using a //URLStreamHandler import java.io.InputStream; import java.net.URL; import org.apache.hadoop.fs.FsUrlStreamHandlerFactory; import org.apache.hadoop.io.IOUtils; public class URLCat { static { URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory()); } public static void main(String[] args) throws Exception { InputStream in = null; try { in = new URL(args[0]).openStream(); IOUtils.copyBytes(in, System.out, 4096, false); } finally { IOUtils.closeStream(in); } } } </code></pre> <p>But when I am running it, I am getting class not found error, as below. </p> <p>Well this clearly says its not finding one class "org.apache.commons.logging.LogFactory" during run time . To overcome this I downloaded the jar file containing the package "org.apache.commons.logging". Then again I ran the code again another class not found error. </p> <p>Is there any solution which will tell me in advance what are the run time dependency jar files I will require ? Please help me out.</p> <pre><code>Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at org.apache.hadoop.conf.Configuration.&lt;clinit&gt;(Configuration.java:165) at org.apache.hadoop.fs.FsUrlStreamHandlerFactory.&lt;init&gt;(FsUrlStreamHandlerFactory.java:54) at org.hdfs.prog.URLCat.&lt;clinit&gt;(URLCat.java:14) Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) ... 3 more Could not find the main class: org.hdfs.prog.URLCat. Program will exit. </code></pre>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload