StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POSqoop export error - cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist
primarykey
Id
14870139
data
AcceptedAnswerId
0
AnswerCount
1
ClosedDate
CommentCount
0
CommunityOwnedDate
CreationDate
2013-02-14T07:51:55.870
FavoriteCount
0
LastActivityDate
2013-02-15T06:23:49.060
LastEditDate
2013-02-15T06:23:49.060
LastEditorUserId
1936853
OwnerUserId
1936853
ParentId
0
PostTypeId
1
Score
0
ViewCount
4168
LastEditorDisplayName
text
Body
I am developing a java program. The java program exports data from hive to mysql. First, I write the code <pre><code>ProcessBuilder pb = new ProcessBuilder("sqoop-export", "export", "--connect", "jdbc:mysql://localhost/mydb", "--hadoop-home", "/home/yoonhok/development/hadoop-1.1.1", "--table", "mytable", "--export-dir", "/user/hive/warehouse/tbl_2", "--username", "yoonhok", "--password", "1234"); try { Process p = pb.start(); if (p.waitFor() != 0) { System.out.println("Error: sqoop-export failed."); return false; } } catch (IOException e) { e.printStackTrace(); } catch (InterruptedException e) { e.printStackTrace(); } </code></pre> It works perfectly. But I learned a new way of using sqoop in java. Sqoop doesn't support client api yet. So I added sqoop lib and just write Sqoop.run() Second, I write the code again with new way. <pre><code>String[] str = {"export", "--connect", "jdbc:mysql://localhost/mydb", "--hadoop-home", "/home/yoonhok/development/hadoop-1.1.1", "--table", "mytable", "--export-dir", "/user/hive/warehouse/tbl_2", "--username", "yoonhok", "--password", "1234" }; if (Sqoop.runTool(str) == 1) { System.out.println("Error: sqoop-export failed."); return false; } </code></pre> But It doesn't running. I got error...... <pre><code>13/02/14 16:17:09 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 13/02/14 17:43:12 WARN sqoop.ConnFactory: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration. 13/02/14 16:17:09 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 13/02/14 16:17:09 INFO tool.CodeGenTool: Beginning code generation 13/02/14 16:17:09 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tbl_2` AS t LIMIT 1 13/02/14 16:17:09 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tbl_2` AS t LIMIT 1 13/02/14 16:17:09 INFO orm.CompilationManager: HADOOP_HOME is /home/yoonhok/development/hadoop-1.1.1 Note: /tmp/sqoop-yoonhok/compile/45dd1a113123726796a4ed4ce10c9110/tbl_2.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 13/02/14 16:17:10 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-yoonhok/compile/45dd1a113123726796a4ed4ce10c9110/tbl_2.jar 13/02/14 16:17:10 INFO mapreduce.ExportJobBase: Beginning export of tbl_2 13/02/14 16:17:10 WARN mapreduce.ExportJobBase: Input path file:/user/hive/warehouse/tbl_2 does not exist 13/02/14 16:17:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 13/02/14 16:17:11 INFO mapred.JobClient: Cleaning up the staging area file:/tmp/hadoop-yoonhok/mapred/staging/yoonhok314601126/.staging/job_local_0001 13/02/14 16:17:11 ERROR security.UserGroupInformation: PriviledgedActionException as:yoonhok cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/user/hive/warehouse/tbl_2 13/02/14 16:17:11 ERROR tool.ExportTool: Encountered IOException running export job: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/user/hive/warehouse/tbl_2 </code></pre> I saw <code>$SQOOP_CONF_DIR has not been set in the environment.</code> so I added <blockquote> SQOOP_CONF_DIR=/home/yoonhok/development/sqoop-1.4.2.bin__hadoop-1.0.0/conf </blockquote> in the <blockquote> /etc/environment </blockquote> And try again, but Error... <pre><code>13/02/14 16:17:09 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 13/02/14 16:17:09 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 13/02/14 16:17:09 INFO tool.CodeGenTool: Beginning code generation 13/02/14 16:17:09 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tbl_2` AS t LIMIT 1 13/02/14 16:17:09 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tbl_2` AS t LIMIT 1 13/02/14 16:17:09 INFO orm.CompilationManager: HADOOP_HOME is /home/yoonhok/development/hadoop-1.1.1 Note: /tmp/sqoop-yoonhok/compile/45dd1a113123726796a4ed4ce10c9110/tbl_2.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 13/02/14 16:17:10 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-yoonhok/compile/45dd1a113123726796a4ed4ce10c9110/tbl_2.jar 13/02/14 16:17:10 INFO mapreduce.ExportJobBase: Beginning export of tbl_2 13/02/14 16:17:10 WARN mapreduce.ExportJobBase: Input path file:/user/hive/warehouse/tbl_2 does not exist 13/02/14 16:17:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 13/02/14 16:17:11 INFO mapred.JobClient: Cleaning up the staging area file:/tmp/hadoop-yoonhok/mapred/staging/yoonhok314601126/.staging/job_local_0001 13/02/14 16:17:11 ERROR security.UserGroupInformation: PriviledgedActionException as:yoonhok cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/user/hive/warehouse/tbl_2 13/02/14 16:17:11 ERROR tool.ExportTool: Encountered IOException running export job: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/user/hive/warehouse/tbl_2 </code></pre> I think that Export-dir is problem. I use "/user/hive/warehouse/tbl_2". And When I run "hadoop fs -ls /user/hive/warehouse/", the table "tbl_2" exist. I think that "Input path does not exist: file:/user/hive/warehouse/tbl_2" is not ok. "Input path does not exist: hdfs:/user/hive/warehouse/tbl_2" is ok. But I don't know how can I fix it. <hr> Ok just before I got a hint. And I edited 'export-dir' <pre><code>--export-dir hdfs://localhost:9000/user/hive/warehouse/tbl_2 </code></pre> But... It's error... T.T <pre><code>13/02/15 15:17:20 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 13/02/15 15:17:20 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 13/02/15 15:17:20 INFO tool.CodeGenTool: Beginning code generation 13/02/15 15:17:20 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tbl_2` AS t LIMIT 1 13/02/15 15:17:20 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tbl_2` AS t LIMIT 1 13/02/15 15:17:20 INFO orm.CompilationManager: HADOOP_HOME is /home/yoonhok/development/hadoop-1.1.1/libexec/.. Note: /tmp/sqoop-yoonhok/compile/697590ee9b90c022fb8518b8a6f1d86b/tbl_2.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 13/02/15 15:17:22 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-yoonhok/compile/697590ee9b90c022fb8518b8a6f1d86b/tbl_2.jar 13/02/15 15:17:22 INFO mapreduce.ExportJobBase: Beginning export of tbl_2 13/02/15 15:17:22 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 13/02/15 15:17:23 INFO input.FileInputFormat: Total input paths to process : 1 13/02/15 15:17:23 INFO input.FileInputFormat: Total input paths to process : 1 13/02/15 15:17:23 INFO mapred.JobClient: Cleaning up the staging area file:/tmp/hadoop-yoonhok/mapred/staging/yoonhok922915382/.staging/job_local_0001 13/02/15 15:17:23 ERROR security.UserGroupInformation: PriviledgedActionException as:yoonhok cause:java.io.FileNotFoundException: File /user/hive/warehouse/tbl_2/000000_0 does not exist. 13/02/15 15:17:23 ERROR tool.ExportTool: Encountered IOException running export job: java.io.FileNotFoundException: File /user/hive/warehouse/tbl_2/000000_0 does not exist. </code></pre> When I checked hdfs, <pre><code>hadoop fs -ls /user/hive/warehouse/tbl_2 </code></pre> or <pre><code>hadoop fs -ls hdfs://localhost:9000/user/hive/warehouse/tbl_2 </code></pre> the file exist. <blockquote> -rw-r--r-- 1 yoonhok supergroup 14029022 2013-02-15 12:16 /user/hive/warehouse/tbl_2/000000_0 </blockquote> I try in the shell command in terminal <pre><code>sqoop-export --connect jdbc:mysql://localhost/detector --table tbl_2 --export-dir hdfs://localhost:9000/user/hive/warehouse/tbl_2 --username yoonhok --password 1234 </code></pre> It's work. What's problem? I don't know. Could you help me?
Tags
<hadoop><export><hive><sqoop>
Title
Sqoop export error - cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. This table or related slice is empty.
PostTypePostTypeId
1. PTQuestion
UserLastEditorUserId
1. USyoonhok
UserOwnerUserId
1. USyoonhok
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. PL
 singulars
 LinkTypeLinkTypeId
 LTLinked
PostsAcceptedAnswerId
1. This table or related slice is empty.
PostsParentIdCreationDate
1. PO
 singulars
 PostTypePostTypeId
 PTAnswer
VotesPostIdCreationDate
1. This table or related slice is empty.
CommentsPostId
1. This table or related slice is empty.

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.