StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POPig MR job failing when run by same user who started the cluster
primarykey
Id
13833479
data
AcceptedAnswerId
0
AnswerCount
1
ClosedDate
CommentCount
0
CommunityOwnedDate
CreationDate
2012-12-12T05:38:08.580
FavoriteCount
0
LastActivityDate
2012-12-14T21:08:11.847
LastEditDate
2012-12-12T06:02:17.860
LastEditorUserId
1664708
OwnerUserId
1664708
ParentId
0
PostTypeId
1
Score
0
ViewCount
214
LastEditorDisplayName
text
Body
I am seeing this exception intermittently for some mappers and reducers in my Pig map reduce job. Most of the times it is retried on some other node and the task succeeds. But sometimes all 4 tasks fails and the map reduce job fails. However the interesting thing is the folder jobcache indeed has permissions 700. I dont understand why it is not able to create the folder inside it. <pre><code>Error initializing attempt_201212101828_0396_m_000028_0: java.io.IOException: Failed to set permissions of path: /apollo/env/TrafficAnalyticsHadoop/var/hadoop/mapred/local_data/taskTracker/trafanly/jobcache/job_201212101828_0396 to 0700 at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:682) at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:671) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344) at org.apache.hadoop.mapred.JobLocalizer.createJobDirs(JobLocalizer.java:221) at org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:184) at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1226) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093) at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1201) at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1116) at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2404) at java.lang.Thread.run(Thread.java:662) </code></pre> I am using Hadoop 1.0.1 if that helps. One more thing which i found while searching online was: <a href="https://issues.apache.org/jira/browse/MAPREDUCE-890" rel="nofollow">https://issues.apache.org/jira/browse/MAPREDUCE-890</a>. In my case the user who started the mapred cluster is indeed running the job and that is when it fails. For any other user the job runs just fine. Any help would be appreciated.
Tags
<hadoop><permissions><mapreduce><apache-pig>
Title
Pig MR job failing when run by same user who started the cluster
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. This table or related slice is empty.
PostTypePostTypeId
1. PTQuestion
UserLastEditorUserId
1. USshaurya
UserOwnerUserId
1. USshaurya
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. This table or related slice is empty.
PostsAcceptedAnswerId
1. This table or related slice is empty.
PostsParentIdCreationDate
1. PO
 singulars
 PostTypePostTypeId
 PTAnswer
VotesPostIdCreationDate
1. This table or related slice is empty.
CommentsPostId
1. This table or related slice is empty.

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.