Databases
StackOverflow2013
Postsdbo
POHow to get Kerberos instead of delegation token in Hadoop mapReduce?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to get Kerberos instead of delegation token in Hadoop mapReduce?
    primarykey
    data
    text
    <p>I'm a Java user, when submitting a job to Hadoop mapReduce, it uses Kerberos to authenticate for Hadoop, and upon success there's the delegation token created and passed with the job submission to Hadoop instead of the kerberos ticket (for security reason as stated by Hadoop). Now the job is running as me, but the job itself needs to use Kerberos to send request to other services outside Hadoop. Now I don't have kerberos TGT on Hadoop and I can't get the service ticket. </p> <p>Is there anyway I can pass the Kerberos ticket with the job? (I know it might be dangerous since we don't want to pass the secret around), JobConf could pass the string to string pairs to Hadoop, but I have to convert the TGT to a json string and revert it during job running?</p> <p>Or is it possible to use the delegation token reform TGT?</p> <p>I tried to google it but not much information, anyone could help? Thank you.</p> <p><em><strong></em>**Editted:<em>*</em>*</strong></p> <p>Looks like there's no easy way of doing this without passing the TGT to Hadoop, so I am going to try the following method by passing the TGT as string via job config map to Hadoop (String only), and convert the string back to TGT object when the job runs in Hadoop. The concern is I am going to pass the credentials over the network, which is not a best practice and one of the very reasons Hadoop didn't pass Kerberos around for security. If I could re-use the reformed TGT passed to Hadoop to get the service tickets, I will try to encrypt the TGT string as much as possible to avoid security issues.</p> <p>So before starting a job in the local machine, the code would be like:</p> <pre><code>import sun.security.krb5.Credentials; Credentials tgt = Credentials.acquireTGTFromCache(null, null); // Make sure kinit is done before this String tgtStr = tgt.convertToJsonString(); //Need to implement this Job job = new Job("Test"); JobConf jobConf = job.getJobConf(); jobConf.set("tgtStr", tgtStr); job.addTask(Test.class, "run", null); job.submit(); job.waitForCompletion(true); </code></pre> <p>Then the function in the job for Hadoop to run would be like:</p> <pre><code>Configuration conf = TaskContext.get().getConfiguration(); String tgtStr = conf.get("tgtStr"); Credentials tgt = reformTGTFromString(tgtStr);//Need to implement this Credentials serviceTicket = Credentials.acquireServiceCreds(servicePrincipal, tgt); //This is to get any service ticket </code></pre> <p>So I need to implement two function to stream TGT object (Credentials.class) to string and then reform it back to object.</p> <p>Anyone knows a better solution for this? Thanks.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USDrawyan
    1. USDrawyan
    plurals
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POHow to get Kerberos instead of delegation token in Hadoop mapReduce?
      1. USDrawyan
      1. VTFavorite
    2. VO
      singulars
      1. POHow to get Kerberos instead of delegation token in Hadoop mapReduce?
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POHow to get Kerberos instead of delegation token in Hadoop mapReduce?
      1. USArup Malakar
      1. VTFavorite
    1. COBy disassembling the Credentials fields and convert them to Strings using Base64 encoder, form a JSON string and pass it to Hadoop using config map or distributed cache suggested by RVM, and then reform the Credentials object in the job running on Hadoop, I can get back the Kerberos TGT and successfully get any service tickets using it. So this method works, and the only thing here needs to be very cautious is the encryption of the keys that are passed over network.
      singulars
      1. POHow to get Kerberos instead of delegation token in Hadoop mapReduce?
      1. USDrawyan
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload