Re: spark with kerberos

2016-10-19 Thread Steve Loughran
On 19 Oct 2016, at 00:18, Michael Segel mailto:msegel_had...@hotmail.com>> wrote: (Sorry sent reply via wrong account.. ) Steve, Kinda hijacking the thread, but I promise its still on topic to OP’s issue.. ;-) Usually you will end up having a local Kerberos set up per cluster. So your machine

Re: spark with kerberos

2016-10-18 Thread Michael Segel
(Sorry sent reply via wrong account.. ) Steve, Kinda hijacking the thread, but I promise its still on topic to OP’s issue.. ;-) Usually you will end up having a local Kerberos set up per cluster. So your machine accounts (hive, yarn, hbase, etc …) are going to be local to the cluster. So you

Re: spark with kerberos

2016-10-18 Thread Steve Loughran
On 17 Oct 2016, at 22:11, Michael Segel mailto:michael_se...@hotmail.com>> wrote: @Steve you are going to have to explain what you mean by ‘turn Kerberos on’. Taken one way… it could mean making cluster B secure and running Kerberos and then you’d have to create some sort of trust between B an

Re: spark with kerberos

2016-10-14 Thread Steve Loughran
On 13 Oct 2016, at 10:50, dbolshak mailto:bolshakov.de...@gmail.com>> wrote: Hello community, We've a challenge and no ideas how to solve it. The problem, Say we have the following environment: 1. `cluster A`, the cluster does not use kerberos and we use it as a source of data, important thin

Re: spark with kerberos

2016-10-13 Thread Saisai Shao
gt; 4. We can turn on/off kerberos on `cluster B`, currently it's turned >>>> off. >>>> 5. Spark app is built on top of RDD and does not depend on spark-sql. >>>> >>>> Does anybody know how to write data using RDD api to remote clust

Re: spark with kerberos

2016-10-13 Thread Denis Bolshakov
on `cluster B`, currently it's turned off. >>> 5. Spark app is built on top of RDD and does not depend on spark-sql. >>> >>> Does anybody know how to write data using RDD api to remote cluster >>> which is >>> running with Kerbe

Re: spark with kerberos

2016-10-13 Thread ayan guha
Does anybody know how to write data using RDD api to remote cluster which >> is >> running with Kerberos? >> >> -- >> //with Best Regards >> --Denis Bolshakov >> e-mail: bolshakov.de

Re: spark with kerberos

2016-10-13 Thread ayan guha
ql. > > Does anybody know how to write data using RDD api to remote cluster which > is > running with Kerberos? > > -- > //with Best Regards > --Denis Bolshakov > e-mail: bolshakov.de...@gmail.com > > > > -- > View this message in context: http://apache-sp

spark with kerberos

2016-10-13 Thread dbolshak
depend on spark-sql. Does anybody know how to write data using RDD api to remote cluster which is running with Kerberos? -- //with Best Regards --Denis Bolshakov e-mail: bolshakov.de...@gmail.com -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-with-k

Spark with kerberos

2016-10-13 Thread Denis Bolshakov
Hello community, We've a challenge and no ideas how to solve it. The problem, Say we have the following environment: 1. `cluster A`, the cluster does not use kerberos and we use it as a source of data, important thing is - we don't manage this cluster. 2. `cluster B`, small cluster where our spa

Re: Error in Spark Executors when trying to read HBase table from Spark with Kerberos enabled

2016-01-18 Thread Vinay Kashyap
Hi Guys, Any help regarding this issue..?? On Wed, Jan 13, 2016 at 6:39 PM, Vinay Kashyap wrote: > Hi all, > > I am using *Spark 1.5.1 in YARN cluster mode in CDH 5.5.* > I am trying to create an RDD by reading HBase table with kerberos enabled. > I am able to launch the spark job to read th

Error in Spark Executors when trying to read HBase table from Spark with Kerberos enabled

2016-01-13 Thread Vinay Kashyap
Hi all, I am using *Spark 1.5.1 in YARN cluster mode in CDH 5.5.* I am trying to create an RDD by reading HBase table with kerberos enabled. I am able to launch the spark job to read the HBase table, but I notice that the executors launched for the job cannot proceed due to an issue with Kerberos