yone's feedback, thanks everyone.
From: Steve Loughran [mailto:ste...@hortonworks.com]
Sent: Monday, June 29, 2015 10:32 AM
To: Dave Ariens
Cc: Tim Chen; Marcelo Vanzin; Olivier Girardot; user@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
On 29 Jun
On 29 Jun 2015, at 14:18, Dave Ariens
mailto:dari...@blackberry.com>> wrote:
I'd like to toss out another idea that doesn't involve a complete end-to-end
Kerberos implementation. Essentially, have the driver authenticate to
Kerberos, instantiate a Hadoop file system, and serialize/cache it f
s
Cc: Tim Chen; Olivier Girardot;
user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens
mailto:dari...@blackberry.com>> wrote:
Would there be any way to have the task
setting the
KRB5CCNAME env variable when starting the process. You can check the Hadoop
sources for details. Not sure if there's another way.
From: Marcelo Vanzin
Sent: Friday, June 26, 2015 6:20 PM
To: Dave Ariens
Cc: Tim Chen; Olivier Girardot;
user@spark.apache.org<mailto:user@spark.a
apache.org
> *Subject: *Re: Accessing Kerberos Secured HDFS Resources from Spark on
> Mesos
>
> On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens
> wrote:
>
>> Would there be any way to have the task instances in the slaves call
>> the UGI login with a principal/keytab pro
p
> sources for details. Not sure if there's another way.
>
>
>
>>
>> *From: *Marcelo Vanzin
>> *Sent: *Friday, June 26, 2015 6:20 PM
>> *To: *Dave Ariens
>> *Cc: *Tim Chen; Olivier Girardot; user@spark.apache.org
>> *Subject: *Re: Accessing
when starting the process. You can check the Hadoop
sources for details. Not sure if there's another way.
>
> *From: *Marcelo Vanzin
> *Sent: *Friday, June 26, 2015 6:20 PM
> *To: *Dave Ariens
> *Cc: *Tim Chen; Olivier Girardot; user@spark.apache.org
> *Subject:
6:20 PM
To: Dave Ariens
Cc: Tim Chen; Olivier Girardot; user@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens
mailto:dari...@blackberry.com>> wrote:
Would there be any way to have the task instances in
On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens wrote:
> Would there be any way to have the task instances in the slaves call the
> UGI login with a principal/keytab provided to the driver?
>
That would only work with a very small number of executors. If you have
many login requests in a short per
stances in the slaves call
the UGI login with a principal/keytab provided to the driver?
From: Marcelo Vanzin
Sent: Friday, June 26, 2015 5:28 PM
To: Tim Chen
Cc: Olivier Girardot; Dave Ariens; user@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
On F
On Fri, Jun 26, 2015 at 2:08 PM, Tim Chen wrote:
> Mesos do support running containers as specific users passed to it.
> Thanks for chiming in, what else does YARN do with Kerberos besides keytab
> file and user?
>
The basic things I'd expect from a system to properly support Kerberos
would be:
Mesos do support running containers as specific users passed to it.
Thanks for chiming in, what else does YARN do with Kerberos besides keytab
file and user?
Tim
On Fri, Jun 26, 2015 at 1:20 PM, Marcelo Vanzin wrote:
> On Fri, Jun 26, 2015 at 1:13 PM, Tim Chen wrote:
>
>> So correct me if I'm
iday, June 26, 2015 4:20 PM
To: Tim Chen
Cc: Olivier Girardot; Dave Ariens; user@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
On Fri, Jun 26, 2015 at 1:13 PM, Tim Chen
mailto:t...@mesosphere.io>> wrote:
So correct me if I'm wrong, sounds l
On Fri, Jun 26, 2015 at 1:13 PM, Tim Chen wrote:
> So correct me if I'm wrong, sounds like all you need is a principal user
> name and also a keytab file downloaded right?
>
I'm not familiar with Mesos so don't know what kinds of features it has,
but at the very least it would need to start cont
as tasks are ran.
>>
>>
>>
>> Any help would be appreciated!
>>
>>
>>
>>
>>
>> *From:* Timothy Chen [mailto:t...@mesosphere.io]
>> *Sent:* Friday, June 26, 2015 12:50 PM
>> *To:* Dave Ariens
>> *Cc:* user@spark.apache.o
rom:* Timothy Chen [mailto:t...@mesosphere.io]
> *Sent:* Friday, June 26, 2015 12:50 PM
> *To:* Dave Ariens
> *Cc:* user@spark.apache.org
> *Subject:* Re: Accessing Kerberos Secured HDFS Resources from Spark on
> Mesos
>
>
>
> Hi Dave,
>
>
>
> I don't
ran.
Any help would be appreciated!
From: Timothy Chen [mailto:t...@mesosphere.io]
Sent: Friday, June 26, 2015 12:50 PM
To: Dave Ariens
Cc: user@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
Hi Dave,
I don't understand Keeberos much but if you
Hi Dave,
I don't understand Keeberos much but if you know the exact steps that needs to
happen I can see how we can make that happen with the Spark framework.
Tim
> On Jun 26, 2015, at 8:49 AM, Dave Ariens wrote:
>
> I understand that Kerberos support for accessing Hadoop resources in Spark
18 matches
Mail list logo
