Re: Keytab, Proxy User & Principal

I just wanted to bump this to see if anyone has any opinions on this? On Fri, Feb 28, 2020 at 3:20 PM Lars Francke wrote: > Hi, > > I understand that we forbid specifying "principal" & "proxy user" at the > same time because the current logic would just s

Keytab, Proxy User & Principal

Hi, I understand that we forbid specifying "principal" & "proxy user" at the same time because the current logic would just stage the keytab and the proxy user could then use that to gain full access circumventing any security. But we have a use-case for Livy where a different semantic would be g

Re: Apache Training contribution for Spark - Feedback welcome

On Mon, Jul 29, 2019 at 2:46 PM Sean Owen wrote: > TL;DR is: take the below as feedback to consider, and proceed as you > see fit. Nobody's suggesting you can't do this. > > On Mon, Jul 29, 2019 at 2:58 AM Lars Francke > wrote: > > The way I read your point is t

Re: Apache Training contribution for Spark - Feedback welcome

ul 27, 2019 at 12:01 AM Sean Owen wrote: > On Fri, Jul 26, 2019 at 4:01 PM Lars Francke > wrote: > > I understand why it might be seen that way and we need to make sure to > point out that we have no intention of becoming "The official Apache Spark > training" becaus

Re: Apache Training contribution for Spark - Feedback welcome

"Data Ingestion" training]) where we can mix and match from the content we have I'd have to dig into the original discuss threads in the incubator to find more but I hope this helps a bit? Cheers, Lars > > > On Fri, Jul 26, 2019 at 9:00 AM Lars Francke &g

Apache Training contribution for Spark - Feedback welcome

Hi Spark community, you may or may not have heard of a new-ish (February 2019) project at Apache: Apache Training (incubating). We aim to develop training material about various projects inside and outside the ASF: < http://training.apache.org/> One of our users wants to contribute material on Sp

Aggregated stats on data locality

Hi, I'm looking for a way to get an aggregated view of data locality across a spark job/stage. I was sure that this existed but I can't find it now. Basically a quick summary on how many tasks were PROCESS_LOCAL vs, NODE_LOCAL etc. Is there a way to do this or a Jira to track (I couldn't find an

Re: SPIP: Executor Plugin (SPARK-24918)

+1 On Fri, Aug 31, 2018 at 8:11 AM, Reynold Xin wrote: > I actually had a similar use case a while ago, but not entirely the same. > In my use case, Spark is already up, but I want to make sure all existing > (and new) executors run some specific code. Can we update the API to > support that? I

Re: JIRA: Wrong dates from imported JIRAs

blems with JIRA > import/export and the benefit here doesn't seem huge (this issue only > impacts people that are searching for the oldest JIRAs across all projects, > which I think is pretty uncommon). Just my two cents. > > - Josh > > On Wed, Dec 16, 2015 at 8:45 A

Re: JIRA: Wrong dates from imported JIRAs

Any other opinions on this? On Fri, Dec 11, 2015 at 9:54 AM, Lars Francke wrote: > That's a good point. I assume there's always a small risk but it's at > least the documented way from Atlassian to change the creation date so I'd > hope it should be okay. I'd

Re: JIRA: Wrong dates from imported JIRAs

gs? > > Most of those are done and I don't know how often people search JIRAs by > date across projects. > > On Fri, Dec 11, 2015 at 3:40 PM, Lars Francke > wrote: > >> Hi, >> >> I've been digging into JIRA a bit and found a couple of old issues (~25

JIRA: Wrong dates from imported JIRAs

Hi, I've been digging into JIRA a bit and found a couple of old issues (~250) and I just assume that they are all from the old JIRA. Here's one example: Old: New: created

Re: Understanding code/closure shipment to Spark workers‏

Hi Arijit, my understanding is the following: RDD actions will at some point call the runJob method of a SparkContext That runJob method calls the clean method which in turn calls ClosureCleaner.clean which removes unneeded stuff from closures and also checks whether they are serializable. The ne

Re: YARN Maven build questions

RK/Contributing+to+Spark> On Fri, Feb 28, 2014 at 6:41 PM, Lars Francke wrote: > Hey, > > so currently it doesn't work because of > <https://github.com/apache/spark/pull/6#issuecomment-36343187> > > IntelliJ reports a lot of warnings with default settings and I have

Re: YARN Maven build questions

ersion=2.2.0 -Pyarn clean package -DskipTests > > Tom > > > > On Friday, February 28, 2014 6:14 AM, Lars Francke > wrote: > > Hey, > > I'm trying to dig into Spark's code but am running into a couple of problems. > > 1) The yarn-common directory is no

YARN Maven build questions

Hey, I'm trying to dig into Spark's code but am running into a couple of problems. 1) The yarn-common directory is not included in the Maven build causing things to fail because the dependency is missing. If I see the history correct it used to be a Maven module but is not anymore. 2) When I try