Re: Spark Packaging Jenkins

2019-01-04 Thread shane knapp
https://issues.apache.org/jira/browse/SPARK-26537 On Fri, Jan 4, 2019 at 11:31 AM shane knapp wrote: > this may push in to early next week... these builds were set up before my > time, and i'm currently unraveling how they all work before pushing a > commit to fix stuff. > > nothing like some c

Re: Spark Packaging Jenkins

2019-01-04 Thread shane knapp
this may push in to early next week... these builds were set up before my time, and i'm currently unraveling how they all work before pushing a commit to fix stuff. nothing like some code archaeology to make my friday more exciting! :) shane On Fri, Jan 4, 2019 at 11:08 AM Dongjoon Hyun wrote

Re: Spark Packaging Jenkins

2019-01-04 Thread Dongjoon Hyun
Thank you, Shane! Bests, Dongjoon. On Fri, Jan 4, 2019 at 10:50 AM shane knapp wrote: > yeah, i'll get on that today. thanks for the heads up. > > On Fri, Jan 4, 2019 at 10:46 AM Dongjoon Hyun > wrote: > >> Hi, All >> >> As a part of release process, we need to check Packaging/Compile/Test >>

Re: Spark Packaging Jenkins

2019-01-04 Thread shane knapp
yeah, i'll get on that today. thanks for the heads up. On Fri, Jan 4, 2019 at 10:46 AM Dongjoon Hyun wrote: > Hi, All > > As a part of release process, we need to check Packaging/Compile/Test > Jenkins status. > > http://spark.apache.org/release-process.html > > 1. Spark Packaging: > https://am

Spark Packaging Jenkins

2019-01-04 Thread Dongjoon Hyun
Hi, All As a part of release process, we need to check Packaging/Compile/Test Jenkins status. http://spark.apache.org/release-process.html 1. Spark Packaging: https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/ 2. Spark QA Compile: https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Q

Re: Spark History UI + Keycloak Integration

2019-01-04 Thread Marcelo Vanzin
On Fri, Jan 4, 2019 at 3:25 AM G, Ajay (Nokia - IN/Bangalore) wrote: ... > Added session handler for all context - > contextHandler.setSessionHandler(new SessionHandler()) ... > Keycloak authentication seems to work, Is this the right approach ? If it is > fine I can submit a PR. I don't reme

Re: [DISCUSS] Handling correctness/data loss jiras

2019-01-04 Thread Reynold Xin
Committers, When you merge tickets fixing correctness bugs, please make sure you tag the tickets with "correctness" label. I've found multiple tickets today that didn't do that. On Fri, Aug 17, 2018 at 7:11 AM, Tom Graves < tgraves...@yahoo.com.invalid > wrote: > > Since we haven't heard any

Re: Remove non-Tungsten mode in Spark 3?

2019-01-04 Thread Sean Owen
OK, maybe leave in tungsten for 3.0. I did a quick check, and removing StaticMemoryManager saves a few hundred lines. It's used in MemoryStore tests internally though, and not a trivial change to remove it. It's also used directly in HashedRelation. It could still be worth removing it as a user-fac

Logging exception of custom spark app

2019-01-04 Thread Alessandro Liparoti
Hi everyone, I have an application that uses spark to perform some computation. It can be used both in a spark-shell or in a spark-submit. I want to log all exceptions throw by my code inside a file in order to have some detailed info when user have an error. I tried with this Thread.currentThre

Spark History UI + Keycloak Integration

2019-01-04 Thread G, Ajay (Nokia - IN/Bangalore)
Hello, We were trying to enable spark-history UI authentication through keycloak. >From the spark documentation we found out that we can use javax filters to enable the UI authentication. Keycloak already provides a java keycloak-servlet-filter-adapter which can be used. I have added the follo