Re: [DISCUSS] Amend the commiter guidelines on the subject of -1s & how we expect PR discussion to be treated.

2020-07-24 Thread Tom Graves
+1 Tom On Tuesday, July 21, 2020, 03:35:18 PM CDT, Holden Karau wrote: Hi Spark Developers, There has been a rather active discussion regarding the specific vetoes that occured during Spark 3. From that I believe we are now mostly in agreement that it would be best to clarify our rule

Python xmlrunner being used?

2020-07-24 Thread Driesprong, Fokko
Hi all, Does anyone know if the xmlrunner package is still being used? We're working on enforcing some static code analysis checks on the Python codebase, and the imports of the xmlrunner generates quite some noise: https://github.com/apache/spark/pull/29121 It looks like the entry point for a l

Re: Python xmlrunner being used?

2020-07-24 Thread Driesprong, Fokko
I found this ticket: https://issues.apache.org/jira/browse/SPARK-7021 Is anybody actually using this? Cheers, Fokko Op vr 24 jul. 2020 om 16:27 schreef Driesprong, Fokko : > Hi all, > > Does anyone know if the xmlrunner package is still being used? > > We're working on enforcing some static cod

Re: InterpretedUnsafeProjection - error in getElementSize

2020-07-24 Thread Wenchen Fan
Can you create a JIRA ticket? There are many people happy to help to fix it. On Tue, Jul 21, 2020 at 9:49 PM Janda Martin wrote: > Hi, > I think that I found error in > InterpretedUnsafeProjection::getElementSize. This method differs from > similar implementation in GenerateUnsafeProjection. >

Re: Starting work on last Scala 2.13 updates

2020-07-24 Thread Sean Owen
Status update - we should have Scala 2.13 compiling, with the exception of the REPL. Looks like 99% or so of tests pass too, but the remaining ones might be hard to debug. I haven't looked hard yet. https://issues.apache.org/jira/browse/SPARK-25075 is the umbrella. Anyone that's interested can try

Re: Starting work on last Scala 2.13 updates

2020-07-24 Thread Holden Karau
This is awesome progress :) On Fri, Jul 24, 2020 at 8:56 AM Sean Owen wrote: > Status update - we should have Scala 2.13 compiling, with the > exception of the REPL. > Looks like 99% or so of tests pass too, but the remaining ones might > be hard to debug. I haven't looked hard yet. > https://is

Re: Starting work on last Scala 2.13 updates

2020-07-24 Thread Dongjoon Hyun
Thank you so much, Sean! Bests, Dongjoon. On Fri, Jul 24, 2020 at 8:56 AM Sean Owen wrote: > Status update - we should have Scala 2.13 compiling, with the > exception of the REPL. > Looks like 99% or so of tests pass too, but the remaining ones might > be hard to debug. I haven't looked hard y

Re: Process for backports?

2020-07-24 Thread venkat_20
Can you please elaborate why don't backport of features is not allowed? -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Process for backports?

2020-07-24 Thread Sean Owen
This is pretty much standard semantic versioning: https://semver.org/ Maintenance branch releases are for bug fixes; minor releases have new features. Of course at some level it all depends on judgment, too. On Fri, Jul 24, 2020 at 1:08 PM venkat_20 wrote: > > Can you please elaborate why don't b

Re: [DISCUSS] [Spark confs] Making spark.jars conf take precedence over spark default classpath

2020-07-24 Thread Imran Rashid
Hi Nupur, Is what you're trying to do already possible via the spark.{driver,executor}.userClassPathFirst options? https://github.com/apache/spark/blob/b890fdc8df64f1d0b0f78b790d36be883e852b0d/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L853 On Wed, Jul 22, 2020 at 5:50 PM nupu

Re: Python xmlrunner being used?

2020-07-24 Thread Hyukjin Kwon
It's used in Jenkins IIRC 2020년 7월 24일 (금) 오후 11:43, Driesprong, Fokko 님이 작성: > I found this ticket: https://issues.apache.org/jira/browse/SPARK-7021 > > Is anybody actually using this? > > Cheers, Fokko > > Op vr 24 jul. 2020 om 16:27 schreef Driesprong, Fokko >: > >> Hi all, >> >> Does anyone