Re: Run Python User Defined Functions / code in Spark with Scala Codebase

Hello Jayant, Thanks for great OSS Contribution :) On Thu, Jul 12, 2018 at 1:36 PM, Jayant Shekhar wrote: > Hello Chetan, > > Sorry missed replying earlier. You can find some sample code here : > > http://sparkflows.readthedocs.io/en/latest/user-guide/ > python/pipe-python.html > > We will cont

Stale PR update and review request

Hi all, I was checking https://spark-prs.appspot.com/users who has PRs more then 10. viirya 13 mgaido91 12 wangyum 12 maropu

Re: Asking for reviewing PRs regarding structured streaming

Bump. I got couple of review comments from contributors including soft LGTM, but still haven't got any (non code style) review from committers, so technically haven't have any progress to be merged. I'm planning to work on adding new feature as well, but it's not easy for me to concentrate on some

Re: [VOTE] SPARK 2.3.2 (RC3)

Hi Sean, I just did a clean build with mvn/sbt on 2.3.2, I didn't meet the errors you pasted here. I'm not sure how it happens. Sean Owen 于2018年7月16日周一 上午6:30写道： > Looks good to me, with the following caveats. > > First see the discussion on > https://issues.apache.org/jira/browse/SPARK-24813 ;

Re: [VOTE] SPARK 2.3.2 (RC3)

Looks good to me, with the following caveats. First see the discussion on https://issues.apache.org/jira/browse/SPARK-24813 ; the flaky HiveExternalCatalogVersionsSuite will probably fail all the time right now. That's not a regression and is a test-only issue, so don't think it must block the rel

Re: Cleaning Spark releases from mirrors, and the flakiness of HiveExternalCatalogVersionsSuite

+1, this was indeed a problem in the past. On Sun, 15 Jul 2018, 22:56 Reynold Xin, wrote: > Makes sense. Thanks for looking into this. > > On Sun, Jul 15, 2018 at 1:51 PM Sean Owen wrote: > >> Yesterday I cleaned out old Spark releases from the mirror system -- >> we're supposed to only keep th

Re: Cleaning Spark releases from mirrors, and the flakiness of HiveExternalCatalogVersionsSuite

Makes sense. Thanks for looking into this. On Sun, Jul 15, 2018 at 1:51 PM Sean Owen wrote: > Yesterday I cleaned out old Spark releases from the mirror system -- we're > supposed to only keep the latest release from active branches out on > mirrors. (All releases are available from the Apache a

Cleaning Spark releases from mirrors, and the flakiness of HiveExternalCatalogVersionsSuite

Yesterday I cleaned out old Spark releases from the mirror system -- we're supposed to only keep the latest release from active branches out on mirrors. (All releases are available from the Apache archive site.) Having done so I realized quickly that the HiveExternalCatalogVersionsSuite relies on

Re: [VOTE] SPARK 2.3.2 (RC3)

+1. The Spark 2.3 regressions I'm aware of are all fixed. On Sun, Jul 15, 2018 at 4:09 PM Saisai Shao wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.3.2. > > The vote is open until July 20 PST and passes if a majority +1 PMC votes > are cast, with a minimum

Re: Pyspark access to scala/java libraries

If you want to see some examples in a library shows a way to do it - https://github.com/sparklingpandas/sparklingml and high performance spark also talks about it. On Sun, Jul 15, 2018, 11:57 AM <0xf0f...@protonmail.com.invalid> wrote: > Check > https://stackoverflow.com/questions/31684842/callin

Re: Pyspark access to scala/java libraries

Check https://stackoverflow.com/questions/31684842/calling-java-scala-function-from-a-task ​Sent with ProtonMail Secure Email.​ ‐‐‐ Original Message ‐‐‐ On July 15, 2018 8:01 AM, Mohit Jaggi wrote: > Trying again…anyone know how to make this work? > > > On Jul 9, 2018, at 3:45 PM, Mo

[VOTE] SPARK 2.3.2 (RC3)

Please vote on releasing the following candidate as Apache Spark version 2.3.2. The vote is open until July 20 PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 2.3.2 [ ] -1 Do not release this package because ... To lea