Re: Apache Spark 3.3 Release

Thanks Maxim, The code freeze by end of this month would be fine. Not sure about SPARK-36057 since the current state. Thanks On Fri, 4 Mar 2022 at 19:27, Jungtaek Lim wrote: > Thanks Maxim for volunteering to drive the release! I support the plan > (March 15th) to perform a release branch cu

Re: ASF board report draft for August

Hi Matei, nice update Just one question, when you mention “ We are working on Spark 3.2.0 as our next release, with a release candidate likely to come soon. Spark 3.2 includes a new Pandas API for Apache Spark based on the Koalas project” Are you referring to what version of Koala project? 1.8.

Re: Welcoming two new committers

Congratulations Herman and Wenchen. On Tue, Feb 9, 2016 at 10:58 AM, Joseph Bradley wrote: > Congrats & welcome! > > On Mon, Feb 8, 2016 at 12:19 PM, Ram Sriharsha > wrote: > >> great job guys! congrats and welcome! >> >> On Mon, Feb 8, 2016 at 12:05 PM, Amit Chavan wrote: >> >>> Welcome. >>>

Re: SPARK_WORKER_MEMORY in Spark Standalone - conf.getenv vs System.getenv?

Actually answering the first question: Is there a reason to use conf to read SPARK_WORKER_MEMORY not System.getenv as for the other env vars? You can use the properties file to change the amount, System.getenv would be bad when you have for example other things running on the JVM which will cause

Re: Ivy support in Spark vs. sbt

Marcelo I've run this problem once, when I was starting with Spark, like you mentioned. I found out that ivy get messy with diff sbt version. My solution was using a previous compatible version with sbt to not cross with version. Best Igor On Thu, Jun 4, 2015 at 8:08 PM, Eron Wright wrote: >

Re: createDataframe from s3 results in error

Hey there Ignacio Like Reynold said, It's related to your build of Spark, try to not compile with Thrift. Also, try to use this command to see what's the error and link to here. sc.wholeTextFile("s3://my-directory/2015*/ignacio/*") Ps( Are you using boto to connect? Which version?) Igor On

Re: Jira Issues

Thank you guys for the info. Actually was a problem with my id on apache. Rather than need to be logged in to view issues. I'm browsing some issues now. Best Regards Igor Costa www.igorcosta.com www.igorcosta.org On Wed, Mar 25, 2015 at 5:58 PM, Sean Owen

Jira Issues

Hi there Guys. I want to be more collaborative to Spark, but I have two questions. Issues are used in Github or jira Issues? If so on Jira, Is there a way I can get in to see the issues? I've tried to login but no success. I'm PMC from another Apache project, flex.apache.org Best Regards I