Re: [DISCUSS] Differentiate Spark without Spark Connect from Spark Connect

+1 for ClassicSent from my iPhoneOn Jul 21, 2024, at 10:15 AM, Ruifeng Zheng wrote:+1 for 'Classic'On Sun, Jul 21, 2024 at 8:03 AM Xiao Li wrote:Classic is much better than Legacy. : ) Hyukjin Kwon 于2024年7月18日周四 16:58写道：Hi all,I noticed that we need t

Re: [VOTE] SPARK-46122: Set spark.sql.legacy.createHiveTableByDefault to false

+1 Sent from my iPhoneOn Apr 30, 2024, at 3:23 PM, DB Tsai wrote:+1 On Apr 29, 2024, at 8:01 PM, Wenchen Fan wrote:To add more color:Spark data source table and Hive Serde table are both stored in the Hive metastore and keep the data files in the table directory. The only difference is they hav

Re: Welcome two new Apache Spark committers

Congratulations!Sent from my iPhoneOn Aug 7, 2023, at 11:16 AM, Yuming Wang wrote:Congratulations!On Mon, Aug 7, 2023 at 11:11 AM Kent Yao wrote:Congrats! Peter and Xiduo! Cheng Pan 于2023年8月7日周一 11:01写道： > > Congratulations! Peter and Xiduo! > > Thanks, > Ch

Re: Random expr in join key not support

> For that, you can add a table subquery and do it in the select list. Do you mean something like this: select * from t1 join (select floor(random()*9) + id as x from t2) m on t1.id = m.x ? Yes, that works. But that raise another question: theses two queries seem semantically equivalent, yet we

Re: [DISCUSS] SPIP: FunctionCatalog

Hi, Thanks for Ryan and Wenchen for leading this. I’d like to add my two cents here. In production environments, the function catalog might be used by multiple systems, such as Spark, Presto and Hive. Is it possible that this function catalog is designed with as an unified function catalog in

Re: Welcoming some new committers and PMC members

Congratulations! Sent from my iPhone > On Sep 10, 2019, at 9:19 AM, Jeff Zhang wrote: > > Congratulations! > > Saisai Shao 于2019年9月10日周二 上午9:16写道： >> Congratulations! >> >> Jungtaek Lim 于2019年9月9日周一 下午6:11写道： >>> Congratulations! Well deserved! >>> On Tue, Sep 10, 2019 at 9:51 AM John

Re: Welcoming three new committers

Congratulations! -- Ye Xianjin Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Wednesday, February 4, 2015 at 6:34 AM, Matei Zaharia wrote: > Hi all, > > The PMC recently voted to add three new committers: Cheng Lian, Joseph > Bradley and Sean Owen. All three hav

Re: [VOTE] Release Apache Spark 1.2.1 (RC2)

Sean, the MQRRStreamSuite is also failed for me on Mac OS X, Though I don’t have time to invest that. -- Ye Xianjin Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Wednesday, January 28, 2015 at 9:17 PM, Sean Owen wrote: > +1 (nonbinding). I verified that all the hash / sign

Re: thank you for reviewing our patches

On Saturday, September 27, 2014 at 4:50 AM, Nicholas Chammas wrote: > > Spark is the first (and currently only) open source project I contribute > regularly to. My first several PRs against the project, as simple as they > were, were definitely patches that I “grew up on” Yeah, me too. I th

Re: spark_classpath in core/pom.xml and yarn/pom.xml

used any more. But I don't think it's worthy to file a JIRA for such small change. Maybe put it into other related JIRA. It's a pity that your pr already got merged. -- Ye Xianjin Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Friday, September 26, 2014 at 6:29

Re: spark_classpath in core/pom.xml and yarn/porm.xml

7;t set SPARK_CLASSPATH env variable. And looked into the SparkConf.scala, If user actually set extraClassPath, the SparkConf will throw SparkException. -- Ye Xianjin Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday, September 23, 2014 at 12:56 AM, Ye Xianjin wrote: > Hi: > I

spark_classpath in core/pom.xml and yarn/porm.xml

Hi: I notice the scalatest-maven-plugin set SPARK_CLASSPATH environment variable for testing. But in the SparkConf.scala, this is deprecated in Spark 1.0+. So what this variable for? should we just remove this variable? -- Ye Xianjin Sent with Sparrow (http

Re: about spark assembly jar

ve no comments. -- Ye Xianjin Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday, September 2, 2014 at 4:58 PM, Sean Owen wrote: > No, usually you unit-test your changes during development. That > doesn't require the assembly. Eventually you may wish to test some >

Re: [VOTE] Release Apache Spark 1.1.0 (RC2)

We just used CDH 4.7 for our production cluster. And I believe we won't use CDH 5 in the next year. Sent from my iPhone > On 2014年8月29日, at 14:39, Matei Zaharia wrote: > > Personally I'd actually consider putting CDH4 back if there are still users > on it. It's always better to be inclusive,

Re: Tests failed after assembling the latest code from github

@Sean Owen, Thanks for your advice. There are still some failing tests on my laptop. I will work on this issue(file move) as soon as I figure out other test related issues. -- Ye Xianjin Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday, April 15, 2014 at 2:41 PM, Sean Owen

Re: Tests failed after assembling the latest code from github

TestUtils.scala to first copy the file to dest then delete the original file. The tests go smoothly. Should I issue an jira about this problem? Then I can send a pr on Github. -- Ye Xianjin Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday, April 15, 2014 at 3:43 AM, Ye Xianjin wrote

Re: Tests failed after assembling the latest code from github

the output is also the same.(The result in my first post). -- Ye Xianjin Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday, April 15, 2014 at 3:14 AM, Aaron Davidson wrote: > This may have something to do with running the tests on a Mac, as there is > a lot of File/URI/UR

Re: Tests failed after assembling the latest code from github

Thank you for your reply. After building the assembly jar, the repl test still failed. The error output is same as I post before. -- Ye Xianjin Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday, April 15, 2014 at 1:39 AM, Michael Armbrust wrote: > I believe you may need

Tests failed after assembling the latest code from github

tent is export SBT_OPTS="-XX:+CMSClassUnloadingEnabled -XX:PermSize=5120M -XX:MaxPermSize=10240M" The test hung after the test failed in the ReplSuite. I have to Ctr-c to close the test. Thank you for you advice. -- Ye Xianjin Sent with Sparrow (http://www.sparrowmailapp.com/?sig)