Difference in behavior for Spark 3.0 vs Spark 3.1 "create database "

Hi Spark Team When creating a database via Spark 3.0 on Hive 1) spark.sql("create database test location '/user/hive'"). It creates the database location on hdfs . As expected 2) When running the same command on 3.1 the database is created on the local file system by default. I have to prefix w

Re: Apache Spark Jenkins Infra 2022

Thank you, Yikun, Shane, and DB. Dongjoon On Sun, Jan 9, 2022 at 10:20 PM DB Tsai wrote: > Thank you, Dongjoon for driving the build infra. > > DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1 > > On Jan 9, 2022, at 6:38 PM, shane knapp ☠ wrote: > > > apache spark jenkins lives on!

[VOTE] Release Spark 3.2.1 (RC1)

Please vote on releasing the following candidate as Apache Spark version 3.2.1. The vote is open until Jan. 13th at 12 PM PST (8 PM UTC) and passes if a majority +1 PMC votes are cast, with a minimum of 3 + 1 votes. [ ] +1 Release this package as Apache Spark 3.2.1 [ ] -1 Do not release this pack

Re: Difference in behavior for Spark 3.0 vs Spark 3.1 "create database "

Hi Pralabh, If it helps, it is probably related to this change https://github.com/apache/spark/pull/28527 Regards On Mon, Jan 10, 2022 at 10:42 AM Pralabh Kumar wrote: > Hi Spark Team > > When creating a database via Spark 3.0 on Hive > > 1) spark.sql("create database test location '/user/hive