Re: [DISCUSS] Upgrade Hive compile time dependency to 4.0

Not an easy task, I guess, but I'm totally for it too. The issue SPARK-49910 is related to this. El mar, 11 mar 2025 a las 23:06, Mich Talebzadeh () escribió: > Yes I am all for it, as I use Hive with Oracle as its metastore > extensively. >

Re: [DISCUSS] Upgrade Hive compile time dependency to 4.0

Agreed. Hive upgrade is more time consuming as it involves backing up Hive schema on your metastore and then running Hive provided upgrade schema scripts against Hive schema that could be problematic,but needs to be done one way or another. HTH Dr Mich Talebzadeh, Architect | Data Science | Finan

Re: [DISCUSS] SPARK-51318: Remove `jar` files from Apache Spark repository and disable affected tests

There is a difference between technical debt and legal issue. ASF may request to pull out release that does not meet ASF policy (and having tests is not ASF policy). IMO, SPARK-51318 should be a blocker for the next release or handled like a blocker. Thank you, Vlad On Mar 10, 2025, at 6:02 P

Re: [VOTE] Retain migration logic of incorrect `spark.databricks.*` configuration in Spark 4.0.x

Russell, Of course, we hear people' voices who aren't having binding votes as well. Personally I think it's more important than committers/PMC members' VOTE this time since we can be biased and be far from user experience. Could you please explicitly cast your vote, like +1 (non-binding)? You se

Re: [VOTE] Retain migration logic of incorrect `spark.databricks.*` configuration in Spark 4.0.x

> > this vote is to allow streaming queries which had been ever run in Spark > 3.5.4 to be upgraded with Spark 4.0.x, "without having to be upgraded with > Spark 3.5.5+ in prior". In the history of Apache Spark, have we ever required users to upgrade to the next maintenance release before moving

Re: [VOTE] Retain migration logic of incorrect `spark.databricks.*` configuration in Spark 4.0.x

I'm just a lurker and aspiring contributor, but as a Spark user upgrading twice is very confusing and would cause many or most users to fail to upgrade successfully to Spark 4 on a first go. That seems like a very bad user experience. I thought it was worthwhile stating this out loud. Russell On

Re: [DISCUSS] SPIP: Constraints in DSv2

Thanks to everyone who provided feedback and participated in the discussion! I made some tweaks to the proposal and submitted a PR with the DSv2 API changes: https://github.com/apache/spark/pull/50253 If there are no objections/feedback, I will start a vote later this week. - Anton пт, 14 лют. 2