Re: Release Manager's official `branch-3.0` Assessment?

I don't have a dog in this race, but: Would it be OK to ship 3.0 with some release notes and/or prominent documentation calling out this issue, and then fixing it in 3.0.1? On Sat, Mar 28, 2020 at 8:45 PM Jungtaek Lim wrote: > I'd say SPARK-31257 as open blocker, because the change in upcoming S

Re: Release Manager's official `branch-3.0` Assessment?

I'd say SPARK-31257 as open blocker, because the change in upcoming Spark 3.0 made the create table be ambiguous, and once it's shipped it will be harder to correct again. On Sun, Mar 29, 2020 at 4:53 AM Reynold Xin wrote: > Let's start cutting RC next week. > > > On Sat, Mar 28, 2020 at 11:51 A

Re: Release Manager's official `branch-3.0` Assessment?

Let's start cutting RC next week. On Sat, Mar 28, 2020 at 11:51 AM, Sean Owen < sro...@gmail.com > wrote: > > I'm also curious - there no open blockers for 3.0 but I know a few are > still floating around open to revert changes. What is the status there? > From my field of view I'm not aware of

Re: Release Manager's official `branch-3.0` Assessment?

I'm also curious - there no open blockers for 3.0 but I know a few are still floating around open to revert changes. What is the status there? >From my field of view I'm not aware of other blocking issues. On Fri, Mar 27, 2020 at 10:56 PM Jungtaek Lim wrote: > Now the end of March is just around

Beautiful Spark Code

You will be please to learn that Mr. Mathew Powers have seen to my needs and answers all my questions. He has seen to all my needs. Mr Powers has shut me up !!! Mr Powers has made Google search stackoverflow and u...@spark.apache.org redundant. That is all you guys and girl had to do , point m

Re: BUG: spark.readStream .schema(staticSchema) not receiving schema information

Thanks for the tip! But if the first thing you come across Is somebody using the trim function to strip away spaces in /etc/hostnames like so from : 127.0.0.1 hostname local To 127.0.0.1hostnamelocal Then there is a log error message showing the outcome of unnecessarily using the trim functio

Re: BUG: spark.readStream .schema(staticSchema) not receiving schema information

So the schema is limited to holding only the DEFINITION of schema. For example as you say the columns, I.e. first column User:Int 2nd column String:password. Not location of source I.e. csv file with or without header. SQL DB tables. I am pleased for once I am wrong about being another bug, and

Apache Spark Serverless pool and preemptive execution support

Hi everyone, I came across Databricks serverless analytics on Spark and Serverless pools. I was wondering if the Apache Spark also supports these pools and if the Apache Spark re

Re: BUG: spark.readStream .schema(staticSchema) not receiving schema information

Very kind of you. On Sat, 28 Mar 2020, 15:24 Russell Spitzer, wrote: > This is probably more of a question for the user support list, but I > believe I understand the issue. > > Schema inside of spark refers to the structure of the output rows, for > example the schema for a particular dataframe

Re: BUG: spark.readStream .schema(staticSchema) not receiving schema information

This is probably more of a question for the user support list, but I believe I understand the issue. Schema inside of spark refers to the structure of the output rows, for example the schema for a particular dataframe could be (User: Int, Password: String) - Two Columns the first is User of type i