Re:[SQL] Syntax "case when" doesn't be supported in JOIN

A workaround is diffcult. You should consider merging this PR into your Spark. "wangshuang [via Apache Spark Developers List]" wroted at 2017-07-13 18:43: I'm trying to execute hive sql on spark sql (Also on spark thriftserver), For optimiz

Re: Faster Spark on ORC with Apache ORC

Awesome, Dong Joon, It's a great improvement. Looking forward its merge. Dong Joon Hyun 于2017年7月12日周三 上午6:53写道： > Hi, All. > > > > Since Apache Spark 2.2 vote passed successfully last week, > > I think it’s a good time for me to ask your opinions again about the > following PR. > > > > https:

Re: Crowdsourced triage Scapegoat compiler plugin warnings

Possibly, but it will take some degree of sophistication and understanding of Spark to evaluate these. I'd want committers or at least Spark developers to look at this rather than users. On Thu, Jul 13, 2017, 15:00 Jorge Sánchez wrote: > It might be worth sharing this with the user list, there m

Re: Crowdsourced triage Scapegoat compiler plugin warnings

It might be worth sharing this with the user list, there must be people willing to collaborate who are not on the dev list. 2017-07-13 10:00 GMT+01:00 Sean Owen : > I don't think everything needs to be triaged. There are a ton of useful > changes that have been identified. I think you could just

Re: [SQL] Syntax "case when" doesn't be supported in JOIN

Just move the case expression into an underlying select clause. On Thu, Jul 13, 2017 at 3:10 PM, Chang Chen wrote: > Hi Wenchen > > Yes. We also find this error is caused by Rand. However, this is classic > way to solve data skew in Hive. Is there any equivalent way in Spark? > > Thanks > Chang

Re: [SQL] Syntax "case when" doesn't be supported in JOIN

Hi Wenchen Yes. We also find this error is caused by Rand. However, this is classic way to solve data skew in Hive. Is there any equivalent way in Spark? Thanks Chang On Thu, Jul 13, 2017 at 8:25 PM, Wenchen Fan wrote: > It’s not about case when, but about rand(). Non-deterministic expression

Re: [SQL] Syntax "case when" doesn't be supported in JOIN

It’s not about case when, but about rand(). Non-deterministic expressions are not allowed in join condition. > On 13 Jul 2017, at 6:43 PM, wangshuang wrote: > > I'm trying to execute hive sql on spark sql (Also on spark thriftserver), For > optimizing data skew, we use "case when" to handle nul

[SQL] Syntax "case when" doesn't be supported in JOIN

I'm trying to execute hive sql on spark sql (Also on spark thriftserver), For optimizing data skew, we use "case when" to handle null. Simple sql as following: SELECT a.col1 FROM tbl1 a LEFT OUTER JOIN tbl2 b ON * CASE WHEN a.col2 IS NULL TNEN c

Re: Crowdsourced triage Scapegoat compiler plugin warnings

I don't think everything needs to be triaged. There are a ton of useful changes that have been identified. I think you could just pick some warning types where they've all been triaged and go fix them. On Thu, Jul 13, 2017 at 9:16 AM Hyukjin Kwon wrote: > Hi all, > > > Another gentle ping for he

Re: Crowdsourced triage Scapegoat compiler plugin warnings

I'm happy to help out in the places I'm more familiar with some starting next week. I suspect things probably just fell to the wayside for some folks during the 2.2.0 release. On Thu, Jul 13, 2017 at 1:16 AM, Hyukjin Kwon wrote: > Hi all, > > > Another gentle ping for help. > > Probably, let me

Re: Crowdsourced triage Scapegoat compiler plugin warnings

Hi all, Another gentle ping for help. Probably, let me open up a JIRA and proceed this after a couple of weeks if no one is going to do this although I hope someone takes this. Thanks. 2017-06-18 2:16 GMT+09:00 Sean Owen : > Looks like a whole lot of the results have been analyzed. I suspect