Re: Filter cannot be pushed via a Join

Hi Xiao, Just report this with JIRA SPARK-28103. https://issues.apache.org/jira/browse/SPARK-28103 Thanks and Regards, William On Wed, 19 Jun 2019 at 1:35 AM, Xiao Li wrote: > Hi, William, > > Thanks for reporting it. Could you open a JIRA? > > Cheers, > > Xiao > >

Re: Filter cannot be pushed via a Join

BTW, I noticed a workaround is creating a custom rule to remove 'empty local relation' from a union table. However, I am not 100% sure if it is the right approach. On Tue, Jun 18, 2019 at 11:53 PM William Wong wrote: > Dear all, > > I am not sure if it is something expected o

Re: Filter cannot be pushed via a Join

l("SELECT * FROM partitioned_table_1 t1, table2 t2 WHERE t1.id = t2.id AND t1.id IN ('a','b','c','d')").queryExecution.optimizedPlan.children(0).constraints res44: org.apache.spark.sql.catalyst.expressions.ExpressionSet = Set(isnotnull(id#0), id#0 IN (

Re: Preserving cache name and storage level upon table refresh

to make the review difficult. It also have been reverted. Would you mind to review again and let me know any issues I should continue to work on ? Thanks and regards, William On Tue, Apr 16, 2019 at 12:21 AM William Wong wrote: > Hi Sean and @gatorsmile <https://github.com/gato

Re: Preserving cache name and storage level upon table refresh

Hi Sean and @gatorsmile <https://github.com/gatorsmile>, Thanks a lot for your previous review. I updated those test ( https://github.com/apache/spark/pull/24221) accordingly. May I know if you can help reviewing them again? Best regards, William On Wed, Apr 3, 2019 at 1:03 AM Willia

Re: Antlr plugin for sql/catalyst project

Hi Sean, I just submitted a PR for updating the develop-tools.html. ( https://github.com/apache/spark-website/pull/195). May i know if you may help review it? Many thanks for your help. Best regards, William On Mon, Apr 15, 2019 at 7:04 AM William Wong wrote: > I built the spark with bu

Re: Antlr plugin for sql/catalyst project

on-specific and > tied to a release. > > On Sun, Apr 14, 2019 at 12:00 PM William Wong > wrote: > > > > Hi Sean, > > > > I would like to open a PR for updating the documentation. However, i > cannot share any file for 'http://spark.apache.org/developer-tools.ht

Re: Antlr plugin for sql/catalyst project

t 11:58 PM William Wong wrote: > Hi Sean, > > I tried the button, but antlr4 source was not generate as expected. I > checked the IntelliJ log and found some error message like: > > *2019-04-14 16:05:24,796 [ 314609] INFO - #org.jetbrains.ide

Re: Antlr plugin for sql/catalyst project

ther docs > for IntelliJ users. > > On Sun, Apr 14, 2019 at 4:24 AM William Wong > wrote: > > > > Dear all, > > > > I tried to follow the guide at ' > http://spark.apache.org/developer-tools.html' to setup an IntelliJ > project for Spark. However, th

Antlr plugin for sql/catalyst project

Dear all, I tried to follow the guide at 'http://spark.apache.org/developer-tools.html' to setup an IntelliJ project for Spark. However, the project was failed to build. It was due to missing classes generated via antlr on sql/catalyst project. I would like to enhance the document to hint other n

Re: [build system] jenkins temporarily unreachable, fixing now

Hi, I could not access Jenkins via url https://amplab.cs.berkeley.edu/ anymore. May I know if the reverse proxy is still having issue? Many thanks Best regards, William On Wed, 10 Apr 2019 at 2:34 AM, shane knapp wrote: > the host serving the reverse proxy to jenkins is wedged and is being >

Preserving cache name and storage level upon table refresh

Dear Spark developers, We noticed that cache name could be changed upon table refreshing. It is because CatalogImpl.refreshTable would first uncached and then recache (lazily) without first preserving cache name (and its storage level). IMHO, it is not what a user would expect. I submitted a JIRA