Pranav, sorry that I didn't see you already posted the question to the mailing list.
Benchao Li <[email protected]> 于2022年8月14日周日 11:28写道： > Pranav, > > Reposted your mail to the dev mailing list. (It's recommended to reply to > dev mailing list instead of someone's email address) > > > So my question is better phrased as: "Is there a planner rule that > converts a join on 2 Table scans to some kind of table?" Are any examples > available on this? I tried looking for ?examples on the same, however; I > was able to observe pushdowns, but not what I just mentioned. > > I don't know any existing rule does this for now. > You can add your own rule to achieve this, it won't be complex. You may > need to do more work to abstract out the interface to accept the 'Join > Pushdown'. > > > > Pranav Deshpande <[email protected]> 于2022年8月13日周六 01:21写道： > >> HI Benchao, >> Thank you very much for your excellent suggestions! >> >> I believe that I have a better understanding about my problem now. I saw >> the pushdowns and also the usage of multiple conventions in the same plan, >> and was able to implement and play around with the same. >> >> I went through talk [4] as well. >> >> So my question is better phrased as: "Is there a planner rule that >> converts a join on 2 Table scans to some kind of table?" Are any examples >> available on this? I tried looking for examples on the same, however; I was >> able to observe pushdowns, but not what I just mentioned. >> >> This is similar to the ProjectTableScan rule in some sense, it converts a >> project on a TableScan to a ProjectableFilterableTable SCAN. >> >> Thanks & Regards, >> Pranav >> >> On Fri, Aug 5, 2022 at 11:57 PM Benchao Li <[email protected]> wrote: >> >>> Pranav, >>> >>> You can reference Calcite adaptors implementation, such as JDBC >>> Adaptor[1], MongoDB Adaptor[2]. >>> Their implementation allows pushing down operations (RelNode) to the >>> adapter as much as possible, and the left RelNodes will be implemented >>> using Enumerable Convention. >>> >>> We have a Converter[3] concept which makes this possible. The converter >>> node allows multiple Conventions in a single query. This also answers your >>> second question. >>> There is a talk[4] about this, it's very helpful for understanding this >>> concept. >>> >>> [1] >>> https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/adapter/jdbc/JdbcToEnumerableConverter.java >>> [2] >>> https://github.com/apache/calcite/blob/main/mongodb/src/main/java/org/apache/calcite/adapter/mongodb/MongoToEnumerableConverter.java >>> [3] >>> https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/convert/Converter.java >>> [4] >>> https://calcite.apache.org/community/#fast-federated-sql-with-apache-calcite >>> >>> Julian Hyde <[email protected]> 于2022年8月6日周六 02:06写道： >>> >>>> Pranav, >>>> >>>> Please subscribe to this list. You have asked several questions, >>>> received replies, not acknowledged those replies, and asked further >>>> questions. Also, since you are not subscribed, each email you post has to >>>> go through manual moderation. >>>> >>>> Julian >>>> >>>> > On Aug 5, 2022, at 9:38 AM, Pranav Deshpande < >>>> [email protected]> wrote: >>>> > >>>> > Dear Apache Calcite Team, >>>> > I have 2 questions. >>>> > >>>> > --------------------------------- >>>> > 1. >>>> > >>>> > There are plenty of examples on how to push down projects and filters >>>> into >>>> > the leaf nodes (tablescans). >>>> > >>>> > However, I could not find any examples to push down joins to >>>> TableScans (or >>>> > joins+filters+projects etc.) [this is helpful for data federation I >>>> think]. >>>> > >>>> > On the mailing list, many folks are suggesting that I use Drill. >>>> However, >>>> > the purpose of my exercise is to gain knowledge about DBMS and Query >>>> > processing etc. >>>> > >>>> > I tried debugging open source engines that use Calcite (Drill, Druid, >>>> Trino >>>> > etc.) but was completely lost. >>>> > >>>> > Any examples/pointers/guidance around the same would be appreciated. >>>> > Example, pushing down a join with a filter to a DBMS(consider jdbc >>>> > msql etc.) >>>> > >>>> > ------------------------------------- >>>> > 2. >>>> > >>>> > The 2nd question I have is regarding conventions and different DBMS. >>>> The >>>> > cluster has a method to replace the trait convention(Bindable,JDBC >>>> etc.), >>>> > and then we optimize and get the physical plan. >>>> > >>>> > But imagine I have both the MYSQL JDBC convnction and a cassandra >>>> > convention and some user is trying to query both tables. >>>> > >>>> > Something like "SELECT users.username, specialdata.country from >>>> > cassandraDB.user join mysqlDB.specialdata ON users.id >>>> =specialdata.userid" >>>> > >>>> > Now, how will calcite do the optimization here? The planner is not >>>> > accepting 2 different conventions. >>>> > >>>> > Thanks & Regards, >>>> > Pranav >>>> >>>> >>> >>> -- >>> >>> Best, >>> Benchao Li >>> >> > > -- > > Best, > Benchao Li > -- Best, Benchao Li
