Hi Jark, Thx a lot for your quick reply and valuable suggestions. For (1): Agree: Since we are in the period of upgrading the new table source api, we really should consider the new interface for the new optimize rule. If the new rule doesn't use the new api, we'll have to upgrade it sooner or later. I have change to use the ability interface for the SupportsAggregatePushDown definition in above proposal.
For (2): Agree: Change to use CallExpression is a better choice, and have resolved this comment in the proposal. For (3): I suggest we first support the JDBC connector, as we don't have Druid connector and ES connector just has sink api at present. But perhaps the biggest question may be whether we should use idea 1 or idea 2 in proposal. What do you think? After we reach the agreement on the proposal, our team can drive to complete this feature. Jark Wu <imj...@gmail.com> 于2020年12月29日周二 下午2:58写道： > Hi Sebastian, > > Thanks for the proposal. I think this is a great improvement for Flink SQL. > I went through the design doc and have following thoughts: > > 1) Flink has deprecated the legacy TableSource in 1.11 and proposed a new > set of DynamicTableSource interfaces. Could you update your proposal to > use the new interfaces? > Follow the existing ability interfaces, e.g. > SupportsFilterPushDown, SupportsProjectionPushDown. > > 2) Personally, I think CallExpression would be a better representation than > separate `FunctionDefinition` and args. Because, it would be easier to know > what's the index and type of the arguments. > > 3) It would be better to list which connectors will be supported in the > plan? > > Best, > Jark > > > On Tue, 29 Dec 2020 at 00:49, Sebastian Liu <liuyang0...@gmail.com> wrote: > > > Hi all, > > > > I'd like to discuss a new feature for the Blink Planner. > > Aggregate operator of Flink SQL is currently fully done at Flink layer. > > With the developing of storage, many downstream storage of Flink SQL has > > the ability to deal with Aggregation operator. > > Pushing down Aggregate to data source layer will improve performance from > > the perspective of the network IO and computation overhead. > > > > I have drafted a design doc for this new feature. > > > > > https://docs.google.com/document/d/1kGwC_h4qBNxF2eMEz6T6arByOB8yilrPLqDN0QBQXW4/edit?usp=sharing > > > > Any comment or discussion is welcome. > > > > -- > > > > *With kind regards > > ------------------------------------------------------------ > > Sebastian Liu 刘洋 > > Institute of Computing Technology, Chinese Academy of Science > > Mobile\WeChat: +86—15201613655 > > E-mail: liuyang0...@gmail.com <liuyang0...@gmail.com> > > QQ: 3239559* > > > -- *With kind regards ------------------------------------------------------------ Sebastian Liu 刘洋 Institute of Computing Technology, Chinese Academy of Science Mobile\WeChat: +86—15201613655 E-mail: liuyang0...@gmail.com <liuyang0...@gmail.com> QQ: 3239559*
