Thanks for the clarification. I have resolved all of the comments and added a conclusion section.
Looking forward to the further feedback from our community. If we get consensus on the design doc, I can push the implementation related work. Kurt Young <ykt...@gmail.com> 于2021年1月5日周二 上午11:04写道： > Sorry for the typo -_-! > I meant idea #2. > > Best, > Kurt > > > On Tue, Jan 5, 2021 at 10:59 AM Sebastian Liu <liuyang0...@gmail.com> > wrote: > >> Hi Kurt, >> >> Thx a lot for your feedback. If local aggregation is more like a physical >> operator rather than logical >> operator, I think your suggestion should be idea #2 which handle all in >> the physical optimization phase? >> >> Looking forward for the further discussion. >> >> >> Kurt Young <ykt...@gmail.com> 于2021年1月5日周二 上午9:52写道： >> >>> Local aggregation is more like a physical operator rather than logical >>> operator. I would suggest going with idea #1. >>> >>> Best, >>> Kurt >>> >>> >>> On Wed, Dec 30, 2020 at 8:31 PM Sebastian Liu <liuyang0...@gmail.com> >>> wrote: >>> >>> > Hi Jark, Thx a lot for your quick reply and valuable suggestions. >>> > For (1): Agree: Since we are in the period of upgrading the new table >>> > source api, >>> > we really should consider the new interface for the new optimize rule. >>> If >>> > the new rule >>> > doesn't use the new api, we'll have to upgrade it sooner or later. I >>> have >>> > change to use >>> > the ability interface for the SupportsAggregatePushDown definition in >>> above >>> > proposal. >>> > >>> > For (2): Agree: Change to use CallExpression is a better choice, and >>> have >>> > resolved this >>> > comment in the proposal. >>> > >>> > For (3): I suggest we first support the JDBC connector, as we don't >>> have >>> > Druid connector >>> > and ES connector just has sink api at present. >>> > >>> > But perhaps the biggest question may be whether we should use idea 1 or >>> > idea 2 in proposal. >>> > >>> > What do you think? After we reach the agreement on the proposal, our >>> team >>> > can drive to >>> > complete this feature. >>> > >>> > Jark Wu <imj...@gmail.com> 于2020年12月29日周二 下午2:58写道： >>> > >>> > > Hi Sebastian, >>> > > >>> > > Thanks for the proposal. I think this is a great improvement for >>> Flink >>> > SQL. >>> > > I went through the design doc and have following thoughts: >>> > > >>> > > 1) Flink has deprecated the legacy TableSource in 1.11 and proposed >>> a new >>> > > set of DynamicTableSource interfaces. Could you update your >>> proposal to >>> > > use the new interfaces? >>> > > Follow the existing ability interfaces, e.g. >>> > > SupportsFilterPushDown, SupportsProjectionPushDown. >>> > > >>> > > 2) Personally, I think CallExpression would be a better >>> representation >>> > than >>> > > separate `FunctionDefinition` and args. Because, it would be easier >>> to >>> > know >>> > > what's the index and type of the arguments. >>> > > >>> > > 3) It would be better to list which connectors will be supported in >>> the >>> > > plan? >>> > > >>> > > Best, >>> > > Jark >>> > > >>> > > >>> > > On Tue, 29 Dec 2020 at 00:49, Sebastian Liu <liuyang0...@gmail.com> >>> > wrote: >>> > > >>> > > > Hi all, >>> > > > >>> > > > I'd like to discuss a new feature for the Blink Planner. >>> > > > Aggregate operator of Flink SQL is currently fully done at Flink >>> layer. >>> > > > With the developing of storage, many downstream storage of Flink >>> SQL >>> > has >>> > > > the ability to deal with Aggregation operator. >>> > > > Pushing down Aggregate to data source layer will improve >>> performance >>> > from >>> > > > the perspective of the network IO and computation overhead. >>> > > > >>> > > > I have drafted a design doc for this new feature. >>> > > > >>> > > > >>> > > >>> > >>> https://docs.google.com/document/d/1kGwC_h4qBNxF2eMEz6T6arByOB8yilrPLqDN0QBQXW4/edit?usp=sharing >>> > > > >>> > > > Any comment or discussion is welcome. >>> > > > >>> > > > -- >>> > > > >>> > > > *With kind regards >>> > > > ------------------------------------------------------------ >>> > > > Sebastian Liu 刘洋 >>> > > > Institute of Computing Technology, Chinese Academy of Science >>> > > > Mobile\WeChat: +86—15201613655 >>> > > > E-mail: liuyang0...@gmail.com <liuyang0...@gmail.com> >>> > > > QQ: 3239559* >>> > > > >>> > > >>> > >>> > >>> > -- >>> > >>> > *With kind regards >>> > ------------------------------------------------------------ >>> > Sebastian Liu 刘洋 >>> > Institute of Computing Technology, Chinese Academy of Science >>> > Mobile\WeChat: +86—15201613655 >>> > E-mail: liuyang0...@gmail.com <liuyang0...@gmail.com> >>> > QQ: 3239559* >>> > >>> >> >> >> -- >> >> *With kind regards >> ------------------------------------------------------------ >> Sebastian Liu 刘洋 >> Institute of Computing Technology, Chinese Academy of Science >> Mobile\WeChat: +86—15201613655 >> E-mail: liuyang0...@gmail.com <liuyang0...@gmail.com> >> QQ: 3239559* >> >> -- *With kind regards ------------------------------------------------------------ Sebastian Liu 刘洋 Institute of Computing Technology, Chinese Academy of Science Mobile\WeChat: +86—15201613655 E-mail: liuyang0...@gmail.com <liuyang0...@gmail.com> QQ: 3239559*
