Re: [DISCUSS] FLIP-155: Introduce a few convenient operations in Table API

2021-01-07 Thread Timo Walther
Hi Dian, Thanks for working on improving the Table API. I went through the entire FLIP and many functions definitely make sense. However, we need to make sure that the general API naming, behavior etc. remains consistent. Here is some feedback from my side: 1) deduplicate Are we planning to

Re: [DISCUSS] FLIP-155: Introduce a few convenient operations in Table API

2021-01-05 Thread Dian Fu
Hi all, I have updated the FLIP about temporal join, sql hints and window TVF. Regards, Dian > 在 2021年1月5日,上午11:58,Dian Fu 写道: > > Thanks a lot for your comments! > > Regarding to Python Table API examples: I thought it should be > straightforward about how to use these operations in Python

Re: [DISCUSS] FLIP-155: Introduce a few convenient operations in Table API

2021-01-04 Thread Dian Fu
Thanks a lot for your comments! Regarding to Python Table API examples: I thought it should be straightforward about how to use these operations in Python Table API and so have not added them. However, the suggestions make sense to me and I have added some examples about how to use them in Pyth

Re: [DISCUSS] FLIP-155: Introduce a few convenient operations in Table API

2021-01-04 Thread Jark Wu
Thanks Dian, +1 to `deduplicate`. Regarding `myTable.coalesce($("a"), 1).as("a")`, I'm afraid it may conflict/confuse the built-in expression `coalesce(f0, 0)` (we may introduce it in the future). Besides that, could we also align other features of Flink SQL, e.g. event-time/processing-time temp

Re: [DISCUSS] FLIP-155: Introduce a few convenient operations in Table API

2021-01-04 Thread Timo Walther
Hi Dian, thanks for the proposed FLIP. I haven't taken a deep look at the proposal yet but will do so shortly. In general, we should aim to make the Table API as concise and self-explaining as possible. E.g. `dropna` does not sound obvious to me. Regarding `myTable.coalesce($("a"), 1).as("a"

Re: [DISCUSS] FLIP-155: Introduce a few convenient operations in Table API

2021-01-04 Thread Seth Wiesman
This makes sense, I have some questions about method names. What do you think about renaming `dropDuplicates` to `deduplicate`? I don't think that drop is the right word to use for this operation, it implies records are filtered where this operator actually issues updates and retractions. Also, de

Re: [DISCUSS] FLIP-155: Introduce a few convenient operations in Table API

2021-01-04 Thread Wei Zhong
Hi Dian, Big +1 for making the Table API easier to use. Java users and Python users can both benefit from it. I think it would be better if we add some Python API examples. Best, Wei > 在 2021年1月4日,20:03,Dian Fu 写道: > > Hi all, > > I'd like to start a discussion about introducing a few con

[DISCUSS] FLIP-155: Introduce a few convenient operations in Table API

2021-01-04 Thread Dian Fu
Hi all, I'd like to start a discussion about introducing a few convenient operations in Table API from the perspective of ease of use. Currently some tasks are not easy to express in Table API e.g. deduplication, topn, etc, or not easy to express when there are hundreds of columns in a table,