Hi all,

I have updated the FLIP about temporal join, sql hints and window TVF.

Regards,
Dian

> 在 2021年1月5日,上午11:58,Dian Fu <dian0511...@gmail.com> 写道:
> 
> Thanks a lot for your comments!
> 
> Regarding to Python Table API examples: I thought it should be 
> straightforward about how to use these operations in Python Table API and so 
> have not added them. However, the suggestions make sense to me and I have 
> added some examples about how to use them in Python Table API to make it more 
> clear.
> 
> Regarding to dropDuplicates vs deduplicate: +1 to use deduplicate. It's more 
> consistent with the feature/concept which is already documented clearly in 
> Flink.
> 
> Regarding to `myTable.coalesce($("a"), 1).as("a")`: I'm still in favor of 
> fillna for now. Compared to coalesce, fillna could handle multiple columns in 
> one method call. For the naming convention, the name "fillna/dropna/replace" 
> comes from Pandas [1][2][3].
> 
> Regarding to `event-time/processing-time temporal join, SQL Hints, window 
> TVF`: Good catch! Definitely we should support them in Table API. I will 
> update the FLIP about these functionalities.
> 
> [1] https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.fillna.html 
> <https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.fillna.html>
> [2] https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.dropna.html 
> <https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.dropna.html>
> [3] 
> https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.replace.html 
> <https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.replace.html>
>> 在 2021年1月4日,下午10:59,Timo Walther <twal...@apache.org 
>> <mailto:twal...@apache.org>> 写道:
>> 
>> Hi Dian,
>> 
>> thanks for the proposed FLIP. I haven't taken a deep look at the proposal 
>> yet but will do so shortly. In general, we should aim to make the Table API 
>> as concise and self-explaining as possible. E.g. `dropna` does not sound 
>> obvious to me.
>> 
>> Regarding `myTable.coalesce($("a"), 1).as("a")`: Instead of introducing more 
>> top-level functions, maybe we should also consider introducing more building 
>> blocks e.g. for applying an expression to every column. A more functional 
>> approach (e.g. with lamba function) could solve more use cases.
>> 
>> Regards,
>> Timo
>> 
>> On 04.01.21 15:35, Seth Wiesman wrote:
>>> This makes sense, I have some questions about method names.
>>> What do you think about renaming `dropDuplicates` to `deduplicate`? I don't
>>> think that drop is the right word to use for this operation, it implies
>>> records are filtered where this operator actually issues updates and
>>> retractions. Also, deduplicate is already how we talk about this feature in
>>> the docs so I think it would be easier for users to find.
>>> For null handling, I don't know how close we want to stick with SQL
>>> conventions but what about making `coalesce` a top-level method? Something
>>> like:
>>> myTable.coalesce($("a"), 1).as("a")
>>> We can require the next method to be an `as`. There is already precedent
>>> for this sort of thing, `GroupedTable#aggregate` can only be followed by
>>> `select`.
>>> Seth
>>> On Mon, Jan 4, 2021 at 6:27 AM Wei Zhong <weizhong0...@gmail.com 
>>> <mailto:weizhong0...@gmail.com>> wrote:
>>>> Hi Dian,
>>>> 
>>>> Big +1 for making the Table API easier to use. Java users and Python users
>>>> can both benefit from it. I think it would be better if we add some Python
>>>> API examples.
>>>> 
>>>> Best,
>>>> Wei
>>>> 
>>>> 
>>>>> 在 2021年1月4日,20:03,Dian Fu <dian0511...@gmail.com 
>>>>> <mailto:dian0511...@gmail.com>> 写道:
>>>>> 
>>>>> Hi all,
>>>>> 
>>>>> I'd like to start a discussion about introducing a few convenient
>>>> operations in Table API from the perspective of ease of use.
>>>>> 
>>>>> Currently some tasks are not easy to express in Table API e.g.
>>>> deduplication, topn, etc, or not easy to express when there are hundreds of
>>>> columns in a table, e.g. null data handling, etc.
>>>>> 
>>>>> I'd like to propose to introduce a few operations in Table API with the
>>>> following purposes:
>>>>> - Make Table API users to easily leverage the powerful features already
>>>> in SQL, e.g. deduplication, topn, etc
>>>>> - Provide some convenient operations, e.g. introducing a series of
>>>> operations for null data handling (it may become a problem when there are
>>>> hundreds of columns), data sampling and splitting (which is a very common
>>>> use case in ML which usually needs to split a table into multiple tables
>>>> for training and validation separately).
>>>>> 
>>>>> Please refer to FLIP-155 [1] for more details.
>>>>> 
>>>>> Looking forward to your feedback!
>>>>> 
>>>>> Regards,
>>>>> Dian
>>>>> 
>>>>> [1]
>>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-155%3A+Introduce+a+few+convenient+operations+in+Table+API
>>>>  
>>>> <https://cwiki.apache.org/confluence/display/FLINK/FLIP-155%3A+Introduce+a+few+convenient+operations+in+Table+API>
>>>> 
>>>> 
>> 
> 

Reply via email to