Re: How to select an event that has max attribute

2021-12-11 Thread Jing Zhang
You are Welcome. Glad to hear the information is helpful. Guoqin Zheng 于2021年12月10日周五 03:28写道: > Hi Jing, > > Thanks for the advice. This is very helpful. > > -Guoqin > > On Wed, Dec 8, 2021 at 11:52 PM Jing Zhang wrote: > >> Hi Guoqin, >> I understand the problem you are suffering. >> I'm sorr

Re: How to select an event that has max attribute

2021-12-09 Thread Guoqin Zheng
Hi Jing, Thanks for the advice. This is very helpful. -Guoqin On Wed, Dec 8, 2021 at 11:52 PM Jing Zhang wrote: > Hi Guoqin, > I understand the problem you are suffering. > I'm sorry I could not find out a perfect solution on Flink 1.13. > > Maybe you could try to use TopN [1] instead of Windo

Re: How to select an event that has max attribute

2021-12-08 Thread Jing Zhang
Hi Guoqin, I understand the problem you are suffering. I'm sorry I could not find out a perfect solution on Flink 1.13. Maybe you could try to use TopN [1] instead of Window TopN by normalizing time into a unit with 5 minute, and add it to be one of partition keys. But the result is an update stre

Re: How to select an event that has max attribute

2021-12-08 Thread Guoqin Zheng
Hi Jing, Just verified that it worked with Flink 1.14. But as you said, Flink 1.13 does not yet support it. Other than waiting for KDA to upgrade the Flink version, is there any workaround for Flink 1.13? Thanks, -Guoqin On Wed, Dec 8, 2021 at 10:00 PM Guoqin Zheng wrote: > Hi Jing, > > Thanks

Re: How to select an event that has max attribute

2021-12-08 Thread Guoqin Zheng
Hi Jing, Thanks for chiming in. This sounds great. Any chance this will work for Flink 1.13 as well, as I am using AWS KDA. Thanks, -Guoqin On Wed, Dec 8, 2021 at 7:47 PM Jing Zhang wrote: > Hi Guoqin, > I guess you have misunderstood Martijn's response. > Martijn suggest you use Window TopN.

Re: How to select an event that has max attribute

2021-12-08 Thread Jing Zhang
Hi Guoqin, I guess you have misunderstood Martijn's response. Martijn suggest you use Window TopN. Besides, Window TopN does not need to follow a Window Aggregate, it could followed with Window TVF directly since Flink 1.14. Please see document [1] attached. You could try the following SQL to get t

Re: How to select an event that has max attribute

2021-12-08 Thread Guoqin Zheng
Hi Martijn, Thanks for your quick response. I tried it, but it does not seem to work. The problem is that I want to select fields that are not in the `GROUP BY`. So in my example, I can have a tumble window on `readtime`, and select max(gauge), but I also want both `deviceId` and `locationId` of

Re: How to select an event that has max attribute

2021-12-08 Thread Martijn Visser
Hi Guoqin, I think you could use the Window Top-N. There's a recipe in the Flink SQL Cookbook [1]. The example uses a SUM which you should change to MAX and of course you change the rownum to 1 instead of 3. Best regards, Martijn [1] https://github.com/ververica/flink-sql-cookbook/blob/main/agg

How to select an event that has max attribute

2021-12-08 Thread Guoqin Zheng
Hi Flink Community, I am curious what the recommended way is to select the event with a max attribute value with SQL api. For example, I have an event stream like: { deviceId, locationId gauge, readtime, <-- eventTime } I want to figure out which device and location has the max gau