Re: replay kinesis events

2022-03-09 Thread Guoqin Zheng
1] > https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/connectors/datastream/kinesis/#configuring-starting-position > > > Hope this helps, > Thanks > > > On Wed, Mar 9, 2022 at 12:15 AM Guoqin Zheng > wrote: > >> Hi Flink experts, >> >>

replay kinesis events

2022-03-08 Thread Guoqin Zheng
Hi Flink experts, Wondering if there is a built-in way to replay already-processed events in an event queue. For example, if I have a flink app processing event stream from Kinesis. Now if I find a bug in the flink app and make a fix. And I would like to re-process events that are already processe

Re: How to select an event that has max attribute

2021-12-09 Thread Guoqin Zheng
gt; state clean. > > [1] > https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/dev/table/sql/queries/topn/ > > Best, > Jing Zhang > > Guoqin Zheng 于2021年12月9日周四 14:16写道: > >> Hi Jing, >> >> Just verified that it worked with Flink 1.14. But as you said, Flink

Re: How to select an event that has max attribute

2021-12-08 Thread Guoqin Zheng
Hi Jing, Just verified that it worked with Flink 1.14. But as you said, Flink 1.13 does not yet support it. Other than waiting for KDA to upgrade the Flink version, is there any workaround for Flink 1.13? Thanks, -Guoqin On Wed, Dec 8, 2021 at 10:00 PM Guoqin Zheng wrote: > Hi J

Re: How to select an event that has max attribute

2021-12-08 Thread Guoqin Zheng
eadtime), INTERVAL '5' > MINUTES)) > ) WHERE rownum <= 1; > > [1] > https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sql/queries/window-topn/#window-top-n-follows-after-windowing-tvf > > Best, > Jing Zhang > > > > > Guo

Re: How to select an event that has max attribute

2021-12-08 Thread Guoqin Zheng
s a SUM which you should change to MAX and of > course you change the rownum to 1 instead of 3. > > Best regards, > > Martijn > > [1] > > https://github.com/ververica/flink-sql-cookbook/blob/main/aggregations-and-analytics/11_window_top_n/11_window_top_n.md > > Op wo

How to select an event that has max attribute

2021-12-08 Thread Guoqin Zheng
Hi Flink Community, I am curious what the recommended way is to select the event with a max attribute value with SQL api. For example, I have an event stream like: { deviceId, locationId gauge, readtime, <-- eventTime } I want to figure out which device and location has the max gau

Re: PyFlink SQL window aggregation data not written out to file

2021-11-22 Thread Guoqin Zheng
t; [1] > https://issues.apache.org/jira/browse/FLINK-18647 > > Regards, > Roman > > On Wed, Nov 17, 2021 at 7:36 PM Guoqin Zheng > wrote: > > > > Hi Roman, > > > > Thanks for the detailed explanation. > > > > I did try 1.13 and 1.14, but it sti

Re: PyFlink SQL window aggregation data not written out to file

2021-11-17 Thread Guoqin Zheng
as part of FLIP-147, as > well as in previous versions. > So I'd suggest trying the most recent Flink version. > > Regards, > Roman > > On Mon, Nov 15, 2021 at 10:17 PM Guoqin Zheng > wrote: > > > > Hi Roman, > > > > Thanks for your quick respon

Re: PyFlink SQL window aggregation data not written out to file

2021-11-15 Thread Guoqin Zheng
pache.org/flink/flink-docs-release-1.13/docs/deployment/config/#execution-checkpointing-interval > [2] > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/create/#watermark > > > Regards, > Roman > > On Mon, Nov 15, 2021 at 6:43 PM Guoqin Zheng

PyFlink SQL window aggregation data not written out to file

2021-11-15 Thread Guoqin Zheng
Hi all, I am reposting my stackoverflow question here. So I would like to do a local test of my pyflink sql application. The sql query does a very simple job: read from source