Re: Working with bounded Datastreams - Flink 1.11.1

2020-10-30 Thread Arvid Heise
Hi Sunitha, you probably need to apply a non-windowed grouping. datastream .keyBy(Event::getVehicleId) .reduce((first, other) -> first); This example will always throw away the second record. You may want to combine the records though by summing up the fuel. Best, Arvid On Wed, Oct 28,

Re: Working with bounded Datastreams - Flink 1.11.1

2020-10-28 Thread Danny Chan
In SQL, you can use the over window to deduplicate the messages by the id [1], but i'm not sure if there are same semantic operators in DataStream. [1] https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication s_penakalap...@yahoo.com 于2020年10月28日周三 下午12:34写

Re: Working with bounded Datastreams - Flink 1.11.1

2020-10-27 Thread s_penakalap...@yahoo.com
Hi All, Request your inputs please. Regards,Sunitha On Tuesday, October 27, 2020, 01:01:41 PM GMT+5:30, s_penakalap...@yahoo.com wrote: Hi Team, I want to use Flink Datastream for Batch operations which involves huge data, I did try to calculate count and average on the whole Datastrea