stream
> processor handle the rest. But if I’m understanding Flink’s windowing and
> watermarks correctly for an event-time based app, there doesn’t really seem
> to be a way of revisiting older data. If you’re using event-time windowing,
> then any window for which (end window range + windo
older events into Kafka and let the stream
processor handle the rest. But if I’m understanding Flink’s windowing and
watermarks correctly for an event-time based app, there doesn’t really seem
to be a way of revisiting older data. If you’re using event-time windowing,
then any window for which (end
To clarify one thing: keep in mind that Flink does not support per-key
watermarks. Watermarks are typically generated per-source, or in the case
of kafka, can be per-partition. An idle source (or in the case of kafka, an
idle partition) can prevent an event-time window from being triggered, but
you
Yes in many cases what we have faced that let's say in a keyed stream an
element of a specific key comes in which triggers a new window. If a
corresponding elements of the same key does not arrive a new watermark is
not generated for the window to purge. Then we faced issues with flink
keeping reco
Hi Taher,
For some questions, I suggest you read the documentation related to Flink
EventTime first, for example [1]
About this question:
What happens if the watermark is same as the timestamp?
Here "timestamp", do you mean the current timestamp of Processing time? If
that's the best, it's an id
Hi All,
Can someone show a good example of how watermarks need to be
generated when using EventTime windows? What happens if the watermark is
same as the timestamp? How does the watermark help in the window to be
triggered and what if watermarks are kept behind the currentTimestamps in
Thanks! I’ll be watching that issue then
Adam
> On 08 Aug 2016, at 05:01, Aljoscha Krettek wrote:
>
> Hi Adam,
> sorry for the inconvenience. This is caused by a new file read operator,
> specifically how it treats watermarks/timestamps. I opened an issue here that
> describes the situation:
Hi Adam,
sorry for the inconvenience. This is caused by a new file read operator,
specifically how it treats watermarks/timestamps. I opened an issue here
that describes the situation:
https://issues.apache.org/jira/browse/FLINK-4329.
I think this should be fixed for an upcoming 1.1.1 bug fixing r
Hello,
I have a very simple stream where I window data using event-time.
As a data source I’m using a CSV file, sorted by increasing timestamps.
Here’s the source:
val env = StreamExecutionEnvironment.getExecutionEnvironment
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
val line
(...)
>> .timeWindow(Time.of(5,TimeUnit.SECONDS))
>> // or .window(SlidingTimeWindows.of(Time.of(5, TimeUnit.SECONDS)))
>> .sum(0)
>> .print()
>>
>> I hope this helps. :D
>>
>> Cheers,
>> Aljoscha
>>
>>
>> On Wed, 7 Oct 2
(...)
>> .timeWindow(Time.of(5,TimeUnit.SECONDS))
>> // or .window(SlidingTimeWindows.of(Time.of(5, TimeUnit.SECONDS)))
>> .sum(0)
>> .print()
>>
>> I hope this helps. :D
>>
>> Cheers,
>> Aljoscha
>>
>>
>> On Wed, 7 Oc
this helps. :D
>
> Cheers,
> Aljoscha
>
>
> On Wed, 7 Oct 2015 at 16:54 Alexander Kolb <
> alexander.k...@mni.fh-giessen.de> wrote:
>
>> Hi Guys,
>>
>> I'm trying to use the event-time windowing feature. But the windowing
>> does not
,TimeUnit.SECONDS))
// or .window(SlidingTimeWindows.of(Time.of(5, TimeUnit.SECONDS)))
.sum(0)
.print()
I hope this helps. :D
Cheers,
Aljoscha
On Wed, 7 Oct 2015 at 16:54 Alexander Kolb
wrote:
> Hi Guys,
>
> I'm trying to use the event-time windowing feature. But the window
Hi Guys,
I'm trying to use the event-time windowing feature. But the windowing does
not work as expected.
What I've been doing is to write my own source which implements the
EventTimeSourceFunction and uses the collectWithTimeStamp method.
Additionally I'm emitting a water
14 matches
Mail list logo
