Hi, Actually all operators should preserve record timestamps if set the correct TimeCharacteritics to event time. A window operator will set the timestamp of all emitted records to the end-timestamp of the window. Not sure what happens if you use a processing time window in an event time application though... Can you show a concise example of your program and explain how to check the timestamps?
In general it is not a good idea to assign timestamps and watermarks in the middle of a program because it is can be quite hard to reason about out-of-orderness after the data was shuffled and processed. You can use reinterpretAsKeyedStream() if an operator does not change the keys and if the parallelism of the source and target operators are the same. Best, Fabian Am Di., 30. Apr. 2019 um 08:59 Uhr schrieb Guowei Ma <guowei....@gmail.com>: > Hi, > Most operators will preserve the input elements timestamp if it has. > Window is a special case. The timestamp of elements emitted by window is > the maxTimestamp of the Window which is triggered. Different Window will > have different implementation.(GlobalWindow/TimeWindow/CustomizedWindow). > Keyby just shuffle data. I think it could not affect the element's > timestamp. > > Hope this could help. > > Best, > Guowei > > > Averell <lvhu...@gmail.com> 于2019年4月30日周二 上午7:28写道： > >> Hello, >> >> I extracted timestamps using BoundedOutOfOrdernessTimestampExtractor from >> my >> sources, have a WindowFunction, and found that my timestamps has been >> lost. >> To do another Window operation, I need to extract timestamp again. I tried >> to find a document for that but haven't found one. >> Could you please help tell which type of operators would preserve records' >> timestamp? >> >> The same question for keyed stream. I have been using the same key >> throughout my flow, but with many tranformations (using different >> operators, >> including coProcessFunction, and converting my data between different >> classes), and I have been trying to use >> DataStreamUtils.reinterpretAsKeyedStream. Is it safe to assume that as >> long >> as I dont do transformation on key, I could use that >> reinterpretAsKeyedStream function? >> >> Thanks and best regards, >> Averell >> >> >> >> -- >> Sent from: >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >> >
