Hi Soheil, Watermark indicates the progress of the Event time. The reason it exists is because there is a Time skew between Event time and Processing time. Hequn is correct and Watermark cannot be used for processing time. The processing time will be based on the TM local system clock. Usually, when there is a time field in your event that indicates when it actually happened, we will choose Event time. When we choose Processing time, we don't rely on the time information carried by the data itself, so the question is how do you define "bad data".
Thanks, vino. 2018-07-30 22:29 GMT+08:00 Hequn Cheng <chenghe...@gmail.com>: > Hi Soheil, > > No, we can't set watermark during processing time. And there are no late > data considering processing time window. > So the problem is what data is bad data when you use processing time? > Maybe there are other ways to solve your problem. > > Best, Hequn > > On Mon, Jul 30, 2018 at 8:22 PM, Soheil Pourbafrani <soheil.i...@gmail.com > > wrote: > >> In Event Time, we can gather bad data using OutputTag, because in Event >> Time we have Watermark and we can detect late data. But in processing time >> mode we don't have any watermark to detect bad data. I want to know can we >> set watermark (for example according to taskmanager's timestamp) and use >> processing time in creating time windows? >> > >
