Re: Trigger Time vs. Latest Acknowledgement

2018-01-31 Thread Kostas Kloudas
Hi Juho, I think that neither a) nor b) hold. The reported times are wall-clock times (or processing time in Flink terminology) when the checkpoint started and when it finished. What you want, if I understand correctly, is these times to reflect the event time of your pipeline. In other wo

Re: Trigger Time vs. Latest Acknowledgement

2018-01-30 Thread Aljoscha Krettek
Hi, I think a) doesn't hold because there is no synchronisation between the CheckpointCoordinator and the sources doing the reading. I think b) will hold but it's also not exact because of clock differences between different machines and whatnot. Best, Aljoscha > On 29. Jan 2018, at 15:34, Ju

Trigger Time vs. Latest Acknowledgement

2018-01-29 Thread Juho Autio
I'm triggering nightly savepoints at 23:59:00 with crontab on the flink cluster. For example last night's savepoint has this information: Trigger Time: 23:59:14 Latest Acknowledgement: 00:00:59 What are the min/max boundaries for the data contained by the savepoint? Can I deduce from this either