The main reason would be to avoid issues related to co-occurrence in processing time. However, your last comment shows that we can neglect the issue for the moment. More work is not an issue per se. But more work for nothing is another story. ;-) In fact, as you underlined, the procTime semantics kinds of makes the time-related sorting relative to the implementation choices. Fair enough.
Thanks for clarifiying. Best, Stefano -----Original Message----- From: Fabian Hueske [mailto:fhue...@gmail.com] Sent: Thursday, April 06, 2017 1:17 PM To: dev@flink.apache.org Subject: Re: StreamSQL procTime granularity Hi Stefano, thanks for starting this discussion. I think treating processing time with a nanosecond resolution might be difficult for a few reasons: - We would need to treat it in all operators as nanotime (joins, etc.) - Flink does not provide tooling for nanotime resolution. The internal timer service is on milliseconds. - We could not use the testing facilities that the DataStream API offers because they work on milliseconds. I'm not sure what the benefits of switching to nanoseconds would be. Sure, higher resolution sounds good, but on the other hand processing time is somewhat arbitrary anyway. Best, Fabian 2017-04-04 10:23 GMT+02:00 Stefano Bortoli <stefano.bort...@huawei.com>: > Hi guys, > > Based on the discussion about time management in > https://github.com/apache/flink/pull/3641 , does it make sense to use > nanoTime for procTime semantic aggregation processing? This way we > will not have the possibility of events falling in the same > "millisecond" processing bucket and keep a consistent aggregation > sorting (also in the state). I understand that event-time cannot be > managed in nanosecond as java does not give wall-clock time in > nanoseconds, but for the procTime within the JVM we should be safe. > > Please let me know what you think. > > Best, > Stefano > >
