Re: Issues with Event Time and Kafka

2017-03-13 Thread Aljoscha Krettek
eriodicWatermarks[Row] { >>>> >>>>override def extractTimestamp(element: Row, >>>>previousElementTimestamp: Long): Long = { element.minTime } >>>> >>>>override def getCurrentWatermark(): Watermark = { new >>>>W

Re: Issues with Event Time and Kafka

2017-03-10 Thread Robert Metzger
>>> Long): Long = { >>> element.minTime >>> } >>> >>> override def getCurrentWatermark(): Watermark = { >>> new Watermark(System.currentTimeMillis() - maxLateness) >>> } >>> } >>> ``` >>>

Re: Issues with Event Time and Kafka

2017-03-09 Thread ext.eformichella
gt; } >> >> override def getCurrentWatermark(): Watermark = { >> new Watermark(System.currentTimeMillis() - maxLateness) >> } >> } >> ``` >> >> Where Row is a class representing the incoming JSON object coming from >> Kafka, which includes

Re: Issues with Event Time and Kafka

2017-03-07 Thread Dawid Wysakowicz
presenting the incoming JSON object coming from > Kafka, which includes the timestamp > > Thanks, > -Ethan > > > > -- > View this message in context: http://apache-flink-user- > mailing-list-archive.2336050.n4.nabble.com/Issues-with- > Event-Time-and-Kafka-tp12061p12090.html > Sent from the Apache Flink User Mailing List archive. mailing list archive > at Nabble.com. >

Re: Issues with Event Time and Kafka

2017-03-07 Thread ext.eformichella
ge in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Issues-with-Event-Time-and-Kafka-tp12061p12090.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Issues with Event Time and Kafka

2017-03-07 Thread Dawid Wysakowicz
ter if folks believe that is >>> the issue I would be happy to provide that code here as well. >>> >>> 2) On top of all this, we would really prefer to write the records >>> directly >>> to Aurora in RDS rather than to an intermediate s3 bucket, but it seems

Re: Issues with Event Time and Kafka

2017-03-07 Thread ext.mwalker
ontext: http://apache-flink-user-maili >> ng-list-archive.2336050.n4.nabble.com/Issues-with-Event- >> Time-and-Kafka-tp12061.html >> Sent from the Apache Flink User Mailing List archive. mailing list >> archive at Nabble.com. >> > > > > ---

Re: Issues with Event Time and Kafka

2017-03-07 Thread Stephan Ewen
Thanks in advance for all the help / insight on this, > > Max Walker > > > > -- > View this message in context: http://apache-flink-user- > mailing-list-archive.2336050.n4.nabble.com/Issues-with- > Event-Time-and-Kafka-tp12061.html > Sent from the Apache Flink User Mailing List archive. mailing list archive > at Nabble.com. >

Issues with Event Time and Kafka

2017-03-06 Thread ext.mwalker
e for all the help / insight on this, Max Walker -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Issues-with-Event-Time-and-Kafka-tp12061.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.