Re: Spark Streaming application code change and stateful transformations

2015-09-17 Thread Cody Koeninger
alues preloaded from DB > 2. By cleaning the checkpoint in between upgrades, data is loaded > only once > > Hope this helps, > -adrian > > From: Ofir Kerker > Date: Wednesday, September 16, 2015 at 6:12 PM > To: Cody Koeninger > Cc: "user@spark.ap

Re: Spark Streaming application code change and stateful transformations

2015-09-17 Thread Adrian Tanase
, data is loaded only once Hope this helps, -adrian From: Ofir Kerker Date: Wednesday, September 16, 2015 at 6:12 PM To: Cody Koeninger Cc: "user@spark.apache.org<mailto:user@spark.apache.org>" Subject: Re: Spark Streaming application code change and stateful transformations Than

Re: Spark Streaming application code change and stateful transformations

2015-09-16 Thread Ofir Kerker
m Kafka from the beginning of retention >> period (last X hours) and ignore events from timestamp smaller or equal >> than >> the last-complete-hour. >> >> I'll be happy to get your feedback! >> >> Thanks, >> O

Re: Spark Streaming application code change and stateful transformations

2015-09-14 Thread Cody Koeninger
y to get your feedback! > > Thanks, > Ofir > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-application-code-change-and-stateful-transformations-tp24692.html > Sent from the Apache Spark User Li

Spark Streaming application code change and stateful transformations

2015-09-14 Thread Ofir Kerker
I'll be happy to get your feedback! Thanks, Ofir -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-application-code-change-and-stateful-transformations-tp24692.html Sent from the Apache Spark User List mail