Re: reprocessing events from multiple topics

2016-05-09 Thread Kyle Mathews
Thanks Ben! Nice to know I was on the right track :-) that Samza doc is very helpful too. On Tue, May 3, 2016 at 8:53 PM Benjamin Manns wrote: > Both of your ideas are doable. Another thing to keep in mind is that > depending on your data source, late arriving data will not be sorted in > front o

Re: reprocessing events from multiple topics

2016-05-03 Thread Benjamin Manns
Both of your ideas are doable. Another thing to keep in mind is that depending on your data source, late arriving data will not be sorted in front of the already committed events. You may need some windowing buffer to recalculate for stragglers. For the multiple-topic approach, check out Samza's M

reprocessing events from multiple topics

2016-05-03 Thread Kyle Mathews
Hi Kafka Users, I'm thinking through how to convert my application to use Kafka. I use an event sourcing model and something I do frequently is reprocess old events when I change a model schema or update my processing code. In my current setup, I have few enough events that I can just load all the