Re: Log compaction recover strategy

2015-03-10 Thread Pierre-Yves Ritschard
On 03/10/2015 05:48 PM, Mayuresh Gharat wrote: > How do you typically handle workers starting, always start at offset 0 > to make sure the view is correctly recreated ? > ---> You will have to reset the offsets to 0 and the offset reset policy to > earliest in consumer. Yup, as expected. > > Ho

Re: Log compaction recover strategy

2015-03-10 Thread Mayuresh Gharat
How do you typically handle workers starting, always start at offset 0 to make sure the view is correctly recreated ? ---> You will have to reset the offsets to 0 and the offset reset policy to earliest in consumer. How do you handle topology changes in consumers, which lead to a redistribution of

Log compaction recover strategy

2015-03-10 Thread Pierre-Yves Ritschard
Hi kafka, I've started implementing simple materialized views with the log compaction feature to test it out, and it works great. I'll share the code and an accompanying article shortly but first wanted to discuss some of the production implications my sandbox has. I've separated the project in t