On Kafka 0.8.2 release there were a few issues like KAFKA-2477 <http://issues.apache.org/jira/browse/KAFKA-2477> that could have impacted you. Did you have issue with just one topic or all topics got data deleted? Also did you have a per topic level retention override configured on top of the base retention policy? Somethings for you to check
On Thu, May 5, 2016 at 7:37 AM, John Bickerstaff <j...@johnbickerstaff.com> wrote: > kafka_2.10-0.8.2.2 > > Thanks > > On Wed, May 4, 2016 at 9:42 PM, Salman Ahmed <ahmed.sal...@gmail.com> > wrote: > > > Which version of Kafka are you on? > > On Wed, May 4, 2016 at 4:54 PM John Bickerstaff < > j...@johnbickerstaff.com> > > wrote: > > > > > Hi, > > > > > > I've been working with Kafka a lot recently and have the log retention > > set > > > to over two years. > > > > > > Today, as I was trying some things I happened to have the log running > in > > > another window and on two different VMs, Kafka decided to delete all my > > log > > > messages, thus losing the entire topic's worth of data - something that > > > takes a couple of hours to load. > > > > > > Can anyone tell me what might have affected this? I set the log > > retention > > > to a very big number in the server.properties file, but I'm unaware of > > > anything else that might affect this. > > > > > > What I observed is that this happened very soon after a reboot of Kafka > > > (under 5 minutes I think - certainly less than 10) and I watched it > occur > > > on two boxes within half an hour of each other as I was restarting > > Kafka... > > > > > >
