Re: Messages lost after broker failure

2017-04-11 Thread Jagadish Venkatraman
;> Aleksandar Bircakovic >> >> -Original Message- >> From: Aleksandar Bircakovic [mailto:a.bircako...@levi9.com] >> Sent: Wednesday, April 5, 2017 5:26 PM >> To: dev@samza.apache.org >> Cc: m.mis...@improvedigital.com >> Subject: RE: Messages lost af

Re: Messages lost after broker failure

2017-04-11 Thread Jagadish Venkatraman
.com] > Sent: Wednesday, April 5, 2017 5:26 PM > To: dev@samza.apache.org > Cc: m.mis...@improvedigital.com > Subject: RE: Messages lost after broker failure > > Thank you Jagadish. > > Regarding leader selection, we exposed some additional metrics that are > supposed to tell us wh

RE: Messages lost after broker failure

2017-04-11 Thread Aleksandar Bircakovic
@samza.apache.org Cc: m.mis...@improvedigital.com Subject: RE: Messages lost after broker failure Thank you Jagadish. Regarding leader selection, we exposed some additional metrics that are supposed to tell us which broker is leader for each partition and we confirmed your theory, Samza consumer if

RE: Messages lost after broker failure

2017-04-05 Thread Aleksandar Bircakovic
a.apache.org Cc: m.mis...@improvedigital.com Subject: Re: Messages lost after broker failure >> All this is leading us to conclusion that Samza's consumers are >> somehow not aware of all of the partitions We have had a number of broker failures at LinkedIn, and have not r

Re: Messages lost after broker failure

2017-04-04 Thread Jagadish Venkatraman
>> All this is leading us to conclusion that Samza's consumers are somehow not aware of all of the partitions We have had a number of broker failures at LinkedIn, and have not run into data loss issues due to consumers being unaware of partitions. You can use the metrics emitted at a per-partition

Messages lost after broker failure

2017-04-04 Thread Aleksandar Bircakovic
Hi everyone, my team is building real-time system using Samza (version 0.11.0) and we are facing some issues with data loss so we would like to hear your thoughts. Due to using some additional tools for monitoring and alerting we exceeded number of allowed open files so TooManyOpenFiles exceptio