Re: MirrorMaker improvements

2015-03-25 Thread Jiangjie Qin
gt; >> consumer like we do for new producer. Since we have correlation ID, >> >> reorder should be easily handled. I haven¹t got a chance to read the >>new >> >> consumer code, but I think it is worth doing if we haven¹t done so. >> >> >> >> Jiangj

Re: MirrorMaker improvements

2015-03-25 Thread vlad...@gmail.com
; >> > >> On 3/25/15, 9:50 AM, "Jun Rao" wrote: > >> > >> >To amortize the long RTT across data centers, you can tune the TCP > >>window > >> >size by configuring a larger socket.receive.buffer.bytes in the > >>consumer. > &

Re: MirrorMaker improvements

2015-03-25 Thread Jiangjie Qin
nfiguring a larger socket.receive.buffer.bytes in the >>consumer. >> > >> >For the last one, it seems that you want identical mirroring. The >>tricky >> >thing is to figure out how to avoid duplicates when there is a >>failure. We >> >

Re: MirrorMaker improvements

2015-03-25 Thread vlad...@gmail.com
d discussion in the context of transactional messaging ( > > > https://cwiki.apache.org/confluence/display/KAFKA/Transactional+Messaging+ > >in+Kafka > >). > > > >Thanks, > > > >Jun > > > >On Tue, Mar 24, 2015 at 11:44 AM, vlad...@gmail.com &g

Re: MirrorMaker improvements

2015-03-25 Thread Jiangjie Qin
>). > >Thanks, > >Jun > >On Tue, Mar 24, 2015 at 11:44 AM, vlad...@gmail.com >wrote: > >> Dear all, >> >> I had a short discussion with Jay yesterday at the ACM meetup and he >> suggested writing an email regarding a few possible MirrorMake

Re: MirrorMaker improvements

2015-03-25 Thread Jun Rao
ay at the ACM meetup and he > suggested writing an email regarding a few possible MirrorMaker > improvements. > > At Turn, we have been using MirrorMaker for a a few months now to > asynchronously replicate our key/value store data between our datacenters. > In a way, our syst

Re: MirrorMaker improvements

2015-03-24 Thread Guozhang Wang
active-active? Guozhang On Tue, Mar 24, 2015 at 12:18 PM, vlad...@gmail.com wrote: > Dear all, > > I had a short discussion with Jay yesterday at the ACM meetup and he > suggested writing an email regarding a few possible MirrorMaker > improvements. > > At Turn, we have been usin

MirrorMaker improvements

2015-03-24 Thread vlad...@gmail.com
Dear all, I had a short discussion with Jay yesterday at the ACM meetup and he suggested writing an email regarding a few possible MirrorMaker improvements. At Turn, we have been using MirrorMaker for a a few months now to asynchronously replicate our key/value store data between our datacenters

Fwd: MirrorMaker improvements

2015-03-24 Thread vlad...@gmail.com
Dear all, I had a short discussion with Jay yesterday at the ACM meetup and he suggested writing an email regarding a few possible MirrorMaker improvements. At Turn, we have been using MirrorMaker for a a few months now to asynchronously replicate our key/value store data between our datacenters