Thanks, I will take a look -Neha
On Thu, Jan 24, 2013 at 12:55 AM, Maxime Brugidou <maxime.brugi...@gmail.com > wrote: > Thanks for your response, here is the JIRA: > https://issues.apache.org/jira/browse/KAFKA-732 > > > On Thu, Jan 24, 2013 at 1:16 AM, Neha Narkhede <neha.narkh...@gmail.com > >wrote: > > > We haven't tried to run this. Please can you file a JIRA ? > > > > > > On Wed, Jan 23, 2013 at 9:30 AM, Maxime Brugidou > > <maxime.brugi...@gmail.com>wrote: > > > > > MirrorMaker between two 0.8 clusters. > > > > > > > > > On Wed, Jan 23, 2013 at 5:54 PM, Neha Narkhede < > neha.narkh...@gmail.com > > > >wrote: > > > > > > > Hi Maxime, > > > > > > > > Do you mean KafkaMigrationTool or MirrorMaker ? Are you trying to > > mirror > > > > data between a Kafka 0.7 and Kafka 0.8 cluster ? > > > > We have fixed the KafkaMigrationTool to always turn off shallow > > iteration > > > > since we have to decompress data in 0.7 format before sending it to a > > 0.8 > > > > broker. > > > > > > > > Thanks, > > > > Neha > > > > > > > > > > > > On Wed, Jan 23, 2013 at 6:39 AM, Maxime Brugidou > > > > <maxime.brugi...@gmail.com>wrote: > > > > > > > > > Hi all, > > > > > > > > > > I am working with MirrorMaker on the 0.8 branch (so I haven't > tested > > > this > > > > > on 0.7 but it looks similar). > > > > > > > > > > When I enable shallow.iterator.enable=true on the consumer config, > > (and > > > > > disable compression on the producer config, as mentioned in the > wiki) > > > the > > > > > throughput is really good but the data produced is unreadable since > > the > > > > > messages are taken "as is" by the producer and i just get a bunch > of > > > > bytes > > > > > stored with the ByteBufferMessageSet header and all the info. > > > > > > > > > > I tested using the kafka-console-consumer to read my mirrored data > > and > > > > it's > > > > > unreadable (unless I pipe the data to gunzip, which manages to read > > > it). > > > > > > > > > > Did I miss something? Is there anything i can activate on the > > producer > > > > side > > > > > of MirrorMaker to send the shallow message "as is"? Did anyone > manage > > > to > > > > > make MirrorMaker work with shallow.iterator.enable=true? > > > > > > > > > > Thanks for your help, > > > > > Maxime > > > > > > > > > > > > > > >
