Thanks Guozhang, Yes we had set to 'largest' and changing it to 'smallest'
resolved the issue. So it was due to the jira
https://issues.apache.org/jira/browse/KAFKA-1006
Thanks,
Raja.
On Tue, Sep 10, 2013 at 1:18 PM, Guozhang Wang wrote:
> Oh got it. Did you set auto.offset.reset = smallest or
Hi Guozhang ,
1) When I say "I send messages to new topic" -> yes I am sending new
messages to source cluster via console producer.
2) The log message "Handling 0 events" doesn't output topic name. But I
would believe its for both old and new topics, because no other app is
sending messages to sou
Hi Raja,
When you say "I send messages to new topic" I guess you mean that you send
messages to the source cluster right? It may be due to the fact that
producers of mirror make have not catched up with the mirror maker consumer.
When you say "I always see Handling 0 events" do you mean that you
Oh got it. Did you set auto.offset.reset = smallest or largest? If it is
largest it could be due to this bug:
https://issues.apache.org/jira/browse/KAFKA-1006
Guozhang
On Tue, Sep 10, 2013 at 10:09 AM, Rajasekar Elango
wrote:
> Hi Guozhang ,
>
> 1) When I say "I send messages to new topic" ->
Thanks Guozhang,
1, 2, 3 all are true. We are using default value 200 for batch.num.messages
and 5000ms queue.buffering.max.ms. I believe it should batch either if
batch.num.messages is reached or queue.buffering.max.ms is reached.
I see log message "5000ms elapsed , Queue time reached. Sending.
yes, the data exists in source cluster, but not in target cluster. I can't
replicate this problem in dev environment and it happens only in prod
environment. I turned on debug logging, but not able to identify the
problem. Basically, whenever I send data to new topic, I don't see any log
messages
Hi Raja,
So just to summarize the scenario:
1) The consumer of mirror maker is successfully consuming all partitions of
the newly created topic.
2) The producer of mirror maker is not producing the new messages
immediately when the topic is created (observed from ProducerSendThread's
log).
3) The
Hi Rajasekar,
You said that ConsumerOffsetChecker shows that new topics are successfully
consumed and the lag is 0. If that's the case, can you verify that there
is data on the source cluster for these new topics? If there is no data at
the source, MirrorMaker will only assign consumer streams to
Could you let me know the process of reproducing this issue?
Guozhang
On Thu, Sep 5, 2013 at 5:04 PM, Rajasekar Elango wrote:
> Yes guozhang
>
> Sent from my iPhone
>
> On Sep 5, 2013, at 7:53 PM, Guozhang Wang wrote:
>
> > Hi Rajasekar,
> >
> > Is auto.create.topics.enable set to true in your
This could be related to https://issues.apache.org/jira/browse/KAFKA-1030
Thanks,
Jun
On Thu, Sep 5, 2013 at 4:39 PM, Rajasekar Elango wrote:
> We having issues that mirormaker not longer replicate newly created topics.
> It continues to replicate data for existing topics and but new topics
>
Yes guozhang
Sent from my iPhone
On Sep 5, 2013, at 7:53 PM, Guozhang Wang wrote:
> Hi Rajasekar,
>
> Is auto.create.topics.enable set to true in your target cluster?
>
> Guozhang
>
>
> On Thu, Sep 5, 2013 at 4:39 PM, Rajasekar Elango
> wrote:
>
>> We having issues that mirormaker not longer r
Hi Rajasekar,
Is auto.create.topics.enable set to true in your target cluster?
Guozhang
On Thu, Sep 5, 2013 at 4:39 PM, Rajasekar Elango wrote:
> We having issues that mirormaker not longer replicate newly created topics.
> It continues to replicate data for existing topics and but new topics
We having issues that mirormaker not longer replicate newly created topics.
It continues to replicate data for existing topics and but new topics
doesn't get created on target cluster. ConsumerOffsetTracker shows that new
topics are successfully consumed and Lag is 0. But those topics doesn't get
c
13 matches
Mail list logo
