any response would be appreciated.
>
>
> -- Forwarded message --
> From: safique ahemad
> Date: Thu, Jun 9, 2016 at 11:18 AM
> Subject: Re: Kafka take too long to update the client with metadata when a
> broker is gone
> To: users@kafka.apac
Hello guys,
Below is the link where kafka logs can be seens with TRACE enabled.
https://drive.google.com/file/d/0B-nANlrsm5ogQkh1NUR2UHYtbkU/view?usp=sharing
I have truncated log as it was very big but it has all the cover of the
time of problem.
Scenario:
1) There were three kafka running i.e.
Hi Gerard,
When trying to reproduce this, did you use the go sarama client Safique
mentioned?
On Fri, Jun 3, 2016 at 5:10 AM, Gerard Klijs
wrote:
> I asume you use a replication factor of 3 for the topics? When I ran some
> test with producer/consumers in a dockerized setup, there where only f
I asume you use a replication factor of 3 for the topics? When I ran some
test with producer/consumers in a dockerized setup, there where only few
failures before the producer switched to to correct new broker again. I
don't know the exact time, but seemed like a few seconds at max, this was
with w
Hi Steve,
There is no way to access that from public side so I won't be able to do
that. Sorry for that.
But the step is quite simple. The only difference is that we have deployed
Kafka cluster using mesos url.
1) launch 3 Kafka broker cluster and create a topic with multiple
partitions at least
I see. I'm not sure if this is a known issue. Do you mind share the
brokers/topics setup and the steps to reproduce this issue?
Cheers, Steve
On Fri, Jun 3, 2016, 9:45 AM safique ahemad wrote:
> you got it right...
>
> But DialTimeout is not a concern here. Client try fetching metadata from
>
you got it right...
But DialTimeout is not a concern here. Client try fetching metadata from
Kafka brokers but Kafka give them stale metadata near 30-40 sec.
It try to fetch 3-4 time in between until it get updated metadata.
This is completely different problem than
https://github.com/Shopify/sara
So you are coming from https://github.com/Shopify/sarama/issues/661 ,
right? I'm not sure if anything from broker side can help but looks like
you already found DialTimeout on client side can help?
Cheers, Steve
On Fri, Jun 3, 2016, 8:33 AM safique ahemad wrote:
> kafka version:0.9.0.0
> go s
kafka version:0.9.0.0
go sarama client version: 1.8
On Thu, Jun 2, 2016 at 5:14 PM, Steve Tian wrote:
> Client version?
>
> On Fri, Jun 3, 2016, 4:44 AM safique ahemad wrote:
>
> > Hi All,
> >
> > We are using Kafka broker cluster in our data center.
> > Recently, It is realized that when a Kaf
Client version?
On Fri, Jun 3, 2016, 4:44 AM safique ahemad wrote:
> Hi All,
>
> We are using Kafka broker cluster in our data center.
> Recently, It is realized that when a Kafka broker goes down then client try
> to refresh the metadata but it get stale metadata upto near 30 seconds.
>
> After
Hi All,
We are using Kafka broker cluster in our data center.
Recently, It is realized that when a Kafka broker goes down then client try
to refresh the metadata but it get stale metadata upto near 30 seconds.
After near 30-35 seconds, updated metadata is obtained by client. This is
really a larg
11 matches
Mail list logo
