Sorry, I meant creating a new producer, not consumer.
Here's the code.
Producer - http://pastebin.com/Kqq1ymCX
Consumer - http://pastebin.com/i2Z8PTYB
Callback - http://pastebin.com/x253z7bG
As you'll notice, I am creating a new producer for each message. So the
bootstrap nodes should be refresh
Rahul, the mailing list filters attachments, you'd have to post the code
somewhere else for people to be able to see it.
But I don't think anyone suggested that creating a new consumer would fix
anything. Creating a new producer *and discarding the old one* basically
just makes it start from scrat
Creating a new consumer instance *does not* solve this problem.
Attaching the producer/consumer code that I used for testing.
On Wed, May 6, 2015 at 6:31 AM, Ewen Cheslack-Postava
wrote:
> I'm not sure about the old producer behavior in this same failure scenario,
> but creating a new produce
I'm not sure about the old producer behavior in this same failure scenario,
but creating a new producer instance would resolve the issue since it would
start with the list of bootstrap nodes and, assuming at least one of them
was up, it would be able to fetch up to date metadata.
On Tue, May 5, 20
Can you clarify, is this issue here specific to the "new" producer? With
the "old" producer, we routinely construct a new producer which makes a
fresh metadata request (via a VIP connected to all nodes in the cluster).
Would this approach work with the new producer?
Jason
On Tue, May 5, 2015 at
Mayuresh,
I was testing this in a development environment and manually brought down a
node to simulate this. So the dead node never came back up.
My colleague and I were able to consistently see this behaviour several
times during the testing.
On 5 May 2015 20:32, "Mayuresh Gharat" wrote:
> I ag
I agree that to find the least Loaded node the producer should fall back to
the bootstrap nodes if its not able to connect to any nodes in the current
metadata. That should resolve this.
Rahul, I suppose the problem went off because the dead node in your case
might have came back up and allowed fo
We observed the exact same error. Not very clear about the root cause
although it appears to be related to leastLoadedNode implementation.
Interestingly, the problem went away by increasing the value of
reconnect.backoff.ms to 1000ms.
On 29 Apr 2015 00:32, "Ewen Cheslack-Postava" wrote:
> Ok, all
Ok, all of that makes sense. The only way to possibly recover from that
state is either for K2 to come back up allowing the metadata refresh to
eventually succeed or to eventually try some other node in the cluster.
Reusing the bootstrap nodes is one possibility. Another would be for the
client to
Hi Ewen,
Thanks for the response. I agree with you, In some case we should use
bootstrap servers.
>
> If you have logs at debug level, are you seeing this message in between the
> connection attempts:
>
> Give up sending metadata request since no node is available
>
Yes, this log came for co
Maybe add this to the description of
https://issues.apache.org/jira/browse/KAFKA-1843 ? I can't find it now, but
I think there was another bug where I described a similar problem -- in
some cases it makes sense to fall back to the list of bootstrap nodes
because you've gotten into a bad state and c
Any comments on this issue?
On Apr 24, 2015 8:05 PM, "Manikumar Reddy" wrote:
> We are testing new producer on a 2 node cluster.
> Under some node failure scenarios, producer is not able
> to update metadata.
>
> Steps to reproduce
> 1. form a 2 node cluster (K1, K2)
> 2. create a topic with sing
12 matches
Mail list logo
