Re: [DISCUSSION] Boot dependency in the new producer

2015-01-27 Thread Guozhang Wang
Steven, You are right, I was wrong about the previous email: it will not set the flag, the Sender thread will trigger Client.poll() but with Int.Max select time, hence this should not be an issue. I am closing this discussion now. Guozhang On Mon, Jan 26, 2015 at 4:42 PM, Steven Wu wrote: > Ja

Re: [DISCUSSION] Boot dependency in the new producer

2015-01-26 Thread Steven Wu
Jay, I don't think this line will bootstrap full metadata (for all topics). it will just construct the cluster object with bootstrap host. you need to do "metadata.add(topic)" to set interest of a topic's partition metadata. Guozhang, I personally think this is ok. it just do a few DNS lookup or

Re: [DISCUSSION] Boot dependency in the new producer

2015-01-26 Thread Jay Kreps
Oh, yes, I guess I thought you meant that construction of the client would block on the metadata request. I don't personally think that is a problem because if it fails it will retry in the background, right? But actually I think this is probably violating another desirable criteria we had talked

Re: [DISCUSSION] Boot dependency in the new producer

2015-01-26 Thread Guozhang Wang
It will set the needUpdate flag to true and hence the background Sender will try to talk to the bootstrap servers. Guozhang On Mon, Jan 26, 2015 at 1:12 PM, Jay Kreps wrote: > Hey Guozhang, > > That line shouldn't cause any connections to Kafka to be established, does > it? All that is doing is

Re: [DISCUSSION] Boot dependency in the new producer

2015-01-26 Thread Jay Kreps
Hey Guozhang, That line shouldn't cause any connections to Kafka to be established, does it? All that is doing is creating the Cluster pojo using the supplied addresses. The use of InetSocketAddress may cause some dns stuff to happen, though... -Jay On Mon, Jan 26, 2015 at 10:50 AM, Guozhang Wan

[DISCUSSION] Boot dependency in the new producer

2015-01-26 Thread Guozhang Wang
Hi all, I am not sure if we have discussed about this before, but recently I realized that we have introduced boot dependency of the kafka-server specified by the "bootstrap.servers" config in the new producer. More specifically, although in the old producer we also have a similar config for speci