Steven, You are right, I was wrong about the previous email: it will not set the flag, the Sender thread will trigger Client.poll() but with Int.Max select time, hence this should not be an issue. I am closing this discussion now.
Guozhang On Mon, Jan 26, 2015 at 4:42 PM, Steven Wu <stevenz...@gmail.com> wrote: > Jay, I don't think this line will bootstrap full metadata (for all topics). > it will just construct the cluster object with bootstrap host. you need to > do "metadata.add(topic)" to set interest of a topic's partition metadata. > > Guozhang, I personally think this is ok. it just do a few DNS lookup or TCP > connection before first send. > > On Mon, Jan 26, 2015 at 2:07 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > Oh, yes, I guess I thought you meant that construction of the client > would > > block on the metadata request. > > > > I don't personally think that is a problem because if it fails it will > > retry in the background, right? > > > > But actually I think this is probably violating another desirable > criteria > > we had talked about which was keeping the producer from bootstrapping the > > full metadata for all partitions. If it is doing that during construction > > time presumably the resulting metadata request is for all partitions, no? > > That isn't a huge problem, but I think isn't what was intended. > > > > -Jay > > > > On Mon, Jan 26, 2015 at 1:34 PM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > > > It will set the needUpdate flag to true and hence the background Sender > > > will try to talk to the bootstrap servers. > > > > > > Guozhang > > > > > > On Mon, Jan 26, 2015 at 1:12 PM, Jay Kreps <jay.kr...@gmail.com> > wrote: > > > > > > > Hey Guozhang, > > > > > > > > That line shouldn't cause any connections to Kafka to be established, > > > does > > > > it? All that is doing is creating the Cluster pojo using the supplied > > > > addresses. The use of InetSocketAddress may cause some dns stuff to > > > happen, > > > > though... > > > > > > > > -Jay > > > > > > > > On Mon, Jan 26, 2015 at 10:50 AM, Guozhang Wang <wangg...@gmail.com> > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > I am not sure if we have discussed about this before, but recently > I > > > > > realized that we have introduced boot dependency of the > kafka-server > > > > > specified by the "bootstrap.servers" config in the new producer. > More > > > > > specifically, although in the old producer we also have a similar > > > config > > > > > for specifying the broker list, the producer will not try to > connect > > to > > > > > those brokers until the first message send call is triggered; > whereas > > > in > > > > > the new producer, it will try to talk to them in construction time > > via: > > > > > > > > > > update(Cluster.bootstrap(addresses), time.milliseconds()); > > > > > > > > > > > > > > > I personally am neutral to this change, as in most cases the > > > > corresponding > > > > > kafka server should be up and running before the producer clients > are > > > > > deployed, but there are still some corner cases when it is not > true, > > > for > > > > > example some standalone deployment tests of the app embedded with > > some > > > > > clients, etc. So I would like to bring this up to people's > attention > > if > > > > we > > > > > have not discussed about it before: do we think this is OK to > > introduce > > > > > this boot dependency in the new producer? > > > > > > > > > > -- Guozhang > > > > > > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > -- -- Guozhang
