> -----Original Message----- > From: Chris Curtin [mailto:curtin.ch...@gmail.com]
> > 1 When you say the iterator may block, do you mean hasNext() may block? > > > > Yes. Is this due to a potential non-blocking fetch (broker/zookeeper returns an empty block if offset is current)? Yet this blocks the network call of the consumer iterator, do I have that right? Are there other reasons it could block? Like the call fails and a backup call is made? > > b. For client crash, what can client do to avoid duplicate messages > > when restarted? What I can think of is to read last message from log > > file and ignore the first few received duplicate messages until > > receiving the last read message. But is it possible for client to read log file > directly? > > > > If you can't tolerate the possibility of duplicates you need to look at the > Simple Consumer example, There you control the offset storage. Do you have example code that manages only once, even when a consumer for a given partition goes away? What does happen with rebalancing when a consumer goes away? Is this behavior of the high-level consumer group? Is there a way to supply one's own simple consumer with only once, within a consumer group that rebalances? What happens if a producer goes away? thanks much, rob
