Re: New consumer pause/reset behaviour

2016-01-28 Thread Jason Gustafson
Hey Tom, Yes, it is possible that the poll() will rebalance and resume fetching for a previously paused partition. First thought is to use a ConsumerRebalanceListener to re-pause the partitions after the rebalance completes.The rebalance listener offers two hooks: onPartitionsRevoked() is called b

New consumer pause/reset behaviour

2016-01-28 Thread Tom Dearman
I am trying to enable throttling on a consumer poller. My understanding of the consumer is that it has a single threaded design so that all comms affecting the consumer are sent during a client threads call to poll and commitSynch. We have therefor designed our throttling by using the consumer