Re: Debugging Kafka Streams Windowing

2017-08-03 Thread sahil aggarwal
Face the similar issue in kafka 0.10.0.1. Going through the kafka code figured that when coordinator goes down the other ISR scans whole log file of partition of __consumer_offsets for my consumer group to update the cache of offsets. In my case its size was around ~600G which took around ~40 mins

Consumer group describe issue

2017-12-21 Thread sahil aggarwal
Hi, Facing issue where *kafka-consumer-groups.sh --describe * get stuck if one of the partition is unavailable i.e no leader. Going through some code found that it does following to get log end offset: * Create consumer * For each partition * assign partition * seek to end * get position

Re: Consumer group describe issue

2017-12-22 Thread sahil aggarwal
is correct. KafkaConsumer.position() is a blocking > call. It's a know issue that there is no configurable timeout value. > > I am not aware of any workaround. > > > -Matthias > > On 12/21/17 6:05 AM, sahil aggarwal wrote: > > Hi, > > > > Facing issu

Re: Consumer group describe issue

2017-12-22 Thread sahil aggarwal
0.11.0 branch and trunk for getEndOffsets but > didn't find any occurrence. > > Mind giving us the location (and class) where getEndOffsets is called ? > > Thanks > > On Fri, Dec 22, 2017 at 11:29 PM, sahil aggarwal > wrote: > > > Fixed it by some code chang

Re: Consumer group describe issue

2017-12-25 Thread sahil aggarwal
Attached the patch. If someone can review it will be very helpfull. Thanks, sahil On 23 December 2017 at 13:11, sahil aggarwal wrote: > my bad its consumer.endOffsets in ConsumerGroupCommand.scala. > > https://github.com/apache/kafka/blob/ef97ed7ee5cb883a30245ea4f77ef9 > 9d4db15

Re: Consumer group describe issue

2017-12-27 Thread sahil aggarwal
out of 7 hunks FAILED -- saving rejects to file > core/src/main/scala/kafka/admin/ConsumerGroupCommand.scala.rej > > FYI > > On Mon, Dec 25, 2017 at 9:45 PM, sahil aggarwal > wrote: > > > Attached the patch. If someone can review it will be very helpfull. > > > &

Re: Consumer group describe issue

2018-02-22 Thread sahil aggarwal
Bill, I have created the jira: https://issues.apache.org/jira/browse/KAFKA-6581 but i am not able to assign it to myself. Can you please help? Thanks, Sahil On 28 December 2017 at 11:50, sahil aggarwal wrote: > @TedYu > From 0.10.0 <https://github.com/apache/kafka/tree/0.10.0&g