Re: How To Push Message to Every Partition On A Topic

2015-03-07 Thread Mike Axiak
Hey Alex, Conceptually you aren't sending a single message to all topics, but to be available in all partitions you'd have to send a message for each partition. You can fan out in the client like this though: https://gist.github.com/anonymous/92bb8b788742e95ee2e8 Best, Mike On Sat, Mar 7, 2015 a

Re: Rebalance not happening even after increasing max retries causing conflict in ZK

2015-03-26 Thread Mike Axiak
Hi guys, At HubSpot we think the issue is related to slow consumers. During a rebalance, one of the first things the consumer does is signal a shutdown to the fetcher [1] [2], in order to relinquish ownership of the partitions. This waits for the shutdown of all shutdown fetcher threads, which ca

Re: Rebalance not happening even after increasing max retries causing conflict in ZK

2015-03-26 Thread Mike Axiak
hanks, > > Mayuresh > > On Thu, Mar 26, 2015 at 1:50 PM, Mike Axiak wrote: > > > Hi guys, > > > > At HubSpot we think the issue is related to slow consumers. During a > > rebalance, one of the first things the consumer does is signal a shutdown > > t

Re: Hitting integer limit when setting log segment.bytes

2015-05-13 Thread Mike Axiak
Jay Kreps has commented on this before: https://issues.apache.org/jira/browse/KAFKA-1670?focusedCommentId=14161185&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14161185 Basically, you can always have more segment files. Having too large of segment files will signif