kafka repo moved to git

2012-12-14 Thread Jun Rao
Hi, Everyone, We have moved Kafka repo to git. The location of the new repo is at: https://git-wip-us.apache.org/repos/asf/kafka.git Please follow the following instruction on how to use it. https://git-wip-us.apache.org Thanks, Jun

Re: Producer questions and more

2012-12-14 Thread Jay Kreps
1. There are two kinds of limits: per server and overall. The per server limits come from the fact that we use one directory and at least one file per partition-replica. The normal rules of unix filesystem scalability apply. The per server limits can be mitigated by adding more servers. The overall

Re: Producer questions and more

2012-12-14 Thread Johan Lundahl
Thanks for some very helpful answers! 1) Great, our needs are somewhere in the thousands of topics and we could probably scale out the number of servers as needed. 2) The reason I would like to separate out the producer is to have as small and simple a library to integrate in our deployment as po

Re: new website

2012-12-14 Thread Jay Kreps
Okay, I set up redirects. Since our old website was updated via some 20 minute rsync script and since ModRewrite is mind boggling it is kind of hard to say whether people will actually be properly redirected to the new site or whether I just destroyed the old site. We will see in 20 minutes. What

Re: new website

2012-12-14 Thread Johan Lundahl
Hi, I ran FF linkchecker plugin on the new site and found a few things: * api docs in the left menu link to 0.7.1 is that right or is there api docs for 0.7.2? * on http://kafka.apache.org/downloads.html the KEYS link: http://svn.apache.org/repos/asf/incubator/kafka/KEYS gives 404 * on ht

Re: Broker Interface Binding

2012-12-14 Thread Matan Amir
While hostname is useful to direct traffic via Zookeeper to specific hostname/IP it does not cause Kafka to only listen on that IP. It will still bind to all interfaces on the box. I guess the solution is to firewall it for now... On Thu, Dec 13, 2012 at 10:16 PM, Jun Rao wrote: > In 0.7, yo

Re: new website

2012-12-14 Thread David Arthur
On 12/14/12 4:30 PM, Jay Kreps wrote: http://kafka.apache.org/ I will set up redirects from the old to the new. If folks could sanity check the links, email addresses, english, and so on that would be swell. -Jay Small nit - Flume has graduated and has a new webpage http://flume.apache.org/

Re: Producer questions and more

2012-12-14 Thread David Arthur
7) Perhaps a dung beetle should be the logo, as featured in The Metamorphosis. Or maybe just a nice stylized version of the word "Kafka" (like Solr and Lucene). ** On 12/14/12 4:58 PM, Johan Lundahl wrote: Thanks for some very helpful answers! 1) Great, our needs are somewhere in the thousand

Re: Producer questions and more

2012-12-14 Thread David Arthur
7) Something like: http://i.imgur.com/R21iF.png ? On 12/14/12 10:04 PM, David Arthur wrote: 7) Perhaps a dung beetle should be the logo, as featured in The Metamorphosis. Or maybe just a nice stylized version of the word "Kafka" (like Solr and Lucene). On 12/14/12 4:58 PM, Johan Lundahl wrote

Question on ZookeeperConsumer resilience when kafka broker fault

2012-12-14 Thread Bae, Jae Hyeon
Hi When one of the broker instances is dead, I can see the producer can acknowledge the dead broker and refresh its producer pool not to send the data to the dead broker. But I don't find any code in ZookeeperConsumerConnector. As I guess, ZookeeprConsumerConnector should do syncRebalance when kaf