Disable anonymous access to zookeeper

2017-11-15 Thread Jamie Wang
Hi, I am using Kafka 0.10.0 version. In this version, zookeeper is required. Recently we found by default zookeeper allows anonymous connect to its port and for some this seems to be a security concern. So I'd like to disable zookeeper's ability to support anonymous connect. I am wondering if I

RE: Support https or ssl

2015-05-14 Thread Jamie Wang
. In this manner, just modify my current Kafka produce client to issue REST API calls instead writing to the socket direct. Am I getting this right? Thanks!! Jamie From: Sriharsha Chintalapani [mailto:harsh...@fastmail.fm] Sent: Friday, May 08, 2015 2:55 PM To: Jamie Wang; users@kafka.apache.org

RE: Support https or ssl

2015-05-08 Thread Jamie Wang
available for GA? Thanks again. Jamie -Original Message- From: Sriharsha Chintalapani [mailto:ka...@harsha.io] Sent: Thursday, May 07, 2015 7:22 PM To: Jamie Wang; users@kafka.apache.org Subject: Re: Support https or ssl Hi Jamie,         I am currently working on providing ssl support for

Support https or ssl

2015-05-07 Thread Jamie Wang
Hello, It's been a while since my team worked on kafka related project. Btw, previous project using Kafka worked wonderfully for us. Now I have requirement to use https or SSL. I am wondering if the latest version has support for SSL. If not, what is the timeline this functionality would suppo

Process for accepting code contribution

2013-04-15 Thread Jamie Wang
Does anyone know what is the process for accepting code contribution such as bug fixes, etc? I understand if there's bug and we need to fix, we must publish the code that fixed the bug which is totally fine. But just want to know the exact process and requirement as our legal is asking. Thanks

RE: How do you handle server down in producer

2013-04-08 Thread Jamie Wang
goes down. Instead, engineer your system so your Producers and Kafka Brokers are reliable and redundant -- and well tested. Philip On Thu, Apr 4, 2013 at 2:30 PM, Jamie Wang wrote: > What is the best way to handle kafka server going down during a message > send by a producer? We have seen ou

How do you handle server down in producer

2013-04-04 Thread Jamie Wang
What is the best way to handle kafka server going down during a message send by a producer? We have seen our producer stuck waiting until kafka server comes back. Is there ways the producer can catch a timeout or ways to test if Kafka server is up so to send the message. Having the producer hang

RE: Loading kafka server within a Java process

2013-02-15 Thread Jamie Wang
into any issues doing this > yet, but it's something to be aware of. > > --Eric > > > On Thu, Feb 14, 2013 at 4:39 PM, Jay Kreps wrote: > > > Yes, it will work. We do this at LinkedIn, actually. > > > > -Jay > > > > On Thu, Feb 14, 2013 at 12

RE: Loading kafka server within a Java process

2013-02-14 Thread Jamie Wang
, Jay Kreps wrote: > Yes, it will work. We do this at LinkedIn, actually. > > -Jay > > On Thu, Feb 14, 2013 at 12:15 PM, Jamie Wang > wrote: > > I have a situation where we have constraint on the number of processes > we can use and hence, is it possible to load kafka se

Loading kafka server within a Java process

2013-02-14 Thread Jamie Wang
I have a situation where we have constraint on the number of processes we can use and hence, is it possible to load kafka server within another Java process? Looking at the kafka.kafka it seems that I can create a Java object that does most of what Kafka.kafka.main() do. Any thoughts if this wi

Jconsole not showing all messages on the broker for a partition

2013-01-28 Thread Jamie Wang
Not sure if anyone else ran into this problem in Jconsole. I rely on this tool to see the number of messages sent by the producer and verifies the number of messages to consume by the consumer. Sometimes I found the Jconsole does not display all messages in a topic on the broker after I restar

RE: How do you keep track of offset in a partition

2013-01-28 Thread Jamie Wang
get the earliest/latest offset available on the broker for a particular partition. Thanks, Neha On Mon, Jan 28, 2013 at 5:05 PM, Jamie Wang wrote: > Hi, > > We are using 0.72 version of Kafka on Windows. I am wondering what is the > right way to fetch data and keep track of offset in

RE: Subscription Issue.

2012-12-13 Thread Jamie Wang
The email below from Jun will should help you on the subscription. On Wed, Dec 5, 2012 at 1:18 PM, Jun Rao wrote: > We recently graduated to a top-level Apache project and have new > mailing lists. You need to subscribe to > dev-subscr...@kafka.apache.org > users-subscr...@kafka.apache.org

RE: Design questions and your opinion and suggestions are appreciated

2012-12-13 Thread Jamie Wang
ect to the Kafka port. 4) No. Just do kill -15 5) It depends. You may want to add a broker if you don't have enough storage space, don't have enough I/Os, or don't have enough network bandwidth. Thanks, Jun On Wed, Dec 12, 2012 at 2:25 PM, Jamie Wang wrote: > Hi, > > We

RE: Running Kafka on Windows?

2012-12-05 Thread Jamie Wang
bat file to include the path to the run-class .bat. Jamie -Original Message----- From: Jamie Wang [mailto:jamie.w...@actuate.com] Sent: Wednesday, December 05, 2012 12:33 PM To: users@kafka.apache.org Subject: RE: Running Kafka on Windows? Thanks for all the replies. I downloaded th

RE: Running Kafka on Windows?

2012-12-05 Thread Jamie Wang
e: > > > The windows scripts are only in the 0.8 branch. Not sure if they work for > > the 0.72 release, but you can give it a try. > > > > Thanks, > > > > Jun > > > > On Tue, Dec 4, 2012 at 4:59 PM, Jamie Wang > wrote: > > > >

Running Kafka on Windows?

2012-12-04 Thread Jamie Wang
Hi, I read the literature somewhere that the windows platform is supported for running kafka. But I didn't find any *.bat command files in the 0.72 release I downloaded. I searched the net and found one entry: https://github.com/apache/kafka/commit/ddbedd5214a3e43afedb51ace71b783751ec8434 for

Security measure

2012-11-30 Thread Jamie Wang
Hi, Is there any security enforcement built into Kafka broker that the system can have control who can send and consume messages? I understand kafka is most likely installed/deployed behind firewalls. But even with that, someone with knowledge of topic can potentially sabotage or sniffing in

RE: producer queue size

2012-11-29 Thread Jamie Wang
x27;s log.flush.interval on the broker? For better performance, you need to set it to a few hundreds or more. Thanks, Jun On Wed, Nov 28, 2012 at 7:12 PM, Jamie Wang wrote: > Hi, > > Thanks for all the replies. It helped me understand the system. I > appreciate it. > > I tried change

RE: producer queue size

2012-11-28 Thread Jamie Wang
n Mon, Nov 26, 2012 at 4:06 PM, Joel Koshy wrote: > To use async, set producer.type to async ; The default queue size is 1; > and the default batch size is 200. > > > On Mon, Nov 26, 2012 at 2:28 PM, Jamie Wang wrote: > >> Hi, I am running the console demo comes with