Re: join all three groups

2013-02-14 Thread David Arthur
You need to send an email to: users-subscr...@kafka.apache.org dev-subscr...@kafka.apache.org commits-subscr...@kafka.apache.org in order to be subscribed to the lists Cheers, David On 2/13/13 9:22 PM, Sining Ma wrote: Hi, I am using kafka API right now. I need to join all these groups so

Async Producer (0.7 version) quick question

2013-02-14 Thread Subhash Agrawal
Hi, We are using kafka-broker 0.7 version and producer Java API to send messages to log server asynchronously. I am interested to know what happens to messages in producer queue if we do hard kill on producer or if we call close function on producer. With default implementation, Does it write t

Loading kafka server within a Java process

2013-02-14 Thread Jamie Wang
I have a situation where we have constraint on the number of processes we can use and hence, is it possible to load kafka server within another Java process? Looking at the kafka.kafka it seems that I can create a Java object that does most of what Kafka.kafka.main() do. Any thoughts if this wi

Re: Loading kafka server within a Java process

2013-02-14 Thread Jay Kreps
Yes, it will work. We do this at LinkedIn, actually. -Jay On Thu, Feb 14, 2013 at 12:15 PM, Jamie Wang wrote: > I have a situation where we have constraint on the number of processes we can > use and hence, is it possible to load kafka server within another Java > process? Looking at the kafka

Re: Loading kafka server within a Java process

2013-02-14 Thread Eric Tschetter
Jamie, As Jay says, this is definitely possible. We also do it at Metamarkets. The one thing that we found odd when doing it though is that you cannot (or couldn't with the version we are using, I'm not sure about 0.8) embed a producer that produces for an embedded broker. You instead have to h

RE: Loading kafka server within a Java process

2013-02-14 Thread Jamie Wang
Thank you both for the info. I'll keep my eye open since I do have a need to embed a consumer in the same process. Jamie -Original Message- From: Eric Tschetter [mailto:ched...@metamarkets.com] Sent: Thursday, February 14, 2013 2:44 PM To: users@kafka.apache.org Subject: Re: Loading kaf

Re: Loading kafka server within a Java process

2013-02-14 Thread Eric Tschetter
Consumer + broker embedding can be done, I believe. It's just producer+broker embedding that they don't like :). --Eric On Thu, Feb 14, 2013 at 4:51 PM, Jamie Wang wrote: > Thank you both for the info. I'll keep my eye open since I do have a need > to embed a consumer in the same process. > >

Re: Async Producer (0.7 version) quick question

2013-02-14 Thread Jun Rao
The data in the producer is only kept in memory. In a clean shutdown, the producer will drain the queue and send all remaining messages. In an unclean shutdown, all unsent messages are lost. Thanks, Jun On Thu, Feb 14, 2013 at 12:09 PM, Subhash Agrawal wrote: > Hi, > > We are using kafka-broker