Re: Kafka server start from resource negotiator(s)

2014-01-22 Thread Joe Stein
Hi Jun, see inline On Tue, Jan 21, 2014 at 1:14 PM, Jun Rao wrote: > Hi, Joe, > > Thanks for the update. This looks great. A couple of questions and > comments. > > 1. How general is the framework? For example, if I want to add the support > for Apache Yarn, is anything there can be reused? > I

Re: Kafka server start from resource negotiator(s)

2014-01-21 Thread Jun Rao
Hi, Joe, Thanks for the update. This looks great. A couple of questions and comments. 1. How general is the framework? For example, if I want to add the support for Apache Yarn, is anything there can be reused? 2. My current feeling is that it's probably better to maintain those container suppor

Re: Kafka server start from resource negotiator(s)

2014-01-18 Thread Joe Stein
I got through some good research and testing this week on getting Kafka to run on Mesos. I updated the Mesos ticket https://issues.apache.org/jira/browse/KAFKA-1207with some thoughts. Next step I am going to draw out some diagrams and update the wiki page. I was thinking of writing this in the c

Re: Kafka server start from resource negotiator(s)

2014-01-15 Thread Joe Stein
Right now I am testing it that way. Once it is working and if that becomes a problem then moving it into a mesosFramework project is no problem. if we do that it would be beneficial if they were organized under some resource manager folder or something. The way I wrote https://issues.apache.org/

Re: Kafka server start from resource negotiator(s)

2014-01-15 Thread Jun Rao
Hi, Joe, Thanks for looking into this. Do you plan to add a dependency to Mesos in kafka jar? Jun On Wed, Jan 15, 2014 at 10:50 AM, Joe Stein wrote: > Hi, I am working at adding some code to the kafka.Kafka.scala file. > > What I was thinking for the Utils.loadProps is to create some > Resour

Kafka server start from resource negotiator(s)

2014-01-15 Thread Joe Stein
Hi, I am working at adding some code to the kafka.Kafka.scala file. What I was thinking for the Utils.loadProps is to create some ResourceNegotiatedConfiguration class or something to sit with it so various frameworks can be supported. For Mesos I am thinking of storing the properties in zookeepe