Oh, BTW, I think Yelp is using this .deb packaging (and shell script) too.

On Jul 21, 2014, at 10:16 AM, Andrew Otto <ao...@wikimedia.org> wrote:

> Hm, curious!
> 
> Would this be useful to contribute upstream?
> 
> https://github.com/wikimedia/operations-debs-kafka/blob/debian/debian/bin/kafka
> 
> Wikimedia uses it instead of the myriad of bin/*.sh scripts that come with 
> Kafka.  We didn’t want to build a .deb package that installed 16ish short 
> shell scripts into /usr/sbin.  I like it better, because the help message 
> shows the available Kafka CLI commands.  It also lets you set a ZOOKEEPER_URL 
> environment variable so you don’t have to pass the —-zookeeper flag with 
> every command.
> 
> Here’s the usage info:
> 
> 
> $ kafka --help
> Usage:
> 
> kafka <command> [opts]
> Run kafka <command> with zero arguments/options to see command usage.
> 
> Commands:
>  kafka create-topic               [opts]
>  kafka list-topic                 [opts]
> 
>  kafka console-producer           [opts]
>  kafka console-consumer           [opts]
>  kafka simple-consumer-shell      [opts]
>  kafka replay-log-producer        [opts]
> 
>  kafka mirror-maker               [opts]
>  kafka consumer-offset-checker    [opts]
> 
>  kafka add-partitions             [opts]
>  kafka reassign-partitions        [opts]
>  kafka check-reassignment-status  [opts]
>  kafka preferred-replica-election [opts]
>  kafka controlled-shutdown        [opts]
> 
>  kafka producer-perf-test         [opts]
>  kafka consumer-perf-test         [opts]
>  kafka simple-consumer-perf-test  [opts]
> 
>  kafka server-start               <server.properties> (Default: 
> /etc/kafka/server.properties)
>  kafka server-stop
> 
>  kafka zookeeper-start            <zookeeper.properties> (Default: 
> /etc/kafka/zookeeper.properties)
>  kafka zookeeper-stop
>  kafka zookeeper-shell            [opts]
> 
> Environment Variables:
>  ZOOKEEPER_URL              - If this is set, any commands that take a 
> --zookeeper flag will be passed with this value.
>  KAFKA_CONFIG               - location of Kafka config files.  Default: 
> /etc/kafka
>  JMX_PORT                   - Set this to expose JMX.  This is set by default 
> for brokers and producers.
>  KAFKA_JVM_PERFORMANCE_OPTS - Any special JVM perfomance options.  This is 
> set by default.
>  KAFKA_HEAP_OPTS            - Any special JVM memory heap options.  This is 
> set by default.
>  KAFKA_LOG4J_OPTS           - Any log4j options.  Especially 
> -Dlog4j.configuration.  This is set by default.
>  KAFKA_OPTS                 - Any extra options you want to pass.
> 
> 
> 
> 
> -Ao
> 
> 
> 
> On Jul 18, 2014, at 6:42 PM, Jay Kreps <jay.kr...@gmail.com> wrote:
> 
>> Yeah that was more or less what I was proposing. I posted my random
>> ideas in the other thread, let me know what you think.
>> 
>> -Jay
>> 
>> On Fri, Jul 18, 2014 at 3:20 PM, Philip O'Toole
>> <philip_o_to...@yahoo.com.invalid> wrote:
>>> That sounds great -- I do think documentation is always very important 
>>> (I've had some ideas, but that's for another time).
>>> 
>>> I would be very interested in more ideas around what you think is missing 
>>> from the eco-system. That way people get to contribute, but can deepen 
>>> their understanding in their chosen field. For example, I've started 
>>> writing more and more Go recently, and would like to write more. So working 
>>> with Kafka (tools, clients, monitoring), but coding in Go, would be 
>>> appealing. Others will have other preferences, I am sure.
>>> 
>>> 
>>> Philip
>>> 
>>> 
>>> ---------------------------
>>> www.philipotoole.com
>>> 
>>> 
>>> 
>>> On Friday, July 18, 2014 2:12 PM, Jay Kreps <jay.kr...@gmail.com> wrote:
>>> 
>>> 
>>> 
>>> Hey Philip,
>>> 
>>> That is awesome! You actually don't have to write Java or Scala code
>>> to contribute.
>>> 
>>> There is the usual thing that presentations (which you are already
>>> doing), improving website docs, and general community participation
>>> are all at least as valuable as new code.
>>> 
>>> However in addition to that, one of the things that actually makes
>>> Kafka useful is the non-java clients. Actually since our hope is to
>>> make Kafka effective as a centralized data pipeline, this is one of
>>> the more critical things (if you can't easily integrate all your
>>> applications with the pipeline because there isn't a client or it
>>> isn't good then this falls apart). I think there is a ton of room to
>>> improve the client ecosystem. I will post some of the ideas I have in
>>> this area in a separate thread to kick off a discussion.
>>> 
>>> -Jay
>>> 
>>> 
>>> On Thu, Jul 17, 2014 at 9:28 PM, Philip O'Toole
>>> <philip_o_to...@yahoo.com.invalid> wrote:
>>>> First things first. I friggin' think Kafka rocks. It's a system that have 
>>>> given me a lot of joy, and I've spent a lot of fun hours (and sometimes 
>>>> not so fun) looking at consumer lag metrics. I'd like to give back, beyond 
>>>> spreading the gospel about it architecturally and operationally.
>>>> 
>>>> 
>>>> My only concern is Scala. I know very little about it, except that it's a 
>>>> JVM-based language. And there's the rub. I really do not like Java. I hate 
>>>> that it's next to impossible to code it without an IDE (so many files, so 
>>>> many Manager classes, so many...). I find Java to have this super high 
>>>> type-to-thought ratio. Would you guys have anything to say about Scala 
>>>> compared to Java? How has your experience been with coding in it, and 
>>>> building large systems with it?
>>>> 
>>>> 
>>>> Philip
>>>> 
>>>> 
>>>> 
>>>> On Thursday, July 17, 2014 2:33 PM, Jay Kreps <jay.kr...@gmail.com> wrote:
>>>> 
>>>> 
>>>> 
>>>> Hey All,
>>>> 
>>>> Wow, glad that there is so much interest.
>>>> 
>>>> As people mentioned a good place to get started to kind of learn the
>>>> basics of the code base are the newbie and newbie++ JIRAs:
>>>> http://bit.ly/1jR3lyJ
>>>> 
>>>> If you take on any of these and get stuck we are very happy to help
>>>> you get unstuck. Feel free to reach out to the mailing list, or, I
>>>> really like Neha's idea of pairing people up with existing committers.
>>>> 
>>>> When you feel like you have gotten the basics down of how to
>>>> contribute a patch and can kind of find your way around the code base,
>>>> you may be looking for something a little more challenging. There are
>>>> two approaches here, you may have some existing things you would like
>>>> to fix or change in Kafka. If this is the case just discuss it on the
>>>> mailing list or a JIRA, get basic consensus on the approach, and then
>>>> dive in. For some though they may not have a specific project in mind,
>>>> if this applies to you we have a huge backlog of good project/feature
>>>> ideas in mind and can help put something together that is appropriate
>>>> for your interests, level of expertise, time commitment, etc.
>>>> 
>>>> Cheers,
>>>> 
>>>> -Jay
>>>> 
>>>> 
>>>> On Thu, Jul 17, 2014 at 7:00 AM, Neha Narkhede <neha.narkh...@gmail.com> 
>>>> wrote:
>>>>> Thanks Jay for bringing this up. A couple things might help -
>>>>> 
>>>>> 1. Be diligent in marking newbie/newbie++ labels. I've seen us do pretty
>>>>> well here.
>>>>> 2. Pair up contributors to committers for a few initial patches to ensure 
>>>>> a
>>>>> smoother ramp up. I've recently done this and have seen it work pretty
>>>>> well. Happy to help more.
>>>>> 3. Jay and I talked about ways of improving patch review turnaround time.
>>>>> Mostly, the problem is that committers are either swamped or not sure 
>>>>> which
>>>>> patches need review. What might work is to assign the JIRA to a committer
>>>>> for review and have the committer shepherd the patch to checkin and
>>>>> reassign the JIRA back to the contributor. I can help with triaging and
>>>>> assigning committers to patch reviews and over time most of the committers
>>>>> will be able to do this.
>>>>> 
>>>>> Thanks,
>>>>> Neha
>>>>> 
>>>>> 
>>>>> On Wed, Jul 16, 2014 at 10:26 PM, pushkar priyadarshi <
>>>>> priyadarshi.push...@gmail.com> wrote:
>>>>> 
>>>>>> I have been using kafka for quite some time now and would really be
>>>>>> interested to contribute to this awesome code base.
>>>>>> 
>>>>>> Regards,
>>>>>> Pushkar
>>>>>> 
>>>>>> 
>>>>>> On Thu, Jul 17, 2014 at 7:17 AM, Joe Stein <joe.st...@stealth.ly> wrote:
>>>>>> 
>>>>>>> ./gradlew scaladoc
>>>>>>> 
>>>>>>> Builds the scala doc, perhaps we can start to publish this again with 
>>>>>>> the
>>>>>>> next release and link it on the website.  For more related check out the
>>>>>>> README
>>>>>>> 
>>>>>>> 
>>>>>>> /*******************************************
>>>>>>> Joe Stein
>>>>>>> Founder, Principal Consultant
>>>>>>> Big Data Open Source Security LLC
>>>>>>> http://www.stealth.ly
>>>>>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>>>>>> ********************************************/
>>>>>>> 
>>>>>>> 
>>>>>>> On Wed, Jul 16, 2014 at 8:39 PM, hsy...@gmail.com <hsy...@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Is there a scala API doc for the entire kafka library?
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Wed, Jul 16, 2014 at 5:34 PM, hsy...@gmail.com <hsy...@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi Jay,
>>>>>>>>> 
>>>>>>>>> I would like to take a look at the code base and maybe start working
>>>>>> on
>>>>>>>>> some jiras.
>>>>>>>>> 
>>>>>>>>> Best,
>>>>>>>>> Siyuan
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Wed, Jul 16, 2014 at 3:09 PM, Jay Kreps <jay.kr...@gmail.com>
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hey All,
>>>>>>>>>> 
>>>>>>>>>> A number of people have been submitting really nice patches
>>>>>> recently.
>>>>>>>>>> 
>>>>>>>>>> If you are interested in contributing and are looking for something
>>>>>> to
>>>>>>>>>> work on, or if you are contributing and are interested in ramping up
>>>>>>>>>> to be a committer on the project, please let us know--we are happy
>>>>>> to
>>>>>>>>>> help you help us :-). It is often hard to know what JIRAs or
>>>>>> projects
>>>>>>>>>> would be good to work on, how hard those will be, and where to get
>>>>>>>>>> started. Feel free to reach out to me, Neha, Jun, or any of the
>>>>>> other
>>>>>>>>>> committers for help with this.
>>>>>>>>>> 
>>>>>>>>>> Cheers,
>>>>>>>>>> 
>>>>>>>>>> -Jay
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
> 

Reply via email to