Oh, BTW, I think Yelp is using this .deb packaging (and shell script) too.
On Jul 21, 2014, at 10:16 AM, Andrew Otto <ao...@wikimedia.org> wrote: > Hm, curious! > > Would this be useful to contribute upstream? > > https://github.com/wikimedia/operations-debs-kafka/blob/debian/debian/bin/kafka > > Wikimedia uses it instead of the myriad of bin/*.sh scripts that come with > Kafka. We didn’t want to build a .deb package that installed 16ish short > shell scripts into /usr/sbin. I like it better, because the help message > shows the available Kafka CLI commands. It also lets you set a ZOOKEEPER_URL > environment variable so you don’t have to pass the —-zookeeper flag with > every command. > > Here’s the usage info: > > > $ kafka --help > Usage: > > kafka <command> [opts] > Run kafka <command> with zero arguments/options to see command usage. > > Commands: > kafka create-topic [opts] > kafka list-topic [opts] > > kafka console-producer [opts] > kafka console-consumer [opts] > kafka simple-consumer-shell [opts] > kafka replay-log-producer [opts] > > kafka mirror-maker [opts] > kafka consumer-offset-checker [opts] > > kafka add-partitions [opts] > kafka reassign-partitions [opts] > kafka check-reassignment-status [opts] > kafka preferred-replica-election [opts] > kafka controlled-shutdown [opts] > > kafka producer-perf-test [opts] > kafka consumer-perf-test [opts] > kafka simple-consumer-perf-test [opts] > > kafka server-start <server.properties> (Default: > /etc/kafka/server.properties) > kafka server-stop > > kafka zookeeper-start <zookeeper.properties> (Default: > /etc/kafka/zookeeper.properties) > kafka zookeeper-stop > kafka zookeeper-shell [opts] > > Environment Variables: > ZOOKEEPER_URL - If this is set, any commands that take a > --zookeeper flag will be passed with this value. > KAFKA_CONFIG - location of Kafka config files. Default: > /etc/kafka > JMX_PORT - Set this to expose JMX. This is set by default > for brokers and producers. > KAFKA_JVM_PERFORMANCE_OPTS - Any special JVM perfomance options. This is > set by default. > KAFKA_HEAP_OPTS - Any special JVM memory heap options. This is > set by default. > KAFKA_LOG4J_OPTS - Any log4j options. Especially > -Dlog4j.configuration. This is set by default. > KAFKA_OPTS - Any extra options you want to pass. > > > > > -Ao > > > > On Jul 18, 2014, at 6:42 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > >> Yeah that was more or less what I was proposing. I posted my random >> ideas in the other thread, let me know what you think. >> >> -Jay >> >> On Fri, Jul 18, 2014 at 3:20 PM, Philip O'Toole >> <philip_o_to...@yahoo.com.invalid> wrote: >>> That sounds great -- I do think documentation is always very important >>> (I've had some ideas, but that's for another time). >>> >>> I would be very interested in more ideas around what you think is missing >>> from the eco-system. That way people get to contribute, but can deepen >>> their understanding in their chosen field. For example, I've started >>> writing more and more Go recently, and would like to write more. So working >>> with Kafka (tools, clients, monitoring), but coding in Go, would be >>> appealing. Others will have other preferences, I am sure. >>> >>> >>> Philip >>> >>> >>> --------------------------- >>> www.philipotoole.com >>> >>> >>> >>> On Friday, July 18, 2014 2:12 PM, Jay Kreps <jay.kr...@gmail.com> wrote: >>> >>> >>> >>> Hey Philip, >>> >>> That is awesome! You actually don't have to write Java or Scala code >>> to contribute. >>> >>> There is the usual thing that presentations (which you are already >>> doing), improving website docs, and general community participation >>> are all at least as valuable as new code. >>> >>> However in addition to that, one of the things that actually makes >>> Kafka useful is the non-java clients. Actually since our hope is to >>> make Kafka effective as a centralized data pipeline, this is one of >>> the more critical things (if you can't easily integrate all your >>> applications with the pipeline because there isn't a client or it >>> isn't good then this falls apart). I think there is a ton of room to >>> improve the client ecosystem. I will post some of the ideas I have in >>> this area in a separate thread to kick off a discussion. >>> >>> -Jay >>> >>> >>> On Thu, Jul 17, 2014 at 9:28 PM, Philip O'Toole >>> <philip_o_to...@yahoo.com.invalid> wrote: >>>> First things first. I friggin' think Kafka rocks. It's a system that have >>>> given me a lot of joy, and I've spent a lot of fun hours (and sometimes >>>> not so fun) looking at consumer lag metrics. I'd like to give back, beyond >>>> spreading the gospel about it architecturally and operationally. >>>> >>>> >>>> My only concern is Scala. I know very little about it, except that it's a >>>> JVM-based language. And there's the rub. I really do not like Java. I hate >>>> that it's next to impossible to code it without an IDE (so many files, so >>>> many Manager classes, so many...). I find Java to have this super high >>>> type-to-thought ratio. Would you guys have anything to say about Scala >>>> compared to Java? How has your experience been with coding in it, and >>>> building large systems with it? >>>> >>>> >>>> Philip >>>> >>>> >>>> >>>> On Thursday, July 17, 2014 2:33 PM, Jay Kreps <jay.kr...@gmail.com> wrote: >>>> >>>> >>>> >>>> Hey All, >>>> >>>> Wow, glad that there is so much interest. >>>> >>>> As people mentioned a good place to get started to kind of learn the >>>> basics of the code base are the newbie and newbie++ JIRAs: >>>> http://bit.ly/1jR3lyJ >>>> >>>> If you take on any of these and get stuck we are very happy to help >>>> you get unstuck. Feel free to reach out to the mailing list, or, I >>>> really like Neha's idea of pairing people up with existing committers. >>>> >>>> When you feel like you have gotten the basics down of how to >>>> contribute a patch and can kind of find your way around the code base, >>>> you may be looking for something a little more challenging. There are >>>> two approaches here, you may have some existing things you would like >>>> to fix or change in Kafka. If this is the case just discuss it on the >>>> mailing list or a JIRA, get basic consensus on the approach, and then >>>> dive in. For some though they may not have a specific project in mind, >>>> if this applies to you we have a huge backlog of good project/feature >>>> ideas in mind and can help put something together that is appropriate >>>> for your interests, level of expertise, time commitment, etc. >>>> >>>> Cheers, >>>> >>>> -Jay >>>> >>>> >>>> On Thu, Jul 17, 2014 at 7:00 AM, Neha Narkhede <neha.narkh...@gmail.com> >>>> wrote: >>>>> Thanks Jay for bringing this up. A couple things might help - >>>>> >>>>> 1. Be diligent in marking newbie/newbie++ labels. I've seen us do pretty >>>>> well here. >>>>> 2. Pair up contributors to committers for a few initial patches to ensure >>>>> a >>>>> smoother ramp up. I've recently done this and have seen it work pretty >>>>> well. Happy to help more. >>>>> 3. Jay and I talked about ways of improving patch review turnaround time. >>>>> Mostly, the problem is that committers are either swamped or not sure >>>>> which >>>>> patches need review. What might work is to assign the JIRA to a committer >>>>> for review and have the committer shepherd the patch to checkin and >>>>> reassign the JIRA back to the contributor. I can help with triaging and >>>>> assigning committers to patch reviews and over time most of the committers >>>>> will be able to do this. >>>>> >>>>> Thanks, >>>>> Neha >>>>> >>>>> >>>>> On Wed, Jul 16, 2014 at 10:26 PM, pushkar priyadarshi < >>>>> priyadarshi.push...@gmail.com> wrote: >>>>> >>>>>> I have been using kafka for quite some time now and would really be >>>>>> interested to contribute to this awesome code base. >>>>>> >>>>>> Regards, >>>>>> Pushkar >>>>>> >>>>>> >>>>>> On Thu, Jul 17, 2014 at 7:17 AM, Joe Stein <joe.st...@stealth.ly> wrote: >>>>>> >>>>>>> ./gradlew scaladoc >>>>>>> >>>>>>> Builds the scala doc, perhaps we can start to publish this again with >>>>>>> the >>>>>>> next release and link it on the website. For more related check out the >>>>>>> README >>>>>>> >>>>>>> >>>>>>> /******************************************* >>>>>>> Joe Stein >>>>>>> Founder, Principal Consultant >>>>>>> Big Data Open Source Security LLC >>>>>>> http://www.stealth.ly >>>>>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> >>>>>>> ********************************************/ >>>>>>> >>>>>>> >>>>>>> On Wed, Jul 16, 2014 at 8:39 PM, hsy...@gmail.com <hsy...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Is there a scala API doc for the entire kafka library? >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jul 16, 2014 at 5:34 PM, hsy...@gmail.com <hsy...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Jay, >>>>>>>>> >>>>>>>>> I would like to take a look at the code base and maybe start working >>>>>> on >>>>>>>>> some jiras. >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Siyuan >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jul 16, 2014 at 3:09 PM, Jay Kreps <jay.kr...@gmail.com> >>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hey All, >>>>>>>>>> >>>>>>>>>> A number of people have been submitting really nice patches >>>>>> recently. >>>>>>>>>> >>>>>>>>>> If you are interested in contributing and are looking for something >>>>>> to >>>>>>>>>> work on, or if you are contributing and are interested in ramping up >>>>>>>>>> to be a committer on the project, please let us know--we are happy >>>>>> to >>>>>>>>>> help you help us :-). It is often hard to know what JIRAs or >>>>>> projects >>>>>>>>>> would be good to work on, how hard those will be, and where to get >>>>>>>>>> started. Feel free to reach out to me, Neha, Jun, or any of the >>>>>> other >>>>>>>>>> committers for help with this. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> >>>>>>>>>> -Jay >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >
