Hm, curious! Would this be useful to contribute upstream?
https://github.com/wikimedia/operations-debs-kafka/blob/debian/debian/bin/kafka Wikimedia uses it instead of the myriad of bin/*.sh scripts that come with Kafka. We didn’t want to build a .deb package that installed 16ish short shell scripts into /usr/sbin. I like it better, because the help message shows the available Kafka CLI commands. It also lets you set a ZOOKEEPER_URL environment variable so you don’t have to pass the —-zookeeper flag with every command. Here’s the usage info: $ kafka --help Usage: kafka <command> [opts] Run kafka <command> with zero arguments/options to see command usage. Commands: kafka create-topic [opts] kafka list-topic [opts] kafka console-producer [opts] kafka console-consumer [opts] kafka simple-consumer-shell [opts] kafka replay-log-producer [opts] kafka mirror-maker [opts] kafka consumer-offset-checker [opts] kafka add-partitions [opts] kafka reassign-partitions [opts] kafka check-reassignment-status [opts] kafka preferred-replica-election [opts] kafka controlled-shutdown [opts] kafka producer-perf-test [opts] kafka consumer-perf-test [opts] kafka simple-consumer-perf-test [opts] kafka server-start <server.properties> (Default: /etc/kafka/server.properties) kafka server-stop kafka zookeeper-start <zookeeper.properties> (Default: /etc/kafka/zookeeper.properties) kafka zookeeper-stop kafka zookeeper-shell [opts] Environment Variables: ZOOKEEPER_URL - If this is set, any commands that take a --zookeeper flag will be passed with this value. KAFKA_CONFIG - location of Kafka config files. Default: /etc/kafka JMX_PORT - Set this to expose JMX. This is set by default for brokers and producers. KAFKA_JVM_PERFORMANCE_OPTS - Any special JVM perfomance options. This is set by default. KAFKA_HEAP_OPTS - Any special JVM memory heap options. This is set by default. KAFKA_LOG4J_OPTS - Any log4j options. Especially -Dlog4j.configuration. This is set by default. KAFKA_OPTS - Any extra options you want to pass. -Ao On Jul 18, 2014, at 6:42 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > Yeah that was more or less what I was proposing. I posted my random > ideas in the other thread, let me know what you think. > > -Jay > > On Fri, Jul 18, 2014 at 3:20 PM, Philip O'Toole > <philip_o_to...@yahoo.com.invalid> wrote: >> That sounds great -- I do think documentation is always very important (I've >> had some ideas, but that's for another time). >> >> I would be very interested in more ideas around what you think is missing >> from the eco-system. That way people get to contribute, but can deepen their >> understanding in their chosen field. For example, I've started writing more >> and more Go recently, and would like to write more. So working with Kafka >> (tools, clients, monitoring), but coding in Go, would be appealing. Others >> will have other preferences, I am sure. >> >> >> Philip >> >> >> --------------------------- >> www.philipotoole.com >> >> >> >> On Friday, July 18, 2014 2:12 PM, Jay Kreps <jay.kr...@gmail.com> wrote: >> >> >> >> Hey Philip, >> >> That is awesome! You actually don't have to write Java or Scala code >> to contribute. >> >> There is the usual thing that presentations (which you are already >> doing), improving website docs, and general community participation >> are all at least as valuable as new code. >> >> However in addition to that, one of the things that actually makes >> Kafka useful is the non-java clients. Actually since our hope is to >> make Kafka effective as a centralized data pipeline, this is one of >> the more critical things (if you can't easily integrate all your >> applications with the pipeline because there isn't a client or it >> isn't good then this falls apart). I think there is a ton of room to >> improve the client ecosystem. I will post some of the ideas I have in >> this area in a separate thread to kick off a discussion. >> >> -Jay >> >> >> On Thu, Jul 17, 2014 at 9:28 PM, Philip O'Toole >> <philip_o_to...@yahoo.com.invalid> wrote: >>> First things first. I friggin' think Kafka rocks. It's a system that have >>> given me a lot of joy, and I've spent a lot of fun hours (and sometimes not >>> so fun) looking at consumer lag metrics. I'd like to give back, beyond >>> spreading the gospel about it architecturally and operationally. >>> >>> >>> My only concern is Scala. I know very little about it, except that it's a >>> JVM-based language. And there's the rub. I really do not like Java. I hate >>> that it's next to impossible to code it without an IDE (so many files, so >>> many Manager classes, so many...). I find Java to have this super high >>> type-to-thought ratio. Would you guys have anything to say about Scala >>> compared to Java? How has your experience been with coding in it, and >>> building large systems with it? >>> >>> >>> Philip >>> >>> >>> >>> On Thursday, July 17, 2014 2:33 PM, Jay Kreps <jay.kr...@gmail.com> wrote: >>> >>> >>> >>> Hey All, >>> >>> Wow, glad that there is so much interest. >>> >>> As people mentioned a good place to get started to kind of learn the >>> basics of the code base are the newbie and newbie++ JIRAs: >>> http://bit.ly/1jR3lyJ >>> >>> If you take on any of these and get stuck we are very happy to help >>> you get unstuck. Feel free to reach out to the mailing list, or, I >>> really like Neha's idea of pairing people up with existing committers. >>> >>> When you feel like you have gotten the basics down of how to >>> contribute a patch and can kind of find your way around the code base, >>> you may be looking for something a little more challenging. There are >>> two approaches here, you may have some existing things you would like >>> to fix or change in Kafka. If this is the case just discuss it on the >>> mailing list or a JIRA, get basic consensus on the approach, and then >>> dive in. For some though they may not have a specific project in mind, >>> if this applies to you we have a huge backlog of good project/feature >>> ideas in mind and can help put something together that is appropriate >>> for your interests, level of expertise, time commitment, etc. >>> >>> Cheers, >>> >>> -Jay >>> >>> >>> On Thu, Jul 17, 2014 at 7:00 AM, Neha Narkhede <neha.narkh...@gmail.com> >>> wrote: >>>> Thanks Jay for bringing this up. A couple things might help - >>>> >>>> 1. Be diligent in marking newbie/newbie++ labels. I've seen us do pretty >>>> well here. >>>> 2. Pair up contributors to committers for a few initial patches to ensure a >>>> smoother ramp up. I've recently done this and have seen it work pretty >>>> well. Happy to help more. >>>> 3. Jay and I talked about ways of improving patch review turnaround time. >>>> Mostly, the problem is that committers are either swamped or not sure which >>>> patches need review. What might work is to assign the JIRA to a committer >>>> for review and have the committer shepherd the patch to checkin and >>>> reassign the JIRA back to the contributor. I can help with triaging and >>>> assigning committers to patch reviews and over time most of the committers >>>> will be able to do this. >>>> >>>> Thanks, >>>> Neha >>>> >>>> >>>> On Wed, Jul 16, 2014 at 10:26 PM, pushkar priyadarshi < >>>> priyadarshi.push...@gmail.com> wrote: >>>> >>>>> I have been using kafka for quite some time now and would really be >>>>> interested to contribute to this awesome code base. >>>>> >>>>> Regards, >>>>> Pushkar >>>>> >>>>> >>>>> On Thu, Jul 17, 2014 at 7:17 AM, Joe Stein <joe.st...@stealth.ly> wrote: >>>>> >>>>>> ./gradlew scaladoc >>>>>> >>>>>> Builds the scala doc, perhaps we can start to publish this again with the >>>>>> next release and link it on the website. For more related check out the >>>>>> README >>>>>> >>>>>> >>>>>> /******************************************* >>>>>> Joe Stein >>>>>> Founder, Principal Consultant >>>>>> Big Data Open Source Security LLC >>>>>> http://www.stealth.ly >>>>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> >>>>>> ********************************************/ >>>>>> >>>>>> >>>>>> On Wed, Jul 16, 2014 at 8:39 PM, hsy...@gmail.com <hsy...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Is there a scala API doc for the entire kafka library? >>>>>>> >>>>>>> >>>>>>> On Wed, Jul 16, 2014 at 5:34 PM, hsy...@gmail.com <hsy...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Jay, >>>>>>>> >>>>>>>> I would like to take a look at the code base and maybe start working >>>>> on >>>>>>>> some jiras. >>>>>>>> >>>>>>>> Best, >>>>>>>> Siyuan >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jul 16, 2014 at 3:09 PM, Jay Kreps <jay.kr...@gmail.com> >>>>>> wrote: >>>>>>>> >>>>>>>>> Hey All, >>>>>>>>> >>>>>>>>> A number of people have been submitting really nice patches >>>>> recently. >>>>>>>>> >>>>>>>>> If you are interested in contributing and are looking for something >>>>> to >>>>>>>>> work on, or if you are contributing and are interested in ramping up >>>>>>>>> to be a committer on the project, please let us know--we are happy >>>>> to >>>>>>>>> help you help us :-). It is often hard to know what JIRAs or >>>>> projects >>>>>>>>> would be good to work on, how hard those will be, and where to get >>>>>>>>> started. Feel free to reach out to me, Neha, Jun, or any of the >>>>> other >>>>>>>>> committers for help with this. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> >>>>>>>>> -Jay >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>
