Re: Kafka Command Line & Shell

2014-11-12 Thread Guozhang Wang
Thanks Joe. I will read the wiki page. On Tue, Nov 11, 2014 at 11:47 PM, Joe Stein wrote: > I started writing this up on the wiki > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements > > Instead of starting a new thread I figure just continue this one

Re: Kafka Command Line & Shell

2014-11-11 Thread Joe Stein
I started writing this up on the wiki https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements Instead of starting a new thread I figure just continue this one I started. I also added another (important) component for centralized management of configuration as

[jira] [Updated] (KAFKA-1694) kafka command line shell

2014-11-11 Thread Joe Stein (JIRA)
consumer lag trending between offset committed and log end offset. kafka>use topic name; kafka>stats; > kafka command line shell > > > Key: KAFKA-1694 > URL: https://issues.apache.org/jira/browse/KAFKA-169

Re: Kafka Command Line & Shell

2014-10-19 Thread Guozhang Wang
One concern I have is that using RQ/RP wire protocol to the controller instead of the current way (via ZK admin path) may expose concurrency on the admin requests, which may not be supported yet. Some initial discussion about this is on KAFKA-1305. Guozhang On Sun, Oct 19, 2014 at 1:55 PM, Joe S

Re: Kafka Command Line & Shell

2014-10-19 Thread Joe Stein
Maybe we should add some AdminMessage RQ/RP wire protocol structure(s) and let the controller handle it? We could then build the CLI and Shell in the project both as useful tools and samples for others. Making a http interface should be simple after KAFKA-1494 is done which all client libraries co

Re: Kafka Command Line & Shell

2014-10-19 Thread Harsha
+1 for Web Api On Sat, Oct 18, 2014, at 11:48 PM, Glen Mazza wrote: > Apache Karaf has been doing this for quite a few years, albeit in Java > not Scala. Still, their coding approach to creating a CLI probably > captures many lessons learned over that time. > > Glen > > On 10/17/2014 08:03 PM

Re: Kafka Command Line & Shell

2014-10-18 Thread Glen Mazza
Apache Karaf has been doing this for quite a few years, albeit in Java not Scala. Still, their coding approach to creating a CLI probably captures many lessons learned over that time. Glen On 10/17/2014 08:03 PM, Joe Stein wrote: Hi, I have been thinking about the ease of use for operations

Re: Kafka Command Line & Shell

2014-10-17 Thread gshapira
+1 to http interface and cli. Http layer will make it easier to integrate with gui like Hue. Gwen — Sent from Mailbox On Fri, Oct 17, 2014 at 10:14 PM, Todd Palino wrote: > Absolutely. My suggestion of an HTTP interface was in addition to a CLI. I > think the CLI should can the HTTP interfac

Re: Kafka Command Line & Shell

2014-10-17 Thread Todd Palino
Absolutely. My suggestion of an HTTP interface was in addition to a CLI. I think the CLI should can the HTTP interface underneath to keep it simple. -Todd > On Oct 17, 2014, at 6:24 PM, Joel Koshy wrote: > > +1 > It would definitely be useful to have a CLI. We had a cursory > discussion on t

Re: Kafka Command Line & Shell

2014-10-17 Thread Joel Koshy
+1 It would definitely be useful to have a CLI. We had a cursory discussion on this in the past [1] but it would be useful to have a full proposal describing everything the CLI should provide. [1] http://grokbase.com/t/kafka/dev/1435tr3pfc/command-line-tools On Fri, Oct 17, 2014 at 05:12:16PM -

Re: Kafka Command Line & Shell

2014-10-17 Thread Colin
+100 -- Colin Clark +1-320-221-9531 > On Oct 17, 2014, at 7:10 PM, Steve Morin wrote: > > Joe I think this is great! > >> On Fri, Oct 17, 2014 at 5:03 PM, Joe Stein wrote: >> >> Hi, I have been thinking about the ease of use for operations with Kafka. >> We have lots of tools doing a lot

Re: Kafka Command Line & Shell

2014-10-17 Thread Todd Palino
We've been talking about this a little internally as well. What about the idea of presenting all the admin functions through a web API interface (restful or not) complete with authentication? That would make it much easier for creating structure around Kafka without having to layer commands on t

Re: Kafka Command Line & Shell

2014-10-17 Thread Steve Morin
Joe I think this is great! On Fri, Oct 17, 2014 at 5:03 PM, Joe Stein wrote: > Hi, I have been thinking about the ease of use for operations with Kafka. > We have lots of tools doing a lot of different things and they are all kind > of in different places. > > So, what I was thinking is to have

Kafka Command Line & Shell

2014-10-17 Thread Joe Stein
Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This

[jira] [Updated] (KAFKA-1694) kafka command line shell

2014-10-09 Thread Joe Stein (JIRA)
c_security['pci','profile','dss'] = true ...etc An important item is that folks that are using existing tools we should have an easy api type way so they can keep doing that and get benefit too. This interface should also hav

[jira] [Updated] (KAFKA-1694) kafka command line shell

2014-10-09 Thread Joe Stein (JIRA)
that are using existing tools we should have an easy api type way so they can keep doing that and get benefit too. > kafka command line shell > > > Key: KAFKA-1694 > URL: https://issues.apache.org/jira/browse/KAFKA-

[jira] [Created] (KAFKA-1694) kafka command line shell

2014-10-09 Thread Joe Stein (JIRA)
Joe Stein created KAFKA-1694: Summary: kafka command line shell Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe