Seems like this aspect of Daniel's request could (would be) included here pretty easily:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements Jeff On Sat, Nov 15, 2014 at 3:29 AM, Daniel Compton <d...@danielcompton.net> wrote: > This has been covered in passing in the preceding threads but I'd like to > make it explicit, can we have a command line option/script for getting the > Kafka version (probably with Scala version too)? > > I ran into this recently, where we wanted to verify that the right version > of Kafka had been installed on a number of machines. I ended up getting the > version number from the bundled jars but it wasn't ideal. Being able to run > kafka --version would have been great. > > --- > Daniel > > > On 15/11/2014, at 8:33 am, Magnus Edenhill <mag...@edenhill.se> wrote: > > > > Hi Gwen, > > > > the protocol version in an InfoResponse message would be the maximum > > supported mainline protocol version, each change to the > > effective protocol specification bumps the version by one. > > > > > > The current per-request-type version ID is unfortunately useless to the > > client, it only allows the broker to act differently > > on different versions of a known request type. > > > > Since the broker currently does not handle unknown request types > gracefully > > - it simply disconnects the client, which from the client's perspective > > could mean anything and nothing - the addition of an InfoRequest/Respones > > would allow the client to know if a desired broker-feature is available > or > > not and thus allows the client library to provide the application (and in > > the end the user) with usable error message (compare "Broker > disconnected" > > to "offsetCommit not supported by 0.8.0 broker"). > > Having the broker return a fabricated response with the header.err field > > set to UNKNOWN_REQUEST_TYPE would be very useful as well of course. > > > > > > Regards, > > Magnus > > > > > > > > 2014-11-14 18:17 GMT+01:00 Gwen Shapira <gshap...@cloudera.com>: > > > >> I'm not sure there's a single "protocol version". Each > >> request/response has its own version ID embedded in the request > >> itself. > >> > >> Broker version, broker ID and (as Joel suggested) git hash are all > >> reasonable. And +1 for adding this to API to support non-JVM clients. > >> > >> Gwen > >> > >> On Fri, Nov 14, 2014 at 8:46 AM, Magnus Edenhill <mag...@edenhill.se> > >> wrote: > >>> Please let's not add dependencies on more third party > protocols/software, > >>> the move away from Zookeeper dependence on the clients is very much > >> welcomed > >>> and it'd be a pity to see a new foreign dependency added for such a > >> trivial > >>> thing > >>> as propagating the version of a broker to its client. > >>> > >>> My suggestion is to add a new protocol request which returns: > >>> - broker version > >>> - protocol version > >>> - broker id > >>> > >>> A generic future-proof solution would be the use of tags (or named > TLVs): > >>> InfoResponse: [InfoTag] > >>> InfoTag: > >>> intX tag ( KAFKA_TAG_BROKER_VERSION, KAFKA_TAG_PROTO_VERSION, > >>> KAFKA_TAG_SSL, ... ) > >>> intX type ( KAFKA_TYPE_STR, KAFKA_TYPE_INT32, KAFKA_TYPE_INT64, > >> ...) > >>> intX len > >>> bytes payload > >>> > >>> Local site/vendor tags could be defined in the configuration file, > >>> > >>> > >>> This would also allow clients to enable/disable features based on > >> protocol > >>> version, > >>> e.g., there is no point in trying offsetCommit on a 0.8.0 broker and > the > >>> client library > >>> can inform the application about this early, rather than having its > >>> offsetCommit requests > >>> fail by connection teardown (which is not much of an error > propagation). > >>> > >>> > >>> My two cents, > >>> Magnus > >>> > >>> > >>> 2014-11-12 20:11 GMT+01:00 Mark Roberts <wiz...@gmail.com>: > >>> > >>>> I haven't worked much with JMX before, but some quick googling (10-20 > >>>> minutes) is very inconclusive as to how I would go about getting the > >> server > >>>> version I'm connecting to from a Python client. Can someone please > >>>> reassure me that it's relatively trivial for non Java clients to query > >> JMX > >>>> for the server version of every server in the cluster? Is there a > reason > >>>> not to include this in the API itself? > >>>> > >>>> -Mark > >>>> > >>>> On Wed, Nov 12, 2014 at 9:50 AM, Joel Koshy <jjkosh...@gmail.com> > >> wrote: > >>>> > >>>>> +1 on the JMX + gradle properties. Is there any (seamless) way of > >>>>> including the exact git hash? That would be extremely useful if users > >>>>> need help debugging and happen to be on an unreleased build (say, off > >>>>> trunk) > >>>>> > >>>>>> On Wed, Nov 12, 2014 at 09:34:35AM -0800, Gwen Shapira wrote: > >>>>>> Actually, Jun suggested exposing this via JMX. > >>>>>> > >>>>>> On Wed, Nov 12, 2014 at 9:31 AM, Gwen Shapira < > >> gshap...@cloudera.com> > >>>>> wrote: > >>>>>> > >>>>>>> Good question. > >>>>>>> > >>>>>>> The server will need to expose this in the protocol, so Kafka > >> clients > >>>>> will > >>>>>>> know what they are talking to. > >>>>>>> > >>>>>>> We may also want to expose this in the producer and consumer, so > >>>> people > >>>>>>> who use Kafka's built-in clients will know which version they > >> have in > >>>>> the > >>>>>>> environment. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Wed, Nov 12, 2014 at 9:09 AM, Mark Roberts <wiz...@gmail.com> > >>>>> wrote: > >>>>>>> > >>>>>>>> Just to be clear: this is going to be exposed via some Api the > >>>> clients > >>>>>>>> can call at startup? > >>>>>>>> > >>>>>>>> > >>>>>>>>> On Nov 12, 2014, at 08:59, Guozhang Wang <wangg...@gmail.com> > >>>>> wrote: > >>>>>>>>> > >>>>>>>>> Sounds great, +1 on this. > >>>>>>>>> > >>>>>>>>>> On Tue, Nov 11, 2014 at 1:36 PM, Gwen Shapira < > >>>>> gshap...@cloudera.com> > >>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> So it looks like we can use Gradle to add properties to > >> manifest > >>>>> file > >>>>>>>> and > >>>>>>>>>> then use getResourceAsStream to read the file and parse it. > >>>>>>>>>> > >>>>>>>>>> The Gradle part would be something like: > >>>>>>>>>> jar.manifest { > >>>>>>>>>> attributes('Implementation-Title': project.name, > >>>>>>>>>> 'Implementation-Version': project.version, > >>>>>>>>>> 'Built-By': System.getProperty('user.name'), > >>>>>>>>>> 'Built-JDK': System.getProperty('java.version'), > >>>>>>>>>> 'Built-Host': getHostname(), > >>>>>>>>>> 'Source-Compatibility': > >> project.sourceCompatibility, > >>>>>>>>>> 'Target-Compatibility': project.targetCompatibility > >>>>>>>>>> ) > >>>>>>>>>> } > >>>>>>>>>> > >>>>>>>>>> The code part would be: > >> > this.getClass().getClassLoader().getResourceAsStream("/META-INF/MANIFEST.MF") > >>>>>>>>>> > >>>>>>>>>> Does that look like the right approach? > >>>>>>>>>> > >>>>>>>>>> Gwen > >>>>>>>>>> > >>>>>>>>>> On Tue, Nov 11, 2014 at 10:43 AM, Bhavesh Mistry < > >>>>>>>>>> mistry.p.bhav...@gmail.com > >>>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> If is maven artifact then you will get following pre-build > >>>>> property > >>>>>>>> file > >>>>>>>>>>> from maven build called pom.properties under > >>>>>>>>>>> /META-INF/maven/groupid/artifactId/pom.properties folder. > >>>>>>>>>>> > >>>>>>>>>>> Here is sample: > >>>>>>>>>>> #Generated by Maven > >>>>>>>>>>> #Mon Oct 10 10:44:31 EDT 2011 > >>>>>>>>>>> version=10.0.1 > >>>>>>>>>>> groupId=com.google.guava > >>>>>>>>>>> artifactId=guava > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> > >>>>>>>>>>> Bhavesh > >>>>>>>>>>> > >>>>>>>>>>> On Tue, Nov 11, 2014 at 10:34 AM, Gwen Shapira < > >>>>> gshap...@cloudera.com > >>>>>>>>> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> In Sqoop we do the following: > >>>>>>>>>>>> > >>>>>>>>>>>> Maven runs a shell script, passing the version as a > >> parameter. > >>>>>>>>>>>> The shell-script generates a small java class, which is then > >>>>> built > >>>>>>>>>> with a > >>>>>>>>>>>> Maven plugin. > >>>>>>>>>>>> Our code references this generated class when we expose > >>>>>>>> "getVersion()". > >>>>>>>>>>>> > >>>>>>>>>>>> Its complex and ugly, so I'm kind of hoping that there's a > >>>>> better way > >>>>>>>>>> to > >>>>>>>>>>> do > >>>>>>>>>>>> it :) > >>>>>>>>>>>> > >>>>>>>>>>>> Gwen > >>>>>>>>>>>> > >>>>>>>>>>>>> On Tue, Nov 11, 2014 at 9:42 AM, Jun Rao <jun...@gmail.com > >>> > >>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Currently, the version number is only stored in our build > >>>> config > >>>>>>>>>> file, > >>>>>>>>>>>>> gradle.properties. Not sure how we can automatically > >> extract > >>>> it > >>>>> and > >>>>>>>>>>>> expose > >>>>>>>>>>>>> it in an mbean. How do other projects do this? > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Jun > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Tue, Nov 11, 2014 at 7:05 AM, Otis Gospodnetic < > >>>>>>>>>>>>> otis.gospodne...@gmail.com> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi Jun, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Sounds good. But is the version number stored anywhere > >> from > >>>>> where > >>>>>>>>>> it > >>>>>>>>>>>>> could > >>>>>>>>>>>>>> be gotten? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>> Otis > >>>>>>>>>>>>>> -- > >>>>>>>>>>>>>> Monitoring * Alerting * Anomaly Detection * Centralized > >> Log > >>>>>>>>>>> Management > >>>>>>>>>>>>>> Solr & Elasticsearch Support * http://sematext.com/ > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Tue, Nov 11, 2014 at 12:45 AM, Jun Rao < > >> jun...@gmail.com> > >>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Otis, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> We don't have an api for that now. We can probably expose > >>>> this > >>>>>>>>>> as a > >>>>>>>>>>>> JMX > >>>>>>>>>>>>>> as > >>>>>>>>>>>>>>> part of kafka-1481. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Jun > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Mon, Nov 10, 2014 at 7:17 PM, Otis Gospodnetic < > >>>>>>>>>>>>>>> otis.gospodne...@gmail.com> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Is there a way to detect which version of Kafka one is > >>>>> running? > >>>>>>>>>>>>>>>> Is there an API for that, or a constant with this > >> value, or > >>>>>>>>>> maybe > >>>>>>>>>>>> an > >>>>>>>>>>>>>>> MBean > >>>>>>>>>>>>>>>> or some other way to get to this info? > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> Otis > >>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>> Monitoring * Alerting * Anomaly Detection * Centralized > >> Log > >>>>>>>>>>>>> Management > >>>>>>>>>>>>>>>> Solr & Elasticsearch Support * http://sematext.com/ > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> -- Guozhang > >> > -- Jeff Holoman Systems Engineer 678-612-9519
