Please let's not add dependencies on more third party protocols/software,
the move away from Zookeeper dependence on the clients is very much welcomed
and it'd be a pity to see a new foreign dependency added for such a trivial
thing
as propagating the version of a broker to its client.
My suggestion is to add a new protocol request which returns:
- broker version
- protocol version
- broker id
A generic future-proof solution would be the use of tags (or named TLVs):
InfoResponse: [InfoTag]
InfoTag:
intX tag ( KAFKA_TAG_BROKER_VERSION, KAFKA_TAG_PROTO_VERSION,
KAFKA_TAG_SSL, ... )
intX type ( KAFKA_TYPE_STR, KAFKA_TYPE_INT32, KAFKA_TYPE_INT64, ...)
intX len
bytes payload
Local site/vendor tags could be defined in the configuration file,
This would also allow clients to enable/disable features based on protocol
version,
e.g., there is no point in trying offsetCommit on a 0.8.0 broker and the
client library
can inform the application about this early, rather than having its
offsetCommit requests
fail by connection teardown (which is not much of an error propagation).
My two cents,
Magnus
2014-11-12 20:11 GMT+01:00 Mark Roberts <wiz...@gmail.com>:
> I haven't worked much with JMX before, but some quick googling (10-20
> minutes) is very inconclusive as to how I would go about getting the server
> version I'm connecting to from a Python client. Can someone please
> reassure me that it's relatively trivial for non Java clients to query JMX
> for the server version of every server in the cluster? Is there a reason
> not to include this in the API itself?
>
> -Mark
>
> On Wed, Nov 12, 2014 at 9:50 AM, Joel Koshy <jjkosh...@gmail.com> wrote:
>
> > +1 on the JMX + gradle properties. Is there any (seamless) way of
> > including the exact git hash? That would be extremely useful if users
> > need help debugging and happen to be on an unreleased build (say, off
> > trunk)
> >
> > On Wed, Nov 12, 2014 at 09:34:35AM -0800, Gwen Shapira wrote:
> > > Actually, Jun suggested exposing this via JMX.
> > >
> > > On Wed, Nov 12, 2014 at 9:31 AM, Gwen Shapira <gshap...@cloudera.com>
> > wrote:
> > >
> > > > Good question.
> > > >
> > > > The server will need to expose this in the protocol, so Kafka clients
> > will
> > > > know what they are talking to.
> > > >
> > > > We may also want to expose this in the producer and consumer, so
> people
> > > > who use Kafka's built-in clients will know which version they have in
> > the
> > > > environment.
> > > >
> > > >
> > > >
> > > > On Wed, Nov 12, 2014 at 9:09 AM, Mark Roberts <wiz...@gmail.com>
> > wrote:
> > > >
> > > >> Just to be clear: this is going to be exposed via some Api the
> clients
> > > >> can call at startup?
> > > >>
> > > >>
> > > >> > On Nov 12, 2014, at 08:59, Guozhang Wang <wangg...@gmail.com>
> > wrote:
> > > >> >
> > > >> > Sounds great, +1 on this.
> > > >> >
> > > >> >> On Tue, Nov 11, 2014 at 1:36 PM, Gwen Shapira <
> > gshap...@cloudera.com>
> > > >> wrote:
> > > >> >>
> > > >> >> So it looks like we can use Gradle to add properties to manifest
> > file
> > > >> and
> > > >> >> then use getResourceAsStream to read the file and parse it.
> > > >> >>
> > > >> >> The Gradle part would be something like:
> > > >> >> jar.manifest {
> > > >> >> attributes('Implementation-Title': project.name,
> > > >> >> 'Implementation-Version': project.version,
> > > >> >> 'Built-By': System.getProperty('user.name'),
> > > >> >> 'Built-JDK': System.getProperty('java.version'),
> > > >> >> 'Built-Host': getHostname(),
> > > >> >> 'Source-Compatibility': project.sourceCompatibility,
> > > >> >> 'Target-Compatibility': project.targetCompatibility
> > > >> >> )
> > > >> >> }
> > > >> >>
> > > >> >> The code part would be:
> > > >> >>
> > > >> >>
> > > >>
> >
> this.getClass().getClassLoader().getResourceAsStream("/META-INF/MANIFEST.MF")
> > > >> >>
> > > >> >> Does that look like the right approach?
> > > >> >>
> > > >> >> Gwen
> > > >> >>
> > > >> >> On Tue, Nov 11, 2014 at 10:43 AM, Bhavesh Mistry <
> > > >> >> mistry.p.bhav...@gmail.com
> > > >> >>> wrote:
> > > >> >>
> > > >> >>> If is maven artifact then you will get following pre-build
> > property
> > > >> file
> > > >> >>> from maven build called pom.properties under
> > > >> >>> /META-INF/maven/groupid/artifactId/pom.properties folder.
> > > >> >>>
> > > >> >>> Here is sample:
> > > >> >>> #Generated by Maven
> > > >> >>> #Mon Oct 10 10:44:31 EDT 2011
> > > >> >>> version=10.0.1
> > > >> >>> groupId=com.google.guava
> > > >> >>> artifactId=guava
> > > >> >>>
> > > >> >>> Thanks,
> > > >> >>>
> > > >> >>> Bhavesh
> > > >> >>>
> > > >> >>> On Tue, Nov 11, 2014 at 10:34 AM, Gwen Shapira <
> > gshap...@cloudera.com
> > > >> >
> > > >> >>> wrote:
> > > >> >>>
> > > >> >>>> In Sqoop we do the following:
> > > >> >>>>
> > > >> >>>> Maven runs a shell script, passing the version as a parameter.
> > > >> >>>> The shell-script generates a small java class, which is then
> > built
> > > >> >> with a
> > > >> >>>> Maven plugin.
> > > >> >>>> Our code references this generated class when we expose
> > > >> "getVersion()".
> > > >> >>>>
> > > >> >>>> Its complex and ugly, so I'm kind of hoping that there's a
> > better way
> > > >> >> to
> > > >> >>> do
> > > >> >>>> it :)
> > > >> >>>>
> > > >> >>>> Gwen
> > > >> >>>>
> > > >> >>>>> On Tue, Nov 11, 2014 at 9:42 AM, Jun Rao <jun...@gmail.com>
> > wrote:
> > > >> >>>>>
> > > >> >>>>> Currently, the version number is only stored in our build
> config
> > > >> >> file,
> > > >> >>>>> gradle.properties. Not sure how we can automatically extract
> it
> > and
> > > >> >>>> expose
> > > >> >>>>> it in an mbean. How do other projects do this?
> > > >> >>>>>
> > > >> >>>>> Thanks,
> > > >> >>>>>
> > > >> >>>>> Jun
> > > >> >>>>>
> > > >> >>>>> On Tue, Nov 11, 2014 at 7:05 AM, Otis Gospodnetic <
> > > >> >>>>> otis.gospodne...@gmail.com> wrote:
> > > >> >>>>>
> > > >> >>>>>> Hi Jun,
> > > >> >>>>>>
> > > >> >>>>>> Sounds good. But is the version number stored anywhere from
> > where
> > > >> >> it
> > > >> >>>>> could
> > > >> >>>>>> be gotten?
> > > >> >>>>>>
> > > >> >>>>>> Thanks,
> > > >> >>>>>> Otis
> > > >> >>>>>> --
> > > >> >>>>>> Monitoring * Alerting * Anomaly Detection * Centralized Log
> > > >> >>> Management
> > > >> >>>>>> Solr & Elasticsearch Support * http://sematext.com/
> > > >> >>>>>>
> > > >> >>>>>>
> > > >> >>>>>> On Tue, Nov 11, 2014 at 12:45 AM, Jun Rao <jun...@gmail.com>
> > > >> >> wrote:
> > > >> >>>>>>
> > > >> >>>>>>> Otis,
> > > >> >>>>>>>
> > > >> >>>>>>> We don't have an api for that now. We can probably expose
> this
> > > >> >> as a
> > > >> >>>> JMX
> > > >> >>>>>> as
> > > >> >>>>>>> part of kafka-1481.
> > > >> >>>>>>>
> > > >> >>>>>>> Thanks,
> > > >> >>>>>>>
> > > >> >>>>>>> Jun
> > > >> >>>>>>>
> > > >> >>>>>>> On Mon, Nov 10, 2014 at 7:17 PM, Otis Gospodnetic <
> > > >> >>>>>>> otis.gospodne...@gmail.com> wrote:
> > > >> >>>>>>>
> > > >> >>>>>>>> Hi,
> > > >> >>>>>>>>
> > > >> >>>>>>>> Is there a way to detect which version of Kafka one is
> > running?
> > > >> >>>>>>>> Is there an API for that, or a constant with this value, or
> > > >> >> maybe
> > > >> >>>> an
> > > >> >>>>>>> MBean
> > > >> >>>>>>>> or some other way to get to this info?
> > > >> >>>>>>>>
> > > >> >>>>>>>> Thanks,
> > > >> >>>>>>>> Otis
> > > >> >>>>>>>> --
> > > >> >>>>>>>> Monitoring * Alerting * Anomaly Detection * Centralized Log
> > > >> >>>>> Management
> > > >> >>>>>>>> Solr & Elasticsearch Support * http://sematext.com/
> > > >> >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > -- Guozhang
> > > >>
> > > >
> > > >
> >
> >
>
