2015-09-26 6:34 GMT+02:00 Ewen Cheslack-Postava <e...@confluent.io>: > The basic functionality is definitely useful here. I'm generally in favor > of exposing some info about broker versions to clients. > > I'd prefer to keep the key/values explicit. Making them extensible > string/string pairs is good for avoiding unnecessary version changes in the > protocol, but I think we should explicitly define the valid key/value > formats in each version of the protocol. New keys can safely be ignored, > but actually specifying the format of the values is important if we ever > need to evolve those formats. >
I agree that the "builtin" key-values populated by the broker itself should be strictly documented, formatted, and strive to be as close to any corresponding configuration properties as possible, both by name and value. > I like some of the examples you've provided for returned key/value pairs > and I think we should provide some of them even when the values should be > obvious from the broker version. > > * broker.version - are we definitely standardizing on this versioning > format? 4 digits, with each level indicating the intuitive levels of > compatibility? Is there any chance we'll have a 0.10.0.0? This might seem > like a trivial consideration, but after fighting versioning in different > packaging systems, I'm a bit more sensitive to the annoying effects that > not considering this carefully can have. We're at a particularly sensitive > point as we hit .9 -> .10 or .9 -> 1.0 (!!!!). > We should specify exactly how to translate the string in to an integer that can be used in numeric comparisons, or better yet, provide broker.version as the human readable representation but also provide a broker.version.int for comparison use? * supported.compression.codecs - nit, but I'd like to figure out a good way > to keep these as close to the actual config name as possible. in this case, > the setting is compression.codec, so just "compression.codecs" seems ideal > to me. > We should probably decide on a format for propagating things that directly map to a (client) configuration property. Prefixed with "config.*"? "config.message.max.bytes"="1000000", or in a separate [key value] list in the response? * rack: I think there's a ton of demand for something like this, but I'd > really like to think through it more before exposing anything. "rack" is > *very* specific to a deployment scenario. I think we're comfortable > adapting the terminology, but the meaning can change drastically, even > under seemingly similar deployments. For example, if you deploy in EC2, you > might create instances in multiple AZs. Within an AZ, you might consider > all the nodes on the same "rack". But there are also placement groups > within each AZ that provide better guarantees on network performance. Are > any nodes in the same AZ considered on the same rack or do you need special > guarantees to be on the same "rack". In general, I don't think there's a > generic "rack" identifier we can expose -- we'll need to do something > specialized depending on different environments. This is a case where > extensibility in the format is probably really useful. > Yeah, I dont think this should be builtin tag at all, but rather be user configurable key-value through server.properties. e.g.: broker.tag.datacenter = amsterdam1 broker.tag.rack = c12r17 broker.tag.aw.zone = ... > * Properties like supported.compression.codecs can presumably be determined > just via the broker version. Is there a good reason for including them? > Perhaps explicit info >> implicit info? > We dont want client implementations to need to track this, it might also be the case that some features are manually disabled on the broker (e.g., disable a broken compression algorithm until next version is released, hi snappy!). > * "All existing clients should be able to handle this gracefully without > any alterations to the code" - which clients does this refer to? For > example, will the Java clients continue to behave the same way they do > today? I'm curious because empty responses seem *very* different from > closing connections. > > I'm assuming here (and I will verify this), but all clients should have some sort of error handling when parsing malformed responses (buffer underflow in this case). > > -Ewen > > On Fri, Sep 25, 2015 at 2:53 PM, Magnus Edenhill <mag...@edenhill.se> > wrote: > > > Good evening, > > > > KIP-35 was created to address current and future broker-client > > compatibility. > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-35+-+Retrieving+protocol+version > > > > Summary: > > * allow clients to retrieve the broker's protocol version > > * make broker handle unknown protocol requests gracefully > > > > Feedback and comments welcome! > > > > Regards, > > Magnus > > > > > > -- > Thanks, > Ewen >
