[jira] [Commented] (KAFKA-2452) enable new consumer in mirror maker

2015-09-30 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939215#comment-14939215 ] Ismael Juma commented on KAFKA-2452: The "not authenticated" warning means that the cl

[jira] [Updated] (KAFKA-2601) ConsoleProducer tool shows stacktrace on invalid command parameters

2015-09-30 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ribeiro updated KAFKA-2601: -- Assignee: (was: Edward Ribeiro) > ConsoleProducer tool shows stacktrace on invalid command p

[jira] [Commented] (KAFKA-2601) ConsoleProducer tool shows stacktrace on invalid command parameters

2015-09-30 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939213#comment-14939213 ] Edward Ribeiro commented on KAFKA-2601: --- Excuse me, Gabriel, I jumped too soon on th

[jira] [Comment Edited] (KAFKA-2601) ConsoleProducer tool shows stacktrace on invalid command parameters

2015-09-30 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939213#comment-14939213 ] Edward Ribeiro edited comment on KAFKA-2601 at 10/1/15 3:17 AM:

[jira] [Assigned] (KAFKA-2601) ConsoleProducer tool shows stacktrace on invalid command parameters

2015-09-30 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ribeiro reassigned KAFKA-2601: - Assignee: Edward Ribeiro > ConsoleProducer tool shows stacktrace on invalid command param

Re: [DISCUSS] KIP-31 - Message format change proposal

2015-09-30 Thread Joel Koshy
The Phase 2 2.* sub-steps don't seem to be right. Can you look over that carefully? Also, "definitive" - you mean "absolute" i.e., not relative offsets right? One more thing that may be worth mentioning is that it is technically possible to canary the new version format on at most one broker (or m

[jira] [Created] (KAFKA-2601) ConsoleProducer tool shows stacktrace on invalid command parameters

2015-09-30 Thread Gabriel Nicolas Avellnaeda (JIRA)
Gabriel Nicolas Avellnaeda created KAFKA-2601: - Summary: ConsoleProducer tool shows stacktrace on invalid command parameters Key: KAFKA-2601 URL: https://issues.apache.org/jira/browse/KAFKA-2601

[GitHub] kafka pull request: Fixed examples on running specific tests.

2015-09-30 Thread GabrielNicolasAvellaneda
GitHub user GabrielNicolasAvellaneda opened a pull request: https://github.com/apache/kafka/pull/268 Fixed examples on running specific tests. Signed-off-by: GabrielNicolasAvellaneda You can merge this pull request into a Git repository by running: $ git pull https://github.co

[jira] [Commented] (KAFKA-2596) Coordinator should return illegal generation for commits from unknown groups with non-negative generation

2015-09-30 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939097#comment-14939097 ] ASF GitHub Bot commented on KAFKA-2596: --- GitHub user hachikuji opened a pull request

[GitHub] kafka pull request: KAFKA-2596: reject commits from unknown groups...

2015-09-30 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/267 KAFKA-2596: reject commits from unknown groups with positive generations You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-

[jira] [Commented] (KAFKA-2452) enable new consumer in mirror maker

2015-09-30 Thread Samuel Zhou (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939075#comment-14939075 ] Samuel Zhou commented on KAFKA-2452: After applying the patch, I was trying to enable

[jira] [Assigned] (KAFKA-2600) Make KStream interfaces compatible with Java 8 java.util.function

2015-09-30 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch reassigned KAFKA-2600: Assignee: Randall Hauch > Make KStream interfaces compatible with Java 8 java.util.function

[jira] [Created] (KAFKA-2600) Make KStream interfaces compatible with Java 8 java.util.function

2015-09-30 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-2600: Summary: Make KStream interfaces compatible with Java 8 java.util.function Key: KAFKA-2600 URL: https://issues.apache.org/jira/browse/KAFKA-2600 Project: Kafka

Re: [DISCUSS] KIP-31 - Message format change proposal

2015-09-30 Thread Magnus Edenhill
The BrokerMetadataRequest API could be used to propagate supported message.formats from broker to client. This is currently discussed in the KIP-35 thread. /Magnus 2015-09-30 1:50 GMT+02:00 Jiangjie Qin : > Hi Joel and other folks. > > I updated the KIP page with the two phase roll out, which

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2015-09-30 Thread Magnus Edenhill
Everyone, thanks for your comments and input this far, here follows an update of the proposal based on the discussions. BrokerMetadataRequest => [NodeId] NodeId => int32 // Request Metadata for these brokers only. // Empty array: retrieve for all brokers.

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2015-09-30 Thread Magnus Edenhill
Very good points, Todd, totally agree. 2015-09-30 1:04 GMT+02:00 Todd Palino : > We should also consider what else should be negotiated between the broker > and the client as this comes together. The version is definitely first, but > there are other things, such as the max message size, that sh

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2015-09-30 Thread Magnus Edenhill
2015-09-29 17:25 GMT+02:00 Grant Henke : > If we create a protocol version negotiation api for clients, can we use it > to replace or improve the ease of upgrades that break inter-broker > messaging? > > Currently upgrades that break the wire protocol take 2 rolling restarts. > The first restart w

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2015-09-30 Thread Magnus Edenhill
2015-09-29 5:26 GMT+02:00 Mayuresh Gharat : > Nice write-up. > > Just had a question, instead of returning an empty response back to the > client, would it be better for the broker to return a response that gives > some more info to the client regarding the min version they need to upgrade > to in

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2015-09-30 Thread Magnus Edenhill
2015-09-29 3:36 GMT+02:00 Jiangjie Qin : > Thanks for the writeup. I also think having a specific protocol for > client-broker version negotiation is better. > > I'm wondering is it better to let the broker to decide the version to use? > It might have some value If brokers have preference for a p

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2015-09-30 Thread Magnus Edenhill
2015-09-28 20:59 GMT+02:00 Jun Rao : > I agree with Ewen that having the keys explicitly specified in the response > is better. > > In addition to the supported protocol version, there are other interesting > metadata at the broker level that could be of interest to things like admin > tools (e.g.

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2015-09-30 Thread Magnus Edenhill
2015-09-28 19:47 GMT+02:00 Jason Gustafson : > Having the version API can make clients more robust, so I'm in favor. One > note on the addition of the "rack" field. Since this is a broker-specific > setting, the client would have to query BrokerMetadata for every new broker > it connects to (unles

Re: [DISCUSS] KIP-35 - Retrieve protocol version

2015-09-30 Thread Magnus Edenhill
2015-09-26 6:34 GMT+02:00 Ewen Cheslack-Postava : > The basic functionality is definitely useful here. I'm generally in favor > of exposing some info about broker versions to clients. > > I'd prefer to keep the key/values explicit. Making them extensible > string/string pairs is good for avoiding

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

2015-09-30 Thread Allen Wang
Can you send me the information on the next KIP hangout? Currently the broker-rack mapping is not cached. In KafkaApis, RackLocator.getRackInfo() is called each time the mapping is needed for auto topic creation. This will ensure latest mapping is used at any time. The ability to get the complete

[jira] [Commented] (KAFKA-2581) Run all existing ducktape tests with SSL-enabled clients and brokers

2015-09-30 Thread Geoff Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938341#comment-14938341 ] Geoff Anderson commented on KAFKA-2581: --- [~rsivaram] Glad to hear you're making good

[jira] [Updated] (KAFKA-2591) Remove Persistent Data before Restoringafter a Fault

2015-09-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2591: - Status: Patch Available (was: In Progress) > Remove Persistent Data before Restoringafter a Fault

[jira] [Work started] (KAFKA-2591) Remove Persistent Data before Restoringafter a Fault

2015-09-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2591 started by Guozhang Wang. > Remove Persistent Data before Restoringafter a Fault >

[jira] [Updated] (KAFKA-2591) Remove Persistent Data before Restoringafter a Fault

2015-09-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2591: - Assignee: Guozhang Wang (was: Yasuhiro Matsuda) > Remove Persistent Data before Restoringafter a

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

2015-09-30 Thread Aditya Auradkar
Perhaps we discuss this during the next KIP hangout? I do see that a pluggable rack locator can be useful but I do see a few concerns: - The RackLocator (as described in the document), implies that it can discover rack information for any node in the cluster. How does it deal with rack location c

[jira] [Updated] (KAFKA-2452) enable new consumer in mirror maker

2015-09-30 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-2452: Status: Patch Available (was: In Progress) > enable new consumer in mirror maker >

[GitHub] kafka pull request: KAFKA-2452: Add new consumer option to mirror ...

2015-09-30 Thread becketqin
GitHub user becketqin opened a pull request: https://github.com/apache/kafka/pull/266 KAFKA-2452: Add new consumer option to mirror maker. You can merge this pull request into a Git repository by running: $ git pull https://github.com/becketqin/kafka KAFKA-2452 Alternatively

[jira] [Commented] (KAFKA-2452) enable new consumer in mirror maker

2015-09-30 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938162#comment-14938162 ] ASF GitHub Bot commented on KAFKA-2452: --- GitHub user becketqin opened a pull request

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

2015-09-30 Thread Allen Wang
I would like to see if we can do both: - Make RackLocator pluggable to facilitate migration with existing broker-rack mapping - Make rack an optional property for broker. If rack is available from broker, treat it as source of truth. For users with existing broker-rack mapping somewhere else, the

[jira] [Commented] (KAFKA-2580) Kafka Broker keeps file handles open for all log files (even if its not written to/read from)

2015-09-30 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937017#comment-14937017 ] Grant Henke commented on KAFKA-2580: A few notes/questions from my initial look at the

[jira] [Commented] (KAFKA-2295) Dynamically loaded classes (encoders, etc.) may not be found by Kafka Producer

2015-09-30 Thread James Netherton (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936958#comment-14936958 ] James Netherton commented on KAFKA-2295: Any news on this? This issue looks criti

[jira] [Commented] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)

2015-09-30 Thread Andrew Olson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936850#comment-14936850 ] Andrew Olson commented on KAFKA-2434: - [~junrao] Yes, this was only intended for parit

[jira] [Assigned] (KAFKA-1843) Metadata fetch/refresh in new producer should handle all node connection states gracefully

2015-09-30 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska reassigned KAFKA-1843: --- Assignee: Eno Thereska > Metadata fetch/refresh in new producer should handle all node connec

[jira] [Commented] (KAFKA-2581) Run all existing ducktape tests with SSL-enabled clients and brokers

2015-09-30 Thread Rajini Sivaram (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936725#comment-14936725 ] Rajini Sivaram commented on KAFKA-2581: --- [~granders] Thank you. 1) I haven't writte

[jira] [Commented] (KAFKA-313) Add JSON/CSV output and looping options to ConsumerGroupCommand

2015-09-30 Thread Dmitry Melanchenko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936518#comment-14936518 ] Dmitry Melanchenko commented on KAFKA-313: -- Hi guys, since this story is incomple