[jira] [Commented] (KAFKA-2510) Prevent broker from re-replicating / losing data due to disk misconfiguration

2015-09-06 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733244#comment-14733244 ] Gwen Shapira commented on KAFKA-2510: - No, the issue I'm trying to prevent is definite

Re: [DISCUSS] KIP-31 - Message format change proposal

2015-09-06 Thread Todd Palino
So, with regards to why you want to search by timestamp, the biggest problem I've seen is with consumers who want to reset their timestamps to a specific point, whether it is to replay a certain amount of messages, or to rewind to before some problem state existed. This happens more often than anyo

Re: [DISCUSS] KIP-31 - Message format change proposal

2015-09-06 Thread Ewen Cheslack-Postava
On Sun, Sep 6, 2015 at 4:57 PM, Jay Kreps wrote: > > 2. Nobody cares what time it is on the server. > This is a good way of summarizing the issue I was trying to get at, from an app's perspective. Of the 3 stated goals of the KIP, #2 (lot retention) is reasonably handled by a server-side timesta

Re: [DISCUSS] KIP-31 - Message format change proposal

2015-09-06 Thread Jay Kreps
One more comment, I think there are really three proposals here: 1. Get a mechanism and policy in place for record format upgrade (we haven't done this so we don't really have the infra). This is kind of implicit. I suspect we'll need to do this multiple times in the future so we should make it eas

Re: [DISCUSS] KIP-31 - Message format change proposal

2015-09-06 Thread Jay Kreps
The magic byte is used to version message format so we'll need to make sure that check is in place--I actually don't see it in the current consumer code which I think is a bug we should fix for the next release (filed KAFKA-2523). The purpose of that field is so there is a clear check on the format

[jira] [Created] (KAFKA-2523) New consumer doesn't check magic version

2015-09-06 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-2523: Summary: New consumer doesn't check magic version Key: KAFKA-2523 URL: https://issues.apache.org/jira/browse/KAFKA-2523 Project: Kafka Issue Type: Bug Re

[GitHub] kafka pull request: 0.8.2

2015-09-06 Thread leoricxu
GitHub user leoricxu opened a pull request: https://github.com/apache/kafka/pull/198 0.8.2 You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/kafka 0.8.2 Alternatively you can review and apply these changes as the patch at:

[jira] [Updated] (KAFKA-2522) ConsumerGroupCommand sends all output to STDOUT

2015-09-06 Thread Dmitry Melanchenko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Melanchenko updated KAFKA-2522: -- Attachment: kafka_2522_print_err_to_stderr.patch Patch for this problem > ConsumerGroup

[GitHub] kafka pull request: KAFKA-2522: ConsumerGroupCommand writes error ...

2015-09-06 Thread melan
GitHub user melan opened a pull request: https://github.com/apache/kafka/pull/197 KAFKA-2522: ConsumerGroupCommand writes error messages to STDERR ConsumerGroupCommand sends errors and valuable output to different streams. It simplifies results parsing. @joestein @johann838

[jira] [Commented] (KAFKA-2522) ConsumerGroupCommand sends all output to STDOUT

2015-09-06 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14732282#comment-14732282 ] ASF GitHub Bot commented on KAFKA-2522: --- GitHub user melan opened a pull request: