[jira] [Commented] (KAFKA-987) Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit

2013-07-23 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718044#comment-13718044 ] Swapnil Ghike commented on KAFKA-987: - I think that any call to createMessageStreams wi

Re: Does Kafka "message manager" tool (read messages from console, delete messages) exist?

2013-07-23 Thread Vladimir Tretyakov
Hi Jun Rao and thanks for your answer. Re 1: Will try to do it way you have described. As I understood "getOffsetBefore" returns offset for set of compressed messages. In my case it will mean that my "from=timestamp_1" and "to=timestamp_2" will work roughly (I will have to double check timestamp f

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Johan Lundahl (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718029#comment-13718029 ] Johan Lundahl commented on KAFKA-982: - +1 for 298, 301 > Logo for Kafk

[jira] [Commented] (KAFKA-987) Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit

2013-07-23 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13717977#comment-13717977 ] Jun Rao commented on KAFKA-987: --- It seems that we don't need to update the offset map in add

Re: Does Kafka "message manager" tool (read messages from console, delete messages) exist?

2013-07-23 Thread Jun Rao
For 1, you can write a tool that uses our getOffsetBefore api and SimpleConsumer. The offset returned is at the log segment boundary. So it's not going to match the specified time precisely. For 2, you can configure a topic to be rolled by timestamp ("log.roll.hours" and "topic.log.roll.hours"). T

[jira] [Commented] (KAFKA-984) Avoid a full rebalance in cases when a new topic is discovered but container/broker set stay the same

2013-07-23 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13717894#comment-13717894 ] Swapnil Ghike commented on KAFKA-984: - Hey Guozhang, this looks like a reasonable appro

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13717807#comment-13717807 ] Jay Kreps commented on KAFKA-982: - Okay, someone just pointed out that 294 is...uh...kind o

[jira] [Updated] (KAFKA-984) Avoid a full rebalance in cases when a new topic is discovered but container/broker set stay the same

2013-07-23 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-984: Attachment: KAFKA-984.v2.patch Added one more unit test in ZookeeperConsumerConnectorTest for wildca

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Benjamin Black (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13717618#comment-13717618 ] Benjamin Black commented on KAFKA-982: -- None of them make much sense, but 296 is least

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Mahendra M (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716697#comment-13716697 ] Mahendra M commented on KAFKA-982: -- +296, 298 > Logo for Kafka >

[jira] [Commented] (KAFKA-987) Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit

2013-07-23 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716683#comment-13716683 ] Neha Narkhede commented on KAFKA-987: - committed v2 to 0.8 > Avoid che

[jira] [Resolved] (KAFKA-987) Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit

2013-07-23 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-987. - Resolution: Fixed > Avoid checkpointing offsets in Kafka consumer that have not changed since

[jira] [Commented] (KAFKA-987) Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit

2013-07-23 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716680#comment-13716680 ] Neha Narkhede commented on KAFKA-987: - +1 on v2. > Avoid checkpointing

[jira] [Updated] (KAFKA-987) Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit

2013-07-23 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-987: Attachment: kafka-987-v2.patch Yes, thank you. Attached v2. > Avoid checkpointing o

[jira] [Updated] (KAFKA-984) Avoid a full rebalance in cases when a new topic is discovered but container/broker set stay the same

2013-07-23 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-984: Attachment: KAFKA-984.v1.patch > Avoid a full rebalance in cases when a new topic is discovered

[jira] [Updated] (KAFKA-984) Avoid a full rebalance in cases when a new topic is discovered but container/broker set stay the same

2013-07-23 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-984: Status: Patch Available (was: Open) > Avoid a full rebalance in cases when a new topic is disco

[jira] [Updated] (KAFKA-883) System Test - update migration tool testsuite after 0.7 ProducerPerformance sends seq MessageID

2013-07-23 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-883: -- Resolution: Fixed Fix Version/s: 0.8 Status: Resolved (was: Patch Available) Thanks for the p

[jira] [Updated] (KAFKA-882) Enhance 0.7 ProducerPerformance to send sequential MessageID as in 0.8

2013-07-23 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-882: -- Resolution: Fixed Fix Version/s: 0.7 Status: Resolved (was: Patch Available) Thanks for the p

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread David Arthur (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716607#comment-13716607 ] David Arthur commented on KAFKA-982: [~jkreps], to me they look like messages in a log

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716601#comment-13716601 ] Jay Kreps commented on KAFKA-982: - Not sure that I get 301, what are the pictures? They kin

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716594#comment-13716594 ] Swapnil Ghike commented on KAFKA-982: - Now that we have 301 back in the race, I change

[jira] [Commented] (KAFKA-925) Add optional partition key override in producer

2013-07-23 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716593#comment-13716593 ] Joel Koshy commented on KAFKA-925: -- +1 , looks good to me. DefaultPartitioner: Do we need

[jira] [Commented] (KAFKA-987) Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit

2013-07-23 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716583#comment-13716583 ] Neha Narkhede commented on KAFKA-987: - Thanks for the patch, Swapnil. Good thinking abo

[jira] [Updated] (KAFKA-932) System Test - set retry.backoff.ms=300 to all test cases

2013-07-23 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-932: -- Resolution: Fixed Fix Version/s: 0.8 Assignee: John Fung Status: Resolved (was: Patch

[jira] [Updated] (KAFKA-922) System Test - set retry.backoff.ms=300 to testcase_0119

2013-07-23 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-922: -- Resolution: Fixed Fix Version/s: 0.8 Assignee: John Fung Status: Resolved (was: Patch

[jira] [Closed] (KAFKA-922) System Test - set retry.backoff.ms=300 to testcase_0119

2013-07-23 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao closed KAFKA-922. - > System Test - set retry.backoff.ms=300 to testcase_0119 > --

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Chris Riccomini (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716567#comment-13716567 ] Chris Riccomini commented on KAFKA-982: --- +1 301 > Logo for Kafka > -

[jira] [Updated] (KAFKA-879) In system test, read the new leader from zookeeper instead of broker log on completion of become-leader state transition

2013-07-23 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-879: -- Resolution: Fixed Fix Version/s: 0.8 Status: Resolved (was: Patch Available) Thanks for the p

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716562#comment-13716562 ] Neha Narkhede commented on KAFKA-982: - Oops, my bad. I meant 294 as my 2nd choice.

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716553#comment-13716553 ] Jay Kreps commented on KAFKA-982: - Wait, Neha, do you mean 294? I don't see a pen in 296...

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716544#comment-13716544 ] Neha Narkhede commented on KAFKA-982: - +1 for 298, it's clean and simple. Second choice

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread David Arthur (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716528#comment-13716528 ] David Arthur commented on KAFKA-982: +1 for 301 as well. Looks like a bit like a log fi

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716519#comment-13716519 ] Guozhang Wang commented on KAFKA-982: - +1 for 289. But it seems not among the favorite

[jira] [Updated] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-982: Attachment: 301.png > Logo for Kafka > -- > > Key: KAFKA-982 >

Does Kafka "message manager" tool (read messages from console, delete messages) exist?

2013-07-23 Thread Vladimir Tretyakov
Hi, we use Kafka (0.7.2 version) in our product as message delivery service and message storage (our retention time 48h) system. Thanks a lot for perfect tool/lib. Now we really need 2 additional features: 1. Sometimes we need look at our messages we have in Kafka from command line, for example: ~

Does Kafka "message manager" tool (read messages from console, delete messages) exist?

2013-07-23 Thread Vladimir Tretyakov
Hi, we use Kafka (0.7.2 version) in our product as message delivery service and message storage (our retention time 48h) system. Thanks a lot for perfect tool/lib. Now we really need 2 additional features: 1. Sometimes we need look at our messages we have in Kafka from command line, for example: ~

[jira] [Commented] (KAFKA-982) Logo for Kafka

2013-07-23 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716285#comment-13716285 ] Joel Koshy commented on KAFKA-982: -- +1 for 298, and I like Jakob's recursive suggestion as