TCP Flow Control strategy VS Push/Pull

2013-03-14 Thread 王仁侠
Hi, I am a beginner of Kafka and I have question about the design of Push/Pull model. I know the Push/Pull framework can avoid the blocking between the Broker and the Consumer. However, as I know, the TCP flow control strategy can also solve this problem, am I right? So why do not directly apply

Re: TCP Flow Control strategy VS Push/Pull

2013-03-14 Thread Jun Rao
It seems that your main question is why Kafka lets consumer pull, instead of broker push? The main benefit that is that the consumer naturally gets data as fast as it can process. If the broker pushes, it has to keep estimating the appropriate rate that it can push to the consumer. Thanks, Jun O

[jira] [Commented] (KAFKA-804) Incorrect index in the log of a follower

2013-03-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602414#comment-13602414 ] Jay Kreps commented on KAFKA-804: - I am not sure about this change. There are two choices t

[jira] [Assigned] (KAFKA-805) log.append() may fail with a compressed messageset containing no uncompressed messages

2013-03-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-805: --- Assignee: Jay Kreps > log.append() may fail with a compressed messageset containing no uncompresse

[jira] [Created] (KAFKA-807) LineMessageReader doesn't correctly parse the key separator

2013-03-14 Thread Dragos Manolescu (JIRA)
Dragos Manolescu created KAFKA-807: -- Summary: LineMessageReader doesn't correctly parse the key separator Key: KAFKA-807 URL: https://issues.apache.org/jira/browse/KAFKA-807 Project: Kafka

[jira] [Commented] (KAFKA-804) Incorrect index in the log of a follower

2013-03-14 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602457#comment-13602457 ] Jun Rao commented on KAFKA-804: --- Yes, the log code is a bit more complicated now to address a

[jira] [Created] (KAFKA-808) Migration tool internal queue between consumer and producer threads should be configurable

2013-03-14 Thread Neha Narkhede (JIRA)
Neha Narkhede created KAFKA-808: --- Summary: Migration tool internal queue between consumer and producer threads should be configurable Key: KAFKA-808 URL: https://issues.apache.org/jira/browse/KAFKA-808

[jira] [Work started] (KAFKA-808) Migration tool internal queue between consumer and producer threads should be configurable

2013-03-14 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-808 started by Neha Narkhede. > Migration tool internal queue between consumer and producer threads should be > configurable > -

[jira] [Updated] (KAFKA-808) Migration tool internal queue between consumer and producer threads should be configurable

2013-03-14 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-808: Attachment: kafka-808.patch Added an optional queue.size config to the migration tool

[jira] [Updated] (KAFKA-808) Migration tool internal queue between consumer and producer threads should be configurable

2013-03-14 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-808: Status: Patch Available (was: In Progress) > Migration tool internal queue between consumer and

[jira] [Commented] (KAFKA-804) Incorrect index in the log of a follower

2013-03-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602569#comment-13602569 ] Jay Kreps commented on KAFKA-804: - So does this patch have the fix for all those?

[jira] Subscription: outstanding kafka patches

2013-03-14 Thread jira
Issue Subscription Filter: outstanding kafka patches (61 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-808 Migration tool internal queue between consumer and producer threads should be configurable https://issues.apach

[jira] [Commented] (KAFKA-808) Migration tool internal queue between consumer and producer threads should be configurable

2013-03-14 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603006#comment-13603006 ] Jun Rao commented on KAFKA-808: --- Thanks for the patch. +1. > Migration tool