[jira] [Commented] (KAFKA-734) Migration tool needs a revamp, it was poorly written and has many performance bugs
[ https://issues.apache.org/jira/browse/KAFKA-734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585201#comment-13585201 ] Jun Rao commented on KAFKA-734: --- Thanks for patch v5. Just some minor comments. Once they are addressed, the patch can be checked in. 50. KafkaMigrationTool: 50.1 Should client.id for producer be set to the client.id in the producer property + the producer thread id? 50.2 In the shutdown hook, we need to call consumerConnector_07.shutdown first. 50.3 In MigrationThread, we don't need isRunning since it's never used. 50.4 In the catch clause of main(), it's probably better to print out the error and stacktrace, in addition to the logging in log4j. This way, if people forget to set log4j properly, they can still see why the tool failed. 50.5 At the end of shutdown hook, it's probably useful to print out sth like "migration tool shuts down successfully". > Migration tool needs a revamp, it was poorly written and has many performance > bugs > -- > > Key: KAFKA-734 > URL: https://issues.apache.org/jira/browse/KAFKA-734 > Project: Kafka > Issue Type: Bug > Components: tools >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: p1 > Attachments: kafka-734-v1.patch, kafka-734-v2.patch, > kafka-734-v3.patch, kafka-734-v4.patch, kafka-734-v5.patch > > > Migration tool has a number of problems ranging from poor logging to poor > design. This needs to be thought through again -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-671) DelayedProduce requests should not hold full producer request data
[ https://issues.apache.org/jira/browse/KAFKA-671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585203#comment-13585203 ] Jun Rao commented on KAFKA-671: --- Patch v4 doesn't apply. 0.8 has moved since the patch is uploaded. Could you rebase again? > DelayedProduce requests should not hold full producer request data > -- > > Key: KAFKA-671 > URL: https://issues.apache.org/jira/browse/KAFKA-671 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Joel Koshy >Assignee: Sriram Subramanian >Priority: Blocker > Labels: bugs, p1 > Fix For: 0.8.1 > > Attachments: outOfMemFix-v1.patch, outOfMemFix-v2.patch, > outOfMemFix-v2-rebase.patch, outOfMemFix-v3.patch, outOfMemFix-v4.patch > > > Per summary, this leads to unnecessary memory usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-733) Fat jar option for build, or override for ivy cache location
[ https://issues.apache.org/jira/browse/KAFKA-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585214#comment-13585214 ] Joe Stein commented on KAFKA-733: - I like the idea of assembly but not everyone is going to want/like another dependency so we adding a dependency for running the server without providing another option. In either case something has to get done here KAFKA-760 for how to star the server for different build options this would be another option > Fat jar option for build, or override for ivy cache location > - > > Key: KAFKA-733 > URL: https://issues.apache.org/jira/browse/KAFKA-733 > Project: Kafka > Issue Type: Improvement > Components: packaging >Affects Versions: 0.8 >Reporter: Dave DeMaagd >Assignee: Dave DeMaagd > Labels: bugs > Attachments: KAFKA-733.patch > > > Need some kind of self-contained mechanism for running kafka to get around > the following: > 1) The location of the source checkout/build is not necessarily the same > place where it will be running (the build location and that user's ivy cache > dir) potentially leading to sync problems (forgetting the ivy dir) or just > adding overhead to the deployment process (additional steps to remember > introduces more chances for mistakes) > 2) The user running the kafka service in a production setting may not even > have a real home directory > Think something like a 'fat jar' packaging (something that contains all > necessary jar versions in one convenient place) would simplify deployment and > reduce the chance for error (only one lib package to worry about, and it > contains everything needed) and would be a little more production friendly -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (KAFKA-733) Fat jar option for build, or override for ivy cache location
[ https://issues.apache.org/jira/browse/KAFKA-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585214#comment-13585214 ] Joe Stein edited comment on KAFKA-733 at 2/23/13 9:14 PM: -- I like the idea of assembly but not everyone is going to want/like another dependency so we adding a dependency for running the server without providing another option. In either case something has to get done here KAFKA-760 for how to start the server for different build options this would be another option was (Author: charmalloc): I like the idea of assembly but not everyone is going to want/like another dependency so we adding a dependency for running the server without providing another option. In either case something has to get done here KAFKA-760 for how to star the server for different build options this would be another option > Fat jar option for build, or override for ivy cache location > - > > Key: KAFKA-733 > URL: https://issues.apache.org/jira/browse/KAFKA-733 > Project: Kafka > Issue Type: Improvement > Components: packaging >Affects Versions: 0.8 >Reporter: Dave DeMaagd >Assignee: Dave DeMaagd > Labels: bugs > Attachments: KAFKA-733.patch > > > Need some kind of self-contained mechanism for running kafka to get around > the following: > 1) The location of the source checkout/build is not necessarily the same > place where it will be running (the build location and that user's ivy cache > dir) potentially leading to sync problems (forgetting the ivy dir) or just > adding overhead to the deployment process (additional steps to remember > introduces more chances for mistakes) > 2) The user running the kafka service in a production setting may not even > have a real home directory > Think something like a 'fat jar' packaging (something that contains all > necessary jar versions in one convenient place) would simplify deployment and > reduce the chance for error (only one lib package to worry about, and it > contains everything needed) and would be a little more production friendly -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
