Re: having problem with 0.8 gzip compression

2013-07-08 Thread Scott Wang
No, I did not start the consumer before the producer. I actually started the producer first and nothing showed up in the consumer unless I commented out this line -- props.put("compression.codec", "gzip").If I commented out the compression codec, everything just works. On Mon, Jul 8, 2013 at

Re: having problem with 0.8 gzip compression

2013-07-08 Thread Jun Rao
Did you start the consumer before the producer? Be default, the consumer gets only the new data? Thanks, Jun On Mon, Jul 8, 2013 at 2:53 PM, Scott Wang < scott.w...@rumbleentertainment.com> wrote: > I am testing with Kafka 0.8 beta and having problem of receiving message in > consumer. There

[jira] [Commented] (KAFKA-959) DefaultEventHandler can send more produce requests than necesary

2013-07-08 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702718#comment-13702718 ] Guozhang Wang commented on KAFKA-959: - Proposed code change: 1. Add a Map to DefaultEv

[jira] [Comment Edited] (KAFKA-718) kafka-run-class.sh should use reasonable gc settings

2013-07-08 Thread Ashwanth Fernando (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702677#comment-13702677 ] Ashwanth Fernando edited comment on KAFKA-718 at 7/9/13 12:24 AM: ---

[jira] [Commented] (KAFKA-718) kafka-run-class.sh should use reasonable gc settings

2013-07-08 Thread Ashwanth Fernando (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702677#comment-13702677 ] Ashwanth Fernando commented on KAFKA-718: - Eitherways, I have a patch that cleanly

[jira] [Updated] (KAFKA-718) kafka-run-class.sh should use reasonable gc settings

2013-07-08 Thread Ashwanth Fernando (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwanth Fernando updated KAFKA-718: Attachment: 718-v1.patch > kafka-run-class.sh should use reasonable gc settings > --

[jira] [Updated] (KAFKA-718) kafka-run-class.sh should use reasonable gc settings

2013-07-08 Thread Ashwanth Fernando (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwanth Fernando updated KAFKA-718: Affects Version/s: (was: 0.8) Status: Patch Available (was: Open) >

[jira] [Commented] (KAFKA-718) kafka-run-class.sh should use reasonable gc settings

2013-07-08 Thread Ashwanth Fernando (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702671#comment-13702671 ] Ashwanth Fernando commented on KAFKA-718: - The change is trivial. I guess where I a

[jira] [Commented] (KAFKA-957) MirrorMaker needs to preserve the key in the source cluster

2013-07-08 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702667#comment-13702667 ] Guozhang Wang commented on KAFKA-957: - The proposed design is the following: 1. Keep t

[jira] [Updated] (KAFKA-896) merge 0.8 (988d4d8e65a14390abd748318a64e281e4a37c19) to trunk

2013-07-08 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-896: -- Resolution: Fixed Fix Version/s: 0.8.1 Status: Resolved (was: Patch Available) Thanks for the

having problem with 0.8 gzip compression

2013-07-08 Thread Scott Wang
I am testing with Kafka 0.8 beta and having problem of receiving message in consumer. There is no error so does anyone have any insights. When I commented out the "compression.code" everything works fine. My producer: public class TestKafka08Prod { public static void main(String [] args) {

[jira] [Updated] (KAFKA-964) Default hadoop-producer configs to request.required.acks = 1

2013-07-08 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-964: -- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patch. Committed to 0.8.

[jira] [Updated] (KAFKA-964) Default hadoop-producer configs to request.required.acks = 1

2013-07-08 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-964: Attachment: kafka-964-v1.patch Set request.required.acks = 1 in hadoop-producer. Maintained the ove

[jira] [Updated] (KAFKA-964) Default hadoop-producer configs to request.required.acks = 1

2013-07-08 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-964: Attachment: (was: kafka-964.patch) > Default hadoop-producer configs to request.required.ack

[jira] [Updated] (KAFKA-964) Default hadoop-producer configs to request.required.acks = 1

2013-07-08 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-964: Summary: Default hadoop-producer configs to request.required.acks = 1 (was: Default hadoop-producer

[jira] [Updated] (KAFKA-559) Garbage collect old consumer metadata entries

2013-07-08 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated KAFKA-559: -- Attachment: KAFKA-559.v2.patch _1. Passing a groupId for cleanup will make the cleanup job tedious since

[jira] Subscription: outstanding kafka patches

2013-07-08 Thread jira
Issue Subscription Filter: outstanding kafka patches (75 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-964 Default hadoop-producer configs to ProducerConfig defaults https://issues.apache.org/jira/browse/KAFKA-964 KAFKA

[jira] [Commented] (KAFKA-943) Move all configuration key string to constants

2013-07-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702279#comment-13702279 ] Jay Kreps commented on KAFKA-943: - That would work for me. > Move all conf

[jira] [Comment Edited] (KAFKA-615) Avoid fsync on log segment roll

2013-07-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702275#comment-13702275 ] Jay Kreps edited comment on KAFKA-615 at 7/8/13 6:51 PM: - New patch

[jira] [Updated] (KAFKA-615) Avoid fsync on log segment roll

2013-07-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-615: Attachment: KAFKA-615-v2.patch New patch with a couple of improvements: 1. Found and fixed a bug in recovery

[jira] [Updated] (KAFKA-964) Default hadoop-producer configs to ProducerConfig defaults

2013-07-08 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-964: Labels: bugs (was: ) Status: Patch Available (was: Open) > Default hadoop-producer configs

[jira] [Updated] (KAFKA-964) Default hadoop-producer configs to ProducerConfig defaults

2013-07-08 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-964: Attachment: kafka-964.patch > Default hadoop-producer configs to ProducerConfig defaults > -

[jira] [Created] (KAFKA-964) Default hadoop-producer configs to ProducerConfig defaults

2013-07-08 Thread Swapnil Ghike (JIRA)
Swapnil Ghike created KAFKA-964: --- Summary: Default hadoop-producer configs to ProducerConfig defaults Key: KAFKA-964 URL: https://issues.apache.org/jira/browse/KAFKA-964 Project: Kafka Issue Ty

[jira] [Commented] (KAFKA-718) kafka-run-class.sh should use reasonable gc settings

2013-07-08 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702201#comment-13702201 ] Jay Kreps commented on KAFKA-718: - Hey Jun, this should be a trivial change and will likely

[jira] [Updated] (KAFKA-915) System Test - Mirror Maker testcase_5001 failed

2013-07-08 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-915: Attachment: kafka-915-v1.patch > System Test - Mirror Maker testcase_5001 failed > -

[jira] [Commented] (KAFKA-915) System Test - Mirror Maker testcase_5001 failed

2013-07-08 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702134#comment-13702134 ] John Fung commented on KAFKA-915: - Thanks Joel for the suggestion. It's working by calling

[jira] [Updated] (KAFKA-915) System Test - Mirror Maker testcase_5001 failed

2013-07-08 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-915: Status: Patch Available (was: Open) > System Test - Mirror Maker testcase_5001 failed > ---

[jira] [Commented] (KAFKA-961) state.change.logger: Error on broker 1 while processing LeaderAndIsr request correlationId 6 received from controller 1 epoch 1 for partition (page_visits,0)

2013-07-08 Thread Garrett Barton (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702129#comment-13702129 ] Garrett Barton commented on KAFKA-961: -- That worked like a charm! Thank you Jun.

[jira] [Resolved] (KAFKA-961) state.change.logger: Error on broker 1 while processing LeaderAndIsr request correlationId 6 received from controller 1 epoch 1 for partition (page_visits,0)

2013-07-08 Thread Garrett Barton (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garrett Barton resolved KAFKA-961. -- Resolution: Not A Problem Fix Version/s: 0.8 Use KafkaServerStartable when embedding a Ka