[jira] [Created] (KAFKA-1877) Expose version via JMX for 'new' producer

2015-01-19 Thread Vladimir Tretyakov (JIRA)
Vladimir Tretyakov created KAFKA-1877: - Summary: Expose version via JMX for 'new' producer Key: KAFKA-1877 URL: https://issues.apache.org/jira/browse/KAFKA-1877 Project: Kafka Issue Type

[jira] [Created] (KAFKA-1878) ProducerFailureHandlingTest.testCannotSendToInternalTopic fails with TimeoutException while trying to fetch metadata for topic

2015-01-19 Thread jaikiran pai (JIRA)
jaikiran pai created KAFKA-1878: --- Summary: ProducerFailureHandlingTest.testCannotSendToInternalTopic fails with TimeoutException while trying to fetch metadata for topic Key: KAFKA-1878 URL: https://issues.apache.or

[jira] [Updated] (KAFKA-1878) ProducerFailureHandlingTest.testCannotSendToInternalTopic fails with TimeoutException while trying to fetch metadata for topic

2015-01-19 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaikiran pai updated KAFKA-1878: Status: Patch Available (was: Open) > ProducerFailureHandlingTest.testCannotSendToInternalTopic fai

Review Request 30026: Patch for KAFKA-1878

2015-01-19 Thread Jaikiran Pai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30026/ --- Review request for kafka. Bugs: KAFKA-1878 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-1878) ProducerFailureHandlingTest.testCannotSendToInternalTopic fails with TimeoutException while trying to fetch metadata for topic

2015-01-19 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaikiran pai updated KAFKA-1878: Attachment: KAFKA-1878.patch > ProducerFailureHandlingTest.testCannotSendToInternalTopic fails with

[jira] [Commented] (KAFKA-1878) ProducerFailureHandlingTest.testCannotSendToInternalTopic fails with TimeoutException while trying to fetch metadata for topic

2015-01-19 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282338#comment-14282338 ] jaikiran pai commented on KAFKA-1878: - Created reviewboard https://reviews.apache.org/

Re: ProducerFailureHandlingTest.testCannotSendToInternalTopic is failing

2015-01-19 Thread Jaikiran Pai
Hi Harsha, I've now created a new JIRA https://issues.apache.org/jira/browse/KAFKA-1878 which explains the issue in a bit more detail (sorry, I initially thought this change could go linked with that existing JIRA, but you are right it needed a different one). The JIRA includes the exception

[jira] [Commented] (KAFKA-1869) Openning some random ports while running kafka service

2015-01-19 Thread QianHu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282350#comment-14282350 ] QianHu commented on KAFKA-1869: --- As you said, one random port is for the JMX RMI registry of

[jira] [Commented] (KAFKA-1869) Openning some random ports while running kafka service

2015-01-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282386#comment-14282386 ] Manikumar Reddy commented on KAFKA-1869: 1. -Dcom.sun.management.jmxremote.port=8

Detecting lost connection in high level consumer

2015-01-19 Thread harikiran
Hi I am using the 0811 Kafka High level consumer and I have configured " consumer.timeout.ms" to a value that is not -1, say 5000ms. I create the consumer iterator and invoke hasNext() method on it. Irrespective of whether kafka broker was shutdown or there was no message written to kafka, I see

Re: Review Request 30026: Patch for KAFKA-1878

2015-01-19 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30026/#review68620 --- core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTe

Re: Review Request 30026: Patch for KAFKA-1878

2015-01-19 Thread Jaikiran Pai
> On Jan. 19, 2015, 4:14 p.m., Jun Rao wrote: > > core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala, > > lines 310-320 > > > > > > It seems it's simpler to just change the defaultOffsetPartit

Re: Review Request 30026: Patch for KAFKA-1878

2015-01-19 Thread Jaikiran Pai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30026/ --- (Updated Jan. 19, 2015, 4:33 p.m.) Review request for kafka. Bugs: KAFKA-1878

[jira] [Updated] (KAFKA-1878) ProducerFailureHandlingTest.testCannotSendToInternalTopic fails with TimeoutException while trying to fetch metadata for topic

2015-01-19 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaikiran pai updated KAFKA-1878: Attachment: KAFKA-1878_2015-01-19_22:02:54.patch > ProducerFailureHandlingTest.testCannotSendToInter

[jira] [Commented] (KAFKA-1878) ProducerFailureHandlingTest.testCannotSendToInternalTopic fails with TimeoutException while trying to fetch metadata for topic

2015-01-19 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282672#comment-14282672 ] jaikiran pai commented on KAFKA-1878: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 30026: Patch for KAFKA-1878

2015-01-19 Thread Jaikiran Pai
> On Jan. 19, 2015, 4:14 p.m., Jun Rao wrote: > > core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala, > > lines 310-320 > > > > > > It seems it's simpler to just change the defaultOffsetPartit

[jira] [Updated] (KAFKA-1761) num.partitions documented default is 1 while actual default is 2

2015-01-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1761: --- Resolution: Fixed Fix Version/s: 0.8.2 Status: Resolved (was: Patch Available) Thanks for t

[jira] [Commented] (KAFKA-1728) update 082 docs

2015-01-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282685#comment-14282685 ] Jun Rao commented on KAFKA-1728: Thanks for the update. Could you attach a patch against

Re: Review Request 30019: Patch for kafka-1876

2015-01-19 Thread Sriharsha Chintalapani
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30019/#review68631 --- Ship it! Ship It! - Sriharsha Chintalapani On Jan. 19, 2015, 1:5

[jira] [Updated] (KAFKA-1878) ProducerFailureHandlingTest.testCannotSendToInternalTopic fails with TimeoutException while trying to fetch metadata for topic

2015-01-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1878: --- Resolution: Fixed Fix Version/s: 0.8.3 Status: Resolved (was: Patch Available) Thanks for t

Build failed in Jenkins: Kafka-trunk #373

2015-01-19 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA-1723; num.partitions documented default is 1 while actual default is 2; patched by Manikumar Reddy; reviewed by Jun Rao [junrao] KAFKA-1878; ProducerFailureHandlingTest.testCannotSendToInternalTopic fails with

NullPointerException in RequestSendThread

2015-01-19 Thread Jaikiran Pai
I often see the following exception while running some tests (ProducerFailureHandlingTest.testNoResponse is one such instance): [2015-01-19 22:30:24,257] ERROR [Controller-0-to-broker-1-send-thread], Controller 0 fails to send a request to broker id:1,host:localhost,port:56729 (kafka.controll

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-19 Thread Jun Rao
For 2, how about we make a change to log a warning for ack > 1 in 0.8.2 and then drop the ack > 1 support in trunk (w/o bumping up the protocol version)? Thanks, Jun On Sun, Jan 18, 2015 at 8:24 PM, Gwen Shapira wrote: > Overall, agree on point #1, less sure on point #2. > > 1. Some protocols n

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-19 Thread Joe Stein
<< For 2, how about we make a change to log a warning for ack > 1 in 0.8.2 and then drop the ack > 1 support in trunk (w/o bumping up the protocol version)? +1 On Mon, Jan 19, 2015 at 12:35 PM, Jun Rao wrote: > For 2, how about we make a change to log a warning for ack > 1 in 0.8.2 > and then d

Re: [kafka-clients] Re: Heads up: KAFKA-1697 - remove code related to ack>1 on the broker

2015-01-19 Thread Gwen Shapira
Sounds good to me. I'll open a new JIRA for 0.8.2 with just an extra log warning, to avoid making KAFKA-1697 any more confusing. On Mon, Jan 19, 2015 at 9:46 AM, Joe Stein wrote: > << For 2, how about we make a change to log a warning for ack > 1 in 0.8.2 > and then drop the ack > 1 support in tr

[jira] [Created] (KAFKA-1879) Log warning when receiving produce requests with acks > 1

2015-01-19 Thread Gwen Shapira (JIRA)
Gwen Shapira created KAFKA-1879: --- Summary: Log warning when receiving produce requests with acks > 1 Key: KAFKA-1879 URL: https://issues.apache.org/jira/browse/KAFKA-1879 Project: Kafka Issue T

[jira] [Commented] (KAFKA-1876) pom file for scala 2.11 should reference a specific version

2015-01-19 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282784#comment-14282784 ] Ewen Cheslack-Postava commented on KAFKA-1876: -- Based on the output of {{./gr

[jira] [Updated] (KAFKA-1876) pom file for scala 2.11 should reference a specific version

2015-01-19 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1876: - Attachment: twoeleven.tgz gradle works yes however maven fails since it doesn't have unicorns and pixie d

[jira] [Updated] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-01-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-1729: --- Fix Version/s: 0.8.2 > add doc for Kafka-based offset management in 0.8.2 > --

[jira] [Updated] (KAFKA-1728) update 082 docs

2015-01-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-1728: --- Attachment: default-config-value-0.8.2.patch Uploaded patch to correct default config values i

[jira] [Updated] (KAFKA-1852) OffsetCommitRequest can commit offset on unknown topic

2015-01-19 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1852: -- Attachment: KAFKA-1852_2015-01-19_10:44:01.patch > OffsetCommitRequest can commi

Re: Review Request 29912: Patch for KAFKA-1852

2015-01-19 Thread Sriharsha Chintalapani
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29912/ --- (Updated Jan. 19, 2015, 6:44 p.m.) Review request for kafka. Bugs: KAFKA-1852

[jira] [Commented] (KAFKA-1852) OffsetCommitRequest can commit offset on unknown topic

2015-01-19 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282844#comment-14282844 ] Sriharsha Chintalapani commented on KAFKA-1852: --- Updated reviewboard https:/

[jira] [Updated] (KAFKA-1879) Log warning when receiving produce requests with acks > 1

2015-01-19 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1879: Status: Patch Available (was: Open) > Log warning when receiving produce requests with acks > 1 > -

[jira] [Commented] (KAFKA-1879) Log warning when receiving produce requests with acks > 1

2015-01-19 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282866#comment-14282866 ] Gwen Shapira commented on KAFKA-1879: - Created reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1879) Log warning when receiving produce requests with acks > 1

2015-01-19 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1879: Attachment: KAFKA-1879.patch > Log warning when receiving produce requests with acks > 1 > -

Review Request 30046: Patch for KAFKA-1879

2015-01-19 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30046/ --- Review request for kafka. Bugs: KAFKA-1879 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1874) missing import util.parsing.json.JSON

2015-01-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282868#comment-14282868 ] Jun Rao commented on KAFKA-1874: You need to add the dependency to scala-parser-combinator

[jira] [Updated] (KAFKA-1876) pom file for scala 2.11 should reference a specific version

2015-01-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1876: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the reviews. Committed to 0.8.2

Re: Review Request 30046: Patch for KAFKA-1879

2015-01-19 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30046/#review68659 --- core/src/main/scala/kafka/server/KafkaApis.scala

[jira] [Updated] (KAFKA-1144) commitOffsets can be passed the offsets to commit

2015-01-19 Thread Tony Stevenson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Stevenson updated KAFKA-1144: -- Reporter: Imran Rashid (was: Imran Rashid) > commitOffsets can be passed the offsets to commit

[jira] [Created] (KAFKA-1880) Add support for checking binary/source compatibility

2015-01-19 Thread Ashish Kumar Singh (JIRA)
Ashish Kumar Singh created KAFKA-1880: - Summary: Add support for checking binary/source compatibility Key: KAFKA-1880 URL: https://issues.apache.org/jira/browse/KAFKA-1880 Project: Kafka

Re: Review Request 30046: Patch for KAFKA-1879

2015-01-19 Thread Jun Rao
> On Jan. 19, 2015, 7:22 p.m., Jun Rao wrote: > > core/src/main/scala/kafka/server/KafkaApis.scala, lines 197-201 > > > > > > Should we include the actual requiredAcks used in the request? Also, we are not deprecating

[jira] [Created] (KAFKA-1881) transient unit test failure in testDeleteTopicWithCleaner due to OOME

2015-01-19 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-1881: -- Summary: transient unit test failure in testDeleteTopicWithCleaner due to OOME Key: KAFKA-1881 URL: https://issues.apache.org/jira/browse/KAFKA-1881 Project: Kafka Issu

[jira] [Commented] (KAFKA-1881) transient unit test failure in testDeleteTopicWithCleaner due to OOME

2015-01-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282917#comment-14282917 ] Jun Rao commented on KAFKA-1881: Perhaps we can just reduce the amount of data written in

Build failed in Jenkins: Kafka-trunk #374

2015-01-19 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA-1876; pom file for scala 2.11 should reference a specific version; patched by Jun Rao; reviewed by Joe Stein and Sriharsha Chintalapani -- [...truncated 1289 lines...] kaf

Re: Review Request 30046: Patch for KAFKA-1879

2015-01-19 Thread Gwen Shapira
> On Jan. 19, 2015, 7:22 p.m., Jun Rao wrote: > > core/src/main/scala/kafka/server/KafkaApis.scala, lines 197-201 > > > > > > Should we include the actual requiredAcks used in the request? > > Jun Rao wrote: > Als

[jira] [Updated] (KAFKA-1879) Log warning when receiving produce requests with acks > 1

2015-01-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1879: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patch. +1. Committed to 0.8.

[jira] [Assigned] (KAFKA-1881) transient unit test failure in testDeleteTopicWithCleaner due to OOME

2015-01-19 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira reassigned KAFKA-1881: --- Assignee: Gwen Shapira > transient unit test failure in testDeleteTopicWithCleaner due to OOM

Kafka Out of Memory error

2015-01-19 Thread Pranay Agarwal
Hi All, I have a kafka cluster setup which has 2 topics topic1 with 10 partitions topic2 with 1000 partitions. While, I am able to consume messages from topic1 just fine, I get following error from the topic2. There is a resolved issue here on the same thing https://issues.apache.org/jira/browse

[jira] [Commented] (KAFKA-1881) transient unit test failure in testDeleteTopicWithCleaner due to OOME

2015-01-19 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283087#comment-14283087 ] Gwen Shapira commented on KAFKA-1881: - We can (although we are looking at around 2K of

[VOTE CANCELLED] 0.8.2.0 Candidate 1

2015-01-19 Thread Jun Rao
Thanks for reporting the issues in RC1. I will prepare RC2 and start a new vote. Jun On Tue, Jan 13, 2015 at 7:16 PM, Jun Rao wrote: > This is the first candidate for release of Apache Kafka 0.8.2.0. There > has been some changes since the 0.8.2 beta release, especially in the new > java produc

Re: Kafka Out of Memory error

2015-01-19 Thread Jonathan Natkins
Hi Pranay, I think the JIRA you're referencing is a bit orthogonal to the OOME that you're experiencing. Based on the stacktrace, it looks like your OOME is coming from a consumer request, which is attempting to allocate 200MB. There was a thread (relatively recently) that discussed what I think i

[jira] [Commented] (KAFKA-1824) in ConsoleProducer - properties key.separator and parse.key no longer work

2015-01-19 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283097#comment-14283097 ] Gwen Shapira commented on KAFKA-1824: - I noticed that [~junrao] committed KAFKA-1711 t

Re: Kafka Out of Memory error

2015-01-19 Thread Pranay Agarwal
Thanks Natty. Is there any config which I need to change on the client side as well? Also, currently I am trying with only 1 consumer thread. Does the equation changes to (#partitions)*(fetchsize)*(#consumer_threads) in case I try to read with 1000 threads from from topic2(1000 partitions)? -Pran

[jira] [Commented] (KAFKA-1688) Add authorization interface and naive implementation

2015-01-19 Thread Don Bosco Durai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283104#comment-14283104 ] Don Bosco Durai commented on KAFKA-1688: [~charmalloc], can you help me create a n

[jira] [Commented] (KAFKA-1688) Add authorization interface and naive implementation

2015-01-19 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283108#comment-14283108 ] Joe Stein commented on KAFKA-1688: -- what is your confluence username so I can grant you p

[jira] [Created] (KAFKA-1882) Create extendable channel interface and default implementations

2015-01-19 Thread Gwen Shapira (JIRA)
Gwen Shapira created KAFKA-1882: --- Summary: Create extendable channel interface and default implementations Key: KAFKA-1882 URL: https://issues.apache.org/jira/browse/KAFKA-1882 Project: Kafka

[jira] [Updated] (KAFKA-1882) Create extendable channel interface and default implementations

2015-01-19 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1882: Description: For the security infrastructure, we need an extendible interface to replace SocketChan

Re: Kafka Out of Memory error

2015-01-19 Thread Jonathan Natkins
The fetch.message.max.size is actually a client-side configuration. With regard to increasing the number of threads, I think the calculation may be a little more subtle than what you're proposing, and frankly, it's unlikely that your servers can handle allocating 200MB x 1000 threads = 200GB of mem

Re: Kafka Out of Memory error

2015-01-19 Thread Pranay Agarwal
Thanks a lot Natty. I am using this Ruby gem on the client side with all the default config https://github.com/joekiller/jruby-kafka/blob/master/lib/jruby-kafka/group.rb and the value fetch.message.max.bytes is set to 1MB. Currently I only have 3 nodes setup in the Kafka cluster (with 8 GB RAM) a

[jira] [Commented] (KAFKA-1688) Add authorization interface and naive implementation

2015-01-19 Thread Don Bosco Durai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283155#comment-14283155 ] Don Bosco Durai commented on KAFKA-1688: My id is "bosco". I think I might already

[jira] [Commented] (KAFKA-1688) Add authorization interface and naive implementation

2015-01-19 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283196#comment-14283196 ] Joe Stein commented on KAFKA-1688: -- you can create a child page or such, just checked you

[jira] [Commented] (KAFKA-1824) in ConsoleProducer - properties key.separator and parse.key no longer work

2015-01-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283216#comment-14283216 ] Jun Rao commented on KAFKA-1824: Gwen, thanks for point this out. I am reverting KAFKA-171

Re: NullPointerException in RequestSendThread

2015-01-19 Thread Guozhang Wang
Hi Jaikiran, This is a real bug, could you file a JIRA? As for the fix, I think your proposal would be the right way to fix it. " Guozhang On Mon, Jan 19, 2015 at 9:07 AM, Jaikiran Pai wrote: > I often see the following exception while running some tests > (ProducerFailureHandlingTest.testNoR

Re: Detecting lost connection in high level consumer

2015-01-19 Thread Guozhang Wang
Hi Hari, For high level consumer the fetching logic is handled by a background fetcher thread and is hidden from user, for either case of 1) broker down or 2) no message is available the fetcher thread will keep retrying while the user thread will wait on the fetcher thread to put some data into t

[jira] [Commented] (KAFKA-1872) Update Developer Setup

2015-01-19 Thread Sree Vaddi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283236#comment-14283236 ] Sree Vaddi commented on KAFKA-1872: --- [~omkreddy] Thanks for the pointer. I changed the o

[jira] [Comment Edited] (KAFKA-1872) Update Developer Setup

2015-01-19 Thread Sree Vaddi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283236#comment-14283236 ] Sree Vaddi edited comment on KAFKA-1872 at 1/20/15 12:51 AM: -

[jira] [Reopened] (KAFKA-1824) in ConsoleProducer - properties key.separator and parse.key no longer work

2015-01-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao reopened KAFKA-1824: Reopening the issue since the followup patch hasn't been committed yet. > in ConsoleProducer - properties key.s

Re: Hi Kafka

2015-01-19 Thread Guozhang Wang
Hi Sree, I saw you have created a new wiki for this, thanks! Could you check the see if it can be merged into this page? https://cwiki.apache.org/confluence/display/KAFKA/Developer+Setup Guozhang On Sat, Jan 17, 2015 at 6:55 AM, Sree V wrote: > Hi All, > I am new to Apache Kafka. I use it at

[jira] [Commented] (KAFKA-1873) scalatest_2.10-1.9.1.jar of core build path is cross-compiled with an incompatible version of Scala (2.10.0)

2015-01-19 Thread Sree Vaddi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283245#comment-14283245 ] Sree Vaddi commented on KAFKA-1873: --- [~junrao] This fixes it. kafka/gradle.properties s

[jira] [Commented] (KAFKA-1874) missing import util.parsing.json.JSON

2015-01-19 Thread Sree Vaddi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283247#comment-14283247 ] Sree Vaddi commented on KAFKA-1874: --- [~junrao] [~omkreddy] This fixes it. kafka/gradle.

[jira] [Resolved] (KAFKA-1873) scalatest_2.10-1.9.1.jar of core build path is cross-compiled with an incompatible version of Scala (2.10.0)

2015-01-19 Thread Sree Vaddi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sree Vaddi resolved KAFKA-1873. --- Resolution: Fixed Reviewer: Manikumar Reddy > scalatest_2.10-1.9.1.jar of core build path is cro

[jira] [Commented] (KAFKA-1872) Update Developer Setup

2015-01-19 Thread Sree Vaddi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283253#comment-14283253 ] Sree Vaddi commented on KAFKA-1872: --- [~junrao] Would you move https://issues.apache.org/

[jira] [Commented] (KAFKA-1872) Update Developer Setup

2015-01-19 Thread Sree Vaddi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283248#comment-14283248 ] Sree Vaddi commented on KAFKA-1872: --- Updated the wiki, as well. > Update Developer Setu

[jira] [Updated] (KAFKA-1872) Update Developer Setup

2015-01-19 Thread Sree Vaddi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sree Vaddi updated KAFKA-1872: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Update Developer Setup > --

[jira] [Updated] (KAFKA-1874) missing import util.parsing.json.JSON

2015-01-19 Thread Sree Vaddi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sree Vaddi updated KAFKA-1874: -- Resolution: Fixed Status: Resolved (was: Patch Available) > missing import util.parsing.json.JS

[jira] [Updated] (KAFKA-1874) missing import util.parsing.json.JSON

2015-01-19 Thread Sree Vaddi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sree Vaddi updated KAFKA-1874: -- Reviewer: Manikumar Reddy Status: Patch Available (was: Open) > missing import util.parsing.json.

Re: Hi Kafka

2015-01-19 Thread Sree V
Hi Guozhang, I will sure. For now, I had put a new link with date on top of the previous entries.So, people know, where to go first. Thanking you. With Regards Sree On Monday, January 19, 2015 4:58 PM, Guozhang Wang wrote: Hi Sree, I saw you have created a new wiki for this, than

Re: Kafka Out of Memory error

2015-01-19 Thread Gwen Shapira
Two things: 1. The OOM happened on the consumer, right? So the memory that matters is the RAM on the consumer machine, not on the Kafka cluster nodes. 2. If the consumers belong to the same consumer group, each will consume a subset of the partitions and will only need to allocate memory for those

[VOTE] 0.8.2.0 Candidate 2

2015-01-19 Thread Jun Rao
This is the second candidate for release of Apache Kafka 0.8.2.0. There has been some changes since the 0.8.2 beta release, especially in the new java producer api and jmx mbean names. It would be great if people can test this out thoroughly. Release Notes for the 0.8.2.0 release *https://people.a

[jira] [Updated] (KAFKA-1824) in ConsoleProducer - properties key.separator and parse.key no longer work

2015-01-19 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1824: Status: Patch Available (was: Reopened) > in ConsoleProducer - properties key.separator and parse.k

[jira] [Commented] (KAFKA-1824) in ConsoleProducer - properties key.separator and parse.key no longer work

2015-01-19 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283351#comment-14283351 ] Gwen Shapira commented on KAFKA-1824: - Thanks, Jun. I changed status to Patch Availab

[jira] [Created] (KAFKA-1883) NullPointerException in RequestSendThread

2015-01-19 Thread jaikiran pai (JIRA)
jaikiran pai created KAFKA-1883: --- Summary: NullPointerException in RequestSendThread Key: KAFKA-1883 URL: https://issues.apache.org/jira/browse/KAFKA-1883 Project: Kafka Issue Type: Bug

Review Request 30062: Patch for KAFKA-1883

2015-01-19 Thread Jaikiran Pai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30062/ --- Review request for kafka. Bugs: KAFKA-1883 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1883) NullPointerException in RequestSendThread

2015-01-19 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283366#comment-14283366 ] jaikiran pai commented on KAFKA-1883: - Created reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1883) NullPointerException in RequestSendThread

2015-01-19 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaikiran pai updated KAFKA-1883: Status: Patch Available (was: Open) > NullPointerException in RequestSendThread > -

[jira] [Updated] (KAFKA-1883) NullPointerException in RequestSendThread

2015-01-19 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaikiran pai updated KAFKA-1883: Attachment: KAFKA-1883.patch > NullPointerException in RequestSendThread > -

Re: NullPointerException in RequestSendThread

2015-01-19 Thread Jaikiran Pai
JIRA created https://issues.apache.org/jira/browse/KAFKA-1883 and patch submitted for review. Thanks Guozhang. -Jaikiran On Tuesday 20 January 2015 05:53 AM, Guozhang Wang wrote: Hi Jaikiran, This is a real bug, could you file a JIRA? As for the fix, I think your proposal would be the right

Re: Review Request 29647: Patch for KAFKA-1697

2015-01-19 Thread Eric Olander
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29647/#review68687 --- core/src/main/scala/kafka/cluster/Partition.scala

Review Request 30063: Patch for KAFKA-1840

2015-01-19 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30063/ --- Review request for kafka. Bugs: KAFKA-1840 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1840) Add a simple message handler in Mirror Maker

2015-01-19 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283390#comment-14283390 ] Jiangjie Qin commented on KAFKA-1840: - Created reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1840) Add a simple message handler in Mirror Maker

2015-01-19 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1840: Attachment: KAFKA-1840.patch > Add a simple message handler in Mirror Maker > --

[jira] [Updated] (KAFKA-1840) Add a simple message handler in Mirror Maker

2015-01-19 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1840: Assignee: Jiangjie Qin Status: Patch Available (was: Open) > Add a simple message handler in

[jira] [Commented] (KAFKA-1881) transient unit test failure in testDeleteTopicWithCleaner due to OOME

2015-01-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283419#comment-14283419 ] Jun Rao commented on KAFKA-1881: Not sure id writeDups is causing the issue. I just ran gr

Re: [DISCUSS] KIPs

2015-01-19 Thread Gwen Shapira
I created a KIP for the multi-port broker change. https://cwiki.apache.org/confluence/display/KAFKA/KIP-2+-+Refactor+brokers+to+allow+listening+on+multiple+ports+and+IPs I'm not re-opening the discussion, since it was agreed on over a month ago and implementation is close to complete (I hope!). Le

[jira] [Assigned] (KAFKA-1334) Add failure detection capability to the coordinator / consumer

2015-01-19 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman reassigned KAFKA-1334: --- Assignee: Onur Karaman > Add failure detection capability to the coordinator / consumer > ---

[jira] [Assigned] (KAFKA-1877) Expose version via JMX for 'new' producer

2015-01-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy reassigned KAFKA-1877: -- Assignee: Manikumar Reddy > Expose version via JMX for 'new' producer > --

Re: Review Request 27799: Patch for KAFKA-1760

2015-01-19 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27799/#review68519 --- clients/src/main/java/org/apache/kafka/clients/RequestCompletionHan

[jira] [Created] (KAFKA-1884) New Producer blocks forever for Invalid topic names

2015-01-19 Thread Manikumar Reddy (JIRA)
Manikumar Reddy created KAFKA-1884: -- Summary: New Producer blocks forever for Invalid topic names Key: KAFKA-1884 URL: https://issues.apache.org/jira/browse/KAFKA-1884 Project: Kafka Issue T

  1   2   >