[jira] [Updated] (KAFKA-1666) Issue for sending more message to Kafka Broker

2014-10-05 Thread rajendram kathees (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajendram kathees updated KAFKA-1666: - Remaining Estimate: 48h (was: 24h) Original Estimate: 48h (was: 24h) > Issue for se

[jira] [Commented] (KAFKA-1666) Issue for sending more message to Kafka Broker

2014-10-05 Thread rajendram kathees (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160041#comment-14160041 ] rajendram kathees commented on KAFKA-1666: -- Reason for this issue is that produce

Re: [New Java Producer Kafka Trunk Branch ] Getting java.lang.IllegalStateException: Wrong condition: this shouldn't happen.

2014-10-05 Thread Bhavesh Mistry
HI Jay and Jun, Thanks for you quick support and patch https://issues.apache.org/jira/secure/attachment/12673049/KAFKA-1673.patch. I will apply this patch tomorrow and try. On the side note, is there any way to skip metric collection all together ? Thanks, Bhavesh On Sun, Oct 5, 2014 at 10:00

Re: [New Java Producer Kafka Trunk Branch ] Getting java.lang.IllegalStateException: Wrong condition: this shouldn't happen.

2014-10-05 Thread Jay Kreps
Ah, gotcha, yeah, I couldn't figure that out. I see the the trouble now--if you are currently blocked that doesn't contribute to the metic. That makes sense. -Jay On Sun, Oct 5, 2014 at 7:49 PM, Jun Rao wrote: > Sorry, that's my bad. The intention was that in case the producer blocks > there fo

Re: Review Request 26355: Patch for KAFKA-1673

2014-10-05 Thread Jay Kreps
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26355/#review55482 --- Ship it! Ship It! - Jay Kreps On Oct. 6, 2014, 2:44 a.m., Jun Ra

[jira] [Commented] (KAFKA-1305) Controller can hang on controlled shutdown with auto leader balance enabled

2014-10-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159830#comment-14159830 ] Jun Rao commented on KAFKA-1305: Thinking about this a bit more. As a short term fix, I th

[jira] [Commented] (KAFKA-1672) zk timeouts with examples from 8.1.1.1

2014-10-05 Thread Matthew Sandoz (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159817#comment-14159817 ] Matthew Sandoz commented on KAFKA-1672: --- ugh I should have considered this. i did an

Re: [New Java Producer Kafka Trunk Branch ] Getting java.lang.IllegalStateException: Wrong condition: this shouldn't happen.

2014-10-05 Thread Jun Rao
Sorry, that's my bad. The intention was that in case the producer blocks there forever (because memory never frees up for some reason), having a timeout will allow us to get a more accurate stat on the waitTime sensor. Filed and patched kafka-1673. Could you take a look? Thanks, Jun On Sun, Oct

[jira] [Commented] (KAFKA-1673) potential java.lang.IllegalStateException in BufferPool.allocate()

2014-10-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159814#comment-14159814 ] Jun Rao commented on KAFKA-1673: Created reviewboard https://reviews.apache.org/r/26355/di

[jira] [Updated] (KAFKA-1673) potential java.lang.IllegalStateException in BufferPool.allocate()

2014-10-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1673: --- Attachment: KAFKA-1673.patch > potential java.lang.IllegalStateException in BufferPool.allocate() > -

[jira] [Updated] (KAFKA-1673) potential java.lang.IllegalStateException in BufferPool.allocate()

2014-10-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1673: --- Assignee: Jun Rao Status: Patch Available (was: Open) > potential java.lang.IllegalStateException in B

Review Request 26355: Patch for KAFKA-1673

2014-10-05 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26355/ --- Review request for kafka. Bugs: KAFKA-1673 https://issues.apache.org/jira/b

[jira] [Created] (KAFKA-1673) potential java.lang.IllegalStateException in BufferPool.allocate()

2014-10-05 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-1673: -- Summary: potential java.lang.IllegalStateException in BufferPool.allocate() Key: KAFKA-1673 URL: https://issues.apache.org/jira/browse/KAFKA-1673 Project: Kafka Issue

Re: [Java New Producer Kafka Trunk ] Need a State Check API Method

2014-10-05 Thread Bhavesh Mistry
Hi Kafka Dev Team, *java.lang.* *IllegalStateException: Cannot send after the producer is closed.* The above seems to bug. If the ProducerRecord is in flight within send method is execute and another thread seems to shutdown in the middle of flight will get error. Thanks, Bhavesh On Sun, Oct

Re: [New Java Producer Kafka Trunk Branch ] Getting java.lang.IllegalStateException: Wrong condition: this shouldn't happen.

2014-10-05 Thread Bhavesh Mistry
Hi Jay, Thanks for this. We do not want exact stats from the producer. If it is off by some we are ok with it we do not want producer to be blocked. So is there any was to add flag say metric.inconsistent=true would not wait for 300 ms. We really need Async Producer other we will have to wrap

Re: [Java New Producer Kafka Trunk ] Need a State Check API Method

2014-10-05 Thread Bhavesh Mistry
Hi Kafka Dev Team, The use case is that we need to know producer state in background Threads and so we can submit the message. This seems to a bug in trunk code. I have notice that KafkaProducer itself does not have close state and inflight message will encounter following issues. Should I file

[jira] [Commented] (KAFKA-1672) zk timeouts with examples from 8.1.1.1

2014-10-05 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159801#comment-14159801 ] Joe Stein commented on KAFKA-1672: -- What do you get trying to connect to your zookeeper s

[jira] [Created] (KAFKA-1672) zk timeouts with examples from 8.1.1.1

2014-10-05 Thread Matthew Sandoz (JIRA)
Matthew Sandoz created KAFKA-1672: - Summary: zk timeouts with examples from 8.1.1.1 Key: KAFKA-1672 URL: https://issues.apache.org/jira/browse/KAFKA-1672 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-1669) Default rebalance retries and backoff should be higher

2014-10-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159768#comment-14159768 ] Jun Rao commented on KAFKA-1669: Thanks for the patch. A 30 secs backoff time will probabl

[jira] [Commented] (KAFKA-234) Make backoff time during consumer rebalance configurable

2014-10-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159766#comment-14159766 ] Jun Rao commented on KAFKA-234: --- The reasoning is the following. A ZK follower can be up to z

[jira] [Updated] (KAFKA-1667) topic-level configuration not validated

2014-10-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1667: --- Labels: newbie (was: ) > topic-level configuration not validated >

Re: Question regarding topic creation

2014-10-05 Thread Jun Rao
In 0.8.x, a topic doesn't necessarily get created on every broker. Each topic can have one or more partitions. Each partition can have one or more replicas. In general, the replicas are spread over the brokers in a cluster. The producer obtains the metadata of a topic to determine which broker to s

[jira] [Commented] (KAFKA-1644) Inherit FetchResponse from RequestOrResponse

2014-10-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159758#comment-14159758 ] Jun Rao commented on KAFKA-1644: Alexey, My only concern of the patch is that it gives pe

[jira] [Commented] (KAFKA-1664) Kafka does not properly parse multiple ZK nodes with non-root chroot

2014-10-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159755#comment-14159755 ] Jun Rao commented on KAFKA-1664: Perhaps we can add a wrapper in ZkUtils for creating a pa

[jira] [Commented] (KAFKA-1663) Controller unable to shutdown after a soft failure

2014-10-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159752#comment-14159752 ] Jun Rao commented on KAFKA-1663: Neha, Since this is a blocker for 0.8.2, could you push

[jira] [Commented] (KAFKA-1670) Corrupt log files for segment.bytes values close to Int.MaxInt

2014-10-05 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159749#comment-14159749 ] Jun Rao commented on KAFKA-1670: The default value in LogConfig is just for unit test (so

Re: Review Request 26346: Patch for KAFKA-1670

2014-10-05 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26346/#review55479 --- Thanks for the patch. Some comments below. core/src/main/scala/kaf

Re: [New Java Producer Kafka Trunk Branch ] Getting java.lang.IllegalStateException: Wrong condition: this shouldn't happen.

2014-10-05 Thread Jay Kreps
Maybe this was introduced in dcc88408c98a07cb9a816ab55cd81e55f1d2217d as part of KAFKA-1488. We seem to add a hard coded 300 ms timeout on the wait time. After this 300 ms timeout the thread is allowed to allocate memory to itself irrespective of whether it is next in line. This is not right at

[Java New Producer Kafka Trunk ] Need a State Check API Method

2014-10-05 Thread Bhavesh Mistry
HI Kafka Dev, I would like to request state check state so I can manage the Life Cycle of Producer better. I you guys agree I will file Jira request. I just give state of producer can be I would like mange or start (create new instance of producer) or restart or close based on state. I just

[New Java Producer Kafka Trunk Branch ] Getting java.lang.IllegalStateException: Wrong condition: this shouldn't happen.

2014-10-05 Thread Bhavesh Mistry
Hi Kafka Dev team, I am getting following exception occasionally, and data is lost due to this exception.What could cause this error ? We have prod release coming up with New Kafka API end of this week. So I any help is greatly appreciated. * java.lang.IllegalStateException: Wrong condition

[jira] [Updated] (KAFKA-1668) TopicCommand doesn't warn if --topic argument doesn't match any topics

2014-10-05 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-1668: --- Assignee: Manikumar Reddy Status: Patch Available (was: Open) > TopicCommand doesn't wa

[jira] [Updated] (KAFKA-1668) TopicCommand doesn't warn if --topic argument doesn't match any topics

2014-10-05 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-1668: --- Attachment: KAFKA-1668.patch > TopicCommand doesn't warn if --topic argument doesn't match any

[jira] [Commented] (KAFKA-1668) TopicCommand doesn't warn if --topic argument doesn't match any topics

2014-10-05 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159471#comment-14159471 ] Manikumar Reddy commented on KAFKA-1668: Created reviewboard https://reviews.apach

Review Request 26351: Patch for KAFKA-1668

2014-10-05 Thread Manikumar Reddy O
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26351/ --- Review request for kafka. Bugs: KAFKA-1668 https://issues.apache.org/jira/b