Re: Review Request 29301: Patch for KAFKA-1694

2015-01-12 Thread Andrii Biletskyi
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29301/ --- (Updated Jan. 12, 2015, 1:28 p.m.) Review request for kafka. Bugs: KAFKA-1694

[jira] [Updated] (KAFKA-1694) kafka command line and centralized operations

2015-01-12 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1694: Attachment: KAFKA-1694_2015-01-12_15:28:41.patch > kafka command line and centralized operat

[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations

2015-01-12 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273588#comment-14273588 ] Andrii Biletskyi commented on KAFKA-1694: - Updated reviewboard https://reviews.apa

Re: [DISCUSS] Compatability and KIPs

2015-01-12 Thread Jun Rao
Jay, Thanks for bringing this up. Yes, we should increase the level of awareness of compatibility. For 1 and 2, they probably should include any functional change. For example, even if there is no change in the binary data format, but the interpretation is changed, we should consider this as a bi

Re: [DISCUSS] 0.8.2-beta2 release

2015-01-12 Thread Jun Rao
Hi, Joe, Yes, we can do that. Also, an alternative to doing an 0.8.2 beta2 is to just do the 0.8.2 final, but leave a bit more time for people to try the RC. Thanks, Jun On Thu, Jan 8, 2015 at 8:46 PM, Joe Stein wrote: > Hi, I was thinking that once all the blockers are committed for 0.8.2 t

Re: Review Request 29301: Patch for KAFKA-1694

2015-01-12 Thread Andrii Biletskyi
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29301/ --- (Updated Jan. 12, 2015, 4:55 p.m.) Review request for kafka. Bugs: KAFKA-1694

[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations

2015-01-12 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273773#comment-14273773 ] Andrii Biletskyi commented on KAFKA-1694: - Updated reviewboard https://reviews.apa

[jira] [Updated] (KAFKA-1694) kafka command line and centralized operations

2015-01-12 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1694: Attachment: KAFKA-1694_2015-01-12_18:54:48.patch > kafka command line and centralized operat

Re: Review Request 29301: Patch for KAFKA-1694

2015-01-12 Thread Andrii Biletskyi
> On Jan. 6, 2015, 8:19 a.m., Joe Stein wrote: > > build.gradle, line 211 > > > > > > If we can do this without an upgrade that would be great if we are in > > fact just requiring 1 function. This feature allows us t

Re: [DISCUSS] Compatability and KIPs

2015-01-12 Thread Jay Kreps
Hey Jun, Good points. I totally agree that the versioning needs to cover both format and behavior if the behavior change is incompatible. I kind of agree about the stable/unstable stuff. What I think this means is not that we would ever evolve the protocol without changing the version, but rathe

Re: New consumer client

2015-01-12 Thread Jay Kreps
Hey Bhavesh, This seems like a serious issue and not one anyone else has reported. I don't know what you mean by corrupt message, are you saying the CRC check fails? If so, that check is done both by the broker (prior to appending to the log) and the consumer so that implies either a bug in the br

Re: [DISCUSS] 0.8.2-beta2 release

2015-01-12 Thread Neha Narkhede
Joe, Thanks for starting the discussion. It may be ok to do an RC and probably two weeks later, cut the final. After releasing the RC, we can help spread the word and have people try out the new API, which is probably our main concern. -Neha On Mon, Jan 12, 2015 at 8:42 AM, Jun Rao wrote: > Hi

[jira] [Commented] (KAFKA-725) Broker Exception: Attempt to read with a maximum offset less than start offset

2015-01-12 Thread Diwakar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273861#comment-14273861 ] Diwakar commented on KAFKA-725: --- Neha, we have 6 brokers and 131 partitions per topic(replic

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29210/ --- (Updated Jan. 12, 2015, 6:34 p.m.) Review request for kafka. Bugs: KAFKA-1819

[jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273906#comment-14273906 ] Gwen Shapira commented on KAFKA-1819: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1819: Attachment: KAFKA-1819_2015-01-12_10:34:07.patch > Cleaner gets confused about deleted and re-create

[jira] [Created] (KAFKA-1858) Make ServerShutdownTest a bit less flaky

2015-01-12 Thread Gwen Shapira (JIRA)
Gwen Shapira created KAFKA-1858: --- Summary: Make ServerShutdownTest a bit less flaky Key: KAFKA-1858 URL: https://issues.apache.org/jira/browse/KAFKA-1858 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273918#comment-14273918 ] Neha Narkhede commented on KAFKA-1819: -- [~gwenshap] Thanks for looking into the possi

[jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273921#comment-14273921 ] Gwen Shapira commented on KAFKA-1819: - Ouch. Yes. Good catch! Let me check how this

[jira] [Commented] (KAFKA-1856) Add PreCommit Patch Testing

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273923#comment-14273923 ] Neha Narkhede commented on KAFKA-1856: -- Thanks for filing this [~singhashish]! Look f

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23702/ --- (Updated Jan. 12, 2015, 6:46 p.m.) Review request for kafka. Bugs: KAFKA-1070

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
> On Jan. 9, 2015, 11:39 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataFileHandler.scala, line 30 > > > > > > I think the way you modelled this checkpoint file, which makes it > > differ

[jira] [Updated] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1070: -- Attachment: KAFKA-1070_2015-01-12_10:46:54.patch > Auto-assign node id > ---

[jira] [Commented] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273928#comment-14273928 ] Sriharsha Chintalapani commented on KAFKA-1070: --- Updated reviewboard https:/

[jira] [Resolved] (KAFKA-1857) Kafka Broker ids are removed ( with zookeeper , Storm )

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-1857. -- Resolution: Invalid Please send such questions on the mailing list. It has higher visibility and

[jira] [Closed] (KAFKA-1857) Kafka Broker ids are removed ( with zookeeper , Storm )

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede closed KAFKA-1857. Assignee: (was: Neha Narkhede) > Kafka Broker ids are removed ( with zookeeper , Storm ) > -

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
> On Jan. 9, 2015, 11:39 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataFileHandler.scala, line 48 > > > > > > Does java serialization for the Properties object allow you to read > > ju

[jira] [Commented] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273946#comment-14273946 ] Sriharsha Chintalapani commented on KAFKA-1070: --- [~nehanarkhede] updated pat

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Neha Narkhede
> On Jan. 9, 2015, 11:39 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataFileHandler.scala, line 48 > > > > > > Does java serialization for the Properties object allow you to read > > ju

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
> On Jan. 9, 2015, 11:39 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataFileHandler.scala, line 48 > > > > > > Does java serialization for the Properties object allow you to read > > ju

[jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273999#comment-14273999 ] Gwen Shapira commented on KAFKA-1819: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29210/ --- (Updated Jan. 12, 2015, 7:17 p.m.) Review request for kafka. Bugs: KAFKA-1819

[jira] [Updated] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1819: Attachment: KAFKA-1819_2015-01-12_11:17:53.patch > Cleaner gets confused about deleted and re-create

Re: Review Request 29692: Patch for kafka-1841

2015-01-12 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29692/#review67667 --- Thanks for patching this. Looks good overall. clients/src/main/jav

Compatibility + Unknown APIs

2015-01-12 Thread Dana Powers
Hi all -- continuing on the compatibility discussion: I've found that it is very difficult to identify when a server does not recognize an api (I'm using kafka-python to submit wire-protocol requests). For example, when I send a ConsumerMetadataRequest to an 0.8.1.1 server, I get a closed socket

Re: Compatibility + Unknown APIs

2015-01-12 Thread Gwen Shapira
We ran into similar difficulties, both when trying to get Kafka to use new APIs when available and when testing the wire protocol. +1 for all three suggestions. #1 sounds like the bare minimum, but I'm not sure how much it will complicate the clients (now we expect either a response or an Unknown

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Neha Narkhede
> On Jan. 9, 2015, 11:39 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataFileHandler.scala, line 48 > > > > > > Does java serialization for the Properties object allow you to read > > ju

Re: Compatibility + Unknown APIs

2015-01-12 Thread Jay Kreps
Yeah I agree there isn't a good way to auto-detect the presence of a given API. I think #1 may be tricky in practice. The response format is always dictated by the request format so how do I know if the bytes I got back are a valid response to the given request or are the UnknownRequestResponse?

Kafka Cluster Monitoring and Documentation of Internals (JMX Metrics) of Rejected Events

2015-01-12 Thread Bhavesh Mistry
Hi Kafka Team, I am trying to find out Kafka Internal and how a message can be corrupted or lost at brokers side. I have refer to following documentations for monitoring: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Internals http://kafka.apache.org/documentation.html#monitoring I am

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Neha Narkhede
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23702/#review67715 --- Ship it! Ship It! core/src/main/scala/kafka/server/BrokerMetadata

Re: Compatibility + Unknown APIs

2015-01-12 Thread Gwen Shapira
> I think #1 may be tricky in practice. The response format is always > dictated by the request format so how do I know if the bytes I got back are > a valid response to the given request or are the UnknownRequestResponse? On the other hand, from the client developer perspective, having to figure

Re: Review Request 29692: Patch for kafka-1841

2015-01-12 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29692/ --- (Updated Jan. 12, 2015, 10:30 p.m.) Review request for kafka. Bugs: kafka-184

[jira] [Commented] (KAFKA-1841) OffsetCommitRequest API - timestamp field is not versioned

2015-01-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274301#comment-14274301 ] Jun Rao commented on KAFKA-1841: Updated reviewboard https://reviews.apache.org/r/29692/di

[jira] [Updated] (KAFKA-1841) OffsetCommitRequest API - timestamp field is not versioned

2015-01-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1841: --- Attachment: kafka-1841_2015-01-12_14:30:24.patch > OffsetCommitRequest API - timestamp field is not versioned

[jira] [Updated] (KAFKA-1835) Kafka new producer needs options to make blocking behavior explicit

2015-01-12 Thread Paul Pearcy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pearcy updated KAFKA-1835: --- Attachment: KAFKA-1835-New-producer--blocking_v0.patch > Kafka new producer needs options to make bloc

[jira] [Updated] (KAFKA-1835) Kafka new producer needs options to make blocking behavior explicit

2015-01-12 Thread Paul Pearcy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pearcy updated KAFKA-1835: --- Affects Version/s: 0.9.0 0.8.3 Status: Patch Available (was: Ope

Re: Review Request 29692: Patch for kafka-1841

2015-01-12 Thread Jun Rao
> On Jan. 12, 2015, 7:47 p.m., Joel Koshy wrote: > > core/src/main/scala/kafka/server/KafkaApis.scala, line 88 > > > > > > We should probably also change OffsetCommitRequest.responseFor . The > > issue is that if you g

[jira] [Updated] (KAFKA-1835) Kafka new producer needs options to make blocking behavior explicit

2015-01-12 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1835: - Fix Version/s: 0.8.2 > Kafka new producer needs options to make blocking behavior explicit > -

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29210/#review67717 --- Minor comments core/src/test/scala/unit/kafka/admin/DeleteTopicTes

[jira] [Updated] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1070: - Resolution: Fixed Status: Resolved (was: Patch Available) [~sriharsha] Thanks for your ef

Re: Compatibility + Unknown APIs

2015-01-12 Thread Jay Kreps
Yeah I totally agree--using the closed socket to indicate "not supported" does work since any network error could lead to that. Arguably we should have a request level error. We discussed this at the time we were defining the protocols for 0.8 and the conclusion was not to do that. The reasoning w

[jira] [Updated] (KAFKA-1836) metadata.fetch.timeout.ms set to zero blocks forever

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1836: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patches! Pushed

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274451#comment-14274451 ] Onur Karaman commented on KAFKA-1476: - I have some stuff for sub-task KAFKA-1826 that

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274455#comment-14274455 ] Neha Narkhede commented on KAFKA-1476: -- [~balaji.sesha...@dish.com], [~onurkaraman] T

Review Request 29831: Patch for KAFKA-1476

2015-01-12 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/ --- Review request for kafka. Bugs: KAFKA-1476 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274460#comment-14274460 ] Onur Karaman commented on KAFKA-1476: - Created reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: KAFKA-1476.patch > Get a list of consumer groups > - > >

Build failed in Jenkins: Kafka-trunk #364

2015-01-12 Thread Apache Jenkins Server
See Changes: [neha.narkhede] KAFKA-1070 Auto assign broker id; reviewed by Neha Narkhede [neha.narkhede] KAFKA-1836 metadata.fetch.timeout.ms set to zero blocks forever; reviewed by Neha Narkhede and Ewen Cheslack-Postava

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: KAFKA-1476_2015-01-12_16:22:26.patch > Get a list of consumer groups > -

Re: Review Request 29831: Patch for KAFKA-1476

2015-01-12 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/ --- (Updated Jan. 13, 2015, 12:22 a.m.) Review request for kafka. Bugs: KAFKA-147

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274473#comment-14274473 ] Onur Karaman commented on KAFKA-1476: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 29831: Patch for KAFKA-1476

2015-01-12 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/ --- (Updated Jan. 13, 2015, 12:31 a.m.) Review request for kafka. Bugs: KAFKA-147

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274481#comment-14274481 ] Onur Karaman commented on KAFKA-1476: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: KAFKA-1476_2015-01-12_16:31:20.patch > Get a list of consumer groups > -

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Gwen Shapira
> On Jan. 12, 2015, 10:53 p.m., Joel Koshy wrote: > > core/src/test/scala/unit/kafka/log/LogCleanerIntegrationTest.scala, line 68 > > > > > > Actually, now that you have the check above in DeleteTopicTest, do you > > t

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Gwen Shapira
> On Jan. 12, 2015, 10:53 p.m., Joel Koshy wrote: > > core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala, line 251 > > > > > > Rather than an arbitrary sleep can we just use waitUntilTrue and the > > condition

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29210/ --- (Updated Jan. 13, 2015, 1:01 a.m.) Review request for kafka. Bugs: KAFKA-1819

[jira] [Updated] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1819: Attachment: KAFKA-1819_2015-01-12_17:01:53.patch > Cleaner gets confused about deleted and re-create

[jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274530#comment-14274530 ] Gwen Shapira commented on KAFKA-1819: - Updated reviewboard https://reviews.apache.org/

Re: [DISCUSS] Compatability and KIPs

2015-01-12 Thread Guozhang Wang
+1 on version evolving with any protocol / data format / functionality changes, and I am wondering if we have a standard process of deprecating old versions? Today with just a couple of versions for the protocol (e.g. offset commit) the code on the server side is already pretty nested and complicat

Re: Compatibility + Unknown APIs

2015-01-12 Thread Dana Powers
Perhaps a bit hacky, but you could also reserve a specific correlationId (maybe -1) to represent errors and send back to the client an UnknownAPIResponse like: Response => -1 UnknownAPIResponse UnknownAPIResponse => originalCorrelationId errorCode The benefit here would be that it does not break

Re: [DISCUSS] Compatability and KIPs

2015-01-12 Thread Jay Kreps
Yeah I think this makes sense. Some of the crazy nesting will get better when we move to the new protocol definition I think, but we will always need some kind of if statement that branches for the different behavior and this makes testing difficult. Probably the best thing to do would be to annou

Re: Compatibility + Unknown APIs

2015-01-12 Thread Jay Kreps
Yeah, adding it to the metadata request probably makes sense. What you describe of making it a per-broker field is technically correct, since each broker could be on a different software version. But I wonder if it might not be more usable to just give back a single list of api versions. This will

Re: Review Request 29692: Patch for kafka-1841

2015-01-12 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29692/#review67775 --- Ship it! Reversed an earlier comment, but up to you to pick which o

Re: [DISCUSS] Compatability and KIPs

2015-01-12 Thread Joe Stein
Having an index for every protocol/API change (like https://www.python.org/dev/peps/pep-0257/ ) will be much better than the flat wire protocol doc we have now. It is impossible ( without jumping into code ) right now to know if an error is supported in one version of Kafka vs another or different

Re: Compatibility + Unknown APIs

2015-01-12 Thread Joe Stein
There are benefits of moving the error code to the response header. 1) I think it is the right thing to-do from an implementation perspective. It makes the most sense. You send a request and you get back a response. The response tells you something is wrong in the header. 2) With such a large cha

[jira] [Commented] (KAFKA-1857) Kafka Broker ids are removed ( with zookeeper , Storm )

2015-01-12 Thread Yoonhyeok Kim (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274595#comment-14274595 ] Yoonhyeok Kim commented on KAFKA-1857: -- [~nehanarkhede] Okay, Thank you. But I don't

Re: [DISCUSS] Compatability and KIPs

2015-01-12 Thread Jay Kreps
Hey Joe, Yeah I think a lot of those items are limitations in that document that we should definitely fix. The first issue you point out is a serious one: We give the total list of errors but don't list which errors can result from which APIs. This is a big issue because actually no one knows and

Re: Kafka Cluster Monitoring and Documentation of Internals (JMX Metrics) of Rejected Events

2015-01-12 Thread Otis Gospodnetic
Hi, I think you could just email user@? There was no attached image. I think Jun committed something about this: https://issues.apache.org/jira/browse/KAFKA-1481?focusedCommentId=14272057&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14272057 Otis -- Monitoring * A

[jira] [Updated] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1070: -- Attachment: KAFKA-1070_2015-01-12_18:30:17.patch > Auto-assign node id > ---

[jira] [Commented] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274614#comment-14274614 ] Sriharsha Chintalapani commented on KAFKA-1070: --- Updated reviewboard https:/

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23702/ --- (Updated Jan. 13, 2015, 2:30 a.m.) Review request for kafka. Bugs: KAFKA-1070

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
> On Jan. 12, 2015, 10:08 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataCheckpoint.scala, line 55 > > > > > > Minor nit which I can address during checkin: According to our coding > > c

Re: Compatibility + Unknown APIs

2015-01-12 Thread Jay Kreps
I totally agree but I still think we shouldn't do it. :-) That change would cause the reimplementation of ALL existing Kafka clients. (You can't chose not to implement a new protocol version or else we are committing to keeping the old version supported both ways on the server forever). The probl

[jira] [Commented] (KAFKA-1857) Kafka Broker ids are removed ( with zookeeper , Storm )

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274620#comment-14274620 ] Neha Narkhede commented on KAFKA-1857: -- You might find this useful - http://kafka.apa

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Neha Narkhede
> On Jan. 12, 2015, 10:53 p.m., Joel Koshy wrote: > > core/src/test/scala/unit/kafka/log/LogCleanerIntegrationTest.scala, line 68 > > > > > > Actually, now that you have the check above in DeleteTopicTest, do you > > t

[jira] [Commented] (KAFKA-1850) Failed reassignment leads to additional replica

2015-01-12 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274657#comment-14274657 ] Joe Stein commented on KAFKA-1850: -- The reassignment isn't going to be able to finish unt

Review Request 29840: Patch for KAFKA-1818

2015-01-12 Thread Eric Olander
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29840/ --- Review request for kafka. Bugs: KAFKA-1818 https://issues.apache.org/jira/b

Re: Review Request 29756: Patch for KAFKA-1854

2015-01-12 Thread Neha Narkhede
> On Jan. 9, 2015, 6:48 p.m., Neha Narkhede wrote: > > kafka-patch-review.py, line 20 > > > > > > I got the following error on this patch > > > > nnarkhed-mn1:kafka nnarkhed$ python kafka-patch-review.py -b tr

Re: Review Request 29756: Patch for KAFKA-1854

2015-01-12 Thread Neha Narkhede
> On Jan. 9, 2015, 6:48 p.m., Neha Narkhede wrote: > > kafka-patch-review.py, line 20 > > > > > > I got the following error on this patch > > > > nnarkhed-mn1:kafka nnarkhed$ python kafka-patch-review.py -b tr

[jira] [Updated] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1819: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patches, [~gwens

Re: Review Request 29831: Patch for KAFKA-1476

2015-01-12 Thread Neha Narkhede
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/#review67789 --- To make review easier, could you add the output of the command for a

Re: Review Request 29756: Patch for KAKFA-1854

2015-01-12 Thread Jaikiran Pai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29756/ --- (Updated Jan. 13, 2015, 5:42 a.m.) Review request for kafka. Summary (updated

Re: Review Request 29756: Patch for KAKFA-1854

2015-01-12 Thread Jaikiran Pai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29756/ --- (Updated Jan. 13, 2015, 5:43 a.m.) Review request for kafka. Changes ---

Re: Review Request 29756: Patch for KAKFA-1854

2015-01-12 Thread Jaikiran Pai
> On Jan. 9, 2015, 6:48 p.m., Neha Narkhede wrote: > > kafka-patch-review.py, line 20 > > > > > > I got the following error on this patch > > > > nnarkhed-mn1:kafka nnarkhed$ python kafka-patch-review.py -b tr

[jira] [Updated] (KAFKA-1760) Implement new consumer client

2015-01-12 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1760: - Fix Version/s: 0.8.3 > Implement new consumer client > - > > K

[jira] [Updated] (KAFKA-1723) make the metrics name in new producer more standard

2015-01-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1723: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patch. +1. Committed to 0.8.

[jira] [Commented] (KAFKA-1854) Allow the JIRA username and password to be prompted during patch submission

2015-01-12 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274783#comment-14274783 ] Guozhang Wang commented on KAFKA-1854: -- Thanks for the patch, I have added you to the

[jira] [Assigned] (KAFKA-1854) Allow the JIRA username and password to be prompted during patch submission

2015-01-12 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaikiran pai reassigned KAFKA-1854: --- Assignee: jaikiran pai > Allow the JIRA username and password to be prompted during patch sub

[jira] [Commented] (KAFKA-1854) Allow the JIRA username and password to be prompted during patch submission

2015-01-12 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274787#comment-14274787 ] jaikiran pai commented on KAFKA-1854: - Thank you [~guozhang]! > Allow the JIRA usern

  1   2   >