Re: unable to sign up for kafka email distribution

2015-01-12 Thread Guozhang Wang
Looks good to me, maybe you can check if your email account is bouncing emails from this mail list or not. On Thu, Jan 8, 2015 at 11:05 AM, Padgett, Ben wrote: > I haven’t gotten any emails from the email list however. When I joined > the Apache Cassandra list I got very frequent emails. > > W

Re: [DISCUSS] 0.8.2-beta2 release

2015-01-12 Thread Joe Stein
So are we saying a) RC in regards to an official Apache VOTE and keeping it open for week(s) and continuously patching as issues come up calling more RC VOTEs until we ship? b) instead of beta name the release RC1 so in Maven Central it would be 0.8.2-RC1 and we could have RCX votes for that? If

[jira] [Commented] (KAFKA-1634) Improve semantics of timestamp in OffsetCommitRequests and update documentation

2015-01-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274801#comment-14274801 ] Jun Rao commented on KAFKA-1634: Committed KAFKA-1841 to 0.8.2. It would be easier to inco

[jira] [Updated] (KAFKA-1841) OffsetCommitRequest API - timestamp field is not versioned

2015-01-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1841: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the review. Committed to 0.8.2.

Jenkins build is back to normal : Kafka-trunk #365

2015-01-12 Thread Apache Jenkins Server
See

[jira] [Commented] (KAFKA-1854) Allow the JIRA username and password to be prompted during patch submission

2015-01-12 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274787#comment-14274787 ] jaikiran pai commented on KAFKA-1854: - Thank you [~guozhang]! > Allow the JIRA usern

[jira] [Assigned] (KAFKA-1854) Allow the JIRA username and password to be prompted during patch submission

2015-01-12 Thread jaikiran pai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jaikiran pai reassigned KAFKA-1854: --- Assignee: jaikiran pai > Allow the JIRA username and password to be prompted during patch sub

[jira] [Commented] (KAFKA-1854) Allow the JIRA username and password to be prompted during patch submission

2015-01-12 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274783#comment-14274783 ] Guozhang Wang commented on KAFKA-1854: -- Thanks for the patch, I have added you to the

[jira] [Updated] (KAFKA-1723) make the metrics name in new producer more standard

2015-01-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1723: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patch. +1. Committed to 0.8.

[jira] [Updated] (KAFKA-1760) Implement new consumer client

2015-01-12 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1760: - Fix Version/s: 0.8.3 > Implement new consumer client > - > > K

Re: Review Request 29756: Patch for KAKFA-1854

2015-01-12 Thread Jaikiran Pai
> On Jan. 9, 2015, 6:48 p.m., Neha Narkhede wrote: > > kafka-patch-review.py, line 20 > > > > > > I got the following error on this patch > > > > nnarkhed-mn1:kafka nnarkhed$ python kafka-patch-review.py -b tr

Re: Review Request 29756: Patch for KAKFA-1854

2015-01-12 Thread Jaikiran Pai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29756/ --- (Updated Jan. 13, 2015, 5:43 a.m.) Review request for kafka. Changes ---

Re: Review Request 29756: Patch for KAKFA-1854

2015-01-12 Thread Jaikiran Pai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29756/ --- (Updated Jan. 13, 2015, 5:42 a.m.) Review request for kafka. Summary (updated

Re: Review Request 29831: Patch for KAFKA-1476

2015-01-12 Thread Neha Narkhede
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/#review67789 --- To make review easier, could you add the output of the command for a

[jira] [Updated] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1819: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patches, [~gwens

Re: Review Request 29756: Patch for KAFKA-1854

2015-01-12 Thread Neha Narkhede
> On Jan. 9, 2015, 6:48 p.m., Neha Narkhede wrote: > > kafka-patch-review.py, line 20 > > > > > > I got the following error on this patch > > > > nnarkhed-mn1:kafka nnarkhed$ python kafka-patch-review.py -b tr

Re: Review Request 29756: Patch for KAFKA-1854

2015-01-12 Thread Neha Narkhede
> On Jan. 9, 2015, 6:48 p.m., Neha Narkhede wrote: > > kafka-patch-review.py, line 20 > > > > > > I got the following error on this patch > > > > nnarkhed-mn1:kafka nnarkhed$ python kafka-patch-review.py -b tr

Review Request 29840: Patch for KAFKA-1818

2015-01-12 Thread Eric Olander
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29840/ --- Review request for kafka. Bugs: KAFKA-1818 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1850) Failed reassignment leads to additional replica

2015-01-12 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274657#comment-14274657 ] Joe Stein commented on KAFKA-1850: -- The reassignment isn't going to be able to finish unt

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Neha Narkhede
> On Jan. 12, 2015, 10:53 p.m., Joel Koshy wrote: > > core/src/test/scala/unit/kafka/log/LogCleanerIntegrationTest.scala, line 68 > > > > > > Actually, now that you have the check above in DeleteTopicTest, do you > > t

[jira] [Commented] (KAFKA-1857) Kafka Broker ids are removed ( with zookeeper , Storm )

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274620#comment-14274620 ] Neha Narkhede commented on KAFKA-1857: -- You might find this useful - http://kafka.apa

Re: Compatibility + Unknown APIs

2015-01-12 Thread Jay Kreps
I totally agree but I still think we shouldn't do it. :-) That change would cause the reimplementation of ALL existing Kafka clients. (You can't chose not to implement a new protocol version or else we are committing to keeping the old version supported both ways on the server forever). The probl

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
> On Jan. 12, 2015, 10:08 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataCheckpoint.scala, line 55 > > > > > > Minor nit which I can address during checkin: According to our coding > > c

[jira] [Commented] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274614#comment-14274614 ] Sriharsha Chintalapani commented on KAFKA-1070: --- Updated reviewboard https:/

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23702/ --- (Updated Jan. 13, 2015, 2:30 a.m.) Review request for kafka. Bugs: KAFKA-1070

[jira] [Updated] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1070: -- Attachment: KAFKA-1070_2015-01-12_18:30:17.patch > Auto-assign node id > ---

Re: Kafka Cluster Monitoring and Documentation of Internals (JMX Metrics) of Rejected Events

2015-01-12 Thread Otis Gospodnetic
Hi, I think you could just email user@? There was no attached image. I think Jun committed something about this: https://issues.apache.org/jira/browse/KAFKA-1481?focusedCommentId=14272057&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14272057 Otis -- Monitoring * A

Re: [DISCUSS] Compatability and KIPs

2015-01-12 Thread Jay Kreps
Hey Joe, Yeah I think a lot of those items are limitations in that document that we should definitely fix. The first issue you point out is a serious one: We give the total list of errors but don't list which errors can result from which APIs. This is a big issue because actually no one knows and

[jira] [Commented] (KAFKA-1857) Kafka Broker ids are removed ( with zookeeper , Storm )

2015-01-12 Thread Yoonhyeok Kim (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274595#comment-14274595 ] Yoonhyeok Kim commented on KAFKA-1857: -- [~nehanarkhede] Okay, Thank you. But I don't

Re: Compatibility + Unknown APIs

2015-01-12 Thread Joe Stein
There are benefits of moving the error code to the response header. 1) I think it is the right thing to-do from an implementation perspective. It makes the most sense. You send a request and you get back a response. The response tells you something is wrong in the header. 2) With such a large cha

Re: [DISCUSS] Compatability and KIPs

2015-01-12 Thread Joe Stein
Having an index for every protocol/API change (like https://www.python.org/dev/peps/pep-0257/ ) will be much better than the flat wire protocol doc we have now. It is impossible ( without jumping into code ) right now to know if an error is supported in one version of Kafka vs another or different

Re: Review Request 29692: Patch for kafka-1841

2015-01-12 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29692/#review67775 --- Ship it! Reversed an earlier comment, but up to you to pick which o

Re: Compatibility + Unknown APIs

2015-01-12 Thread Jay Kreps
Yeah, adding it to the metadata request probably makes sense. What you describe of making it a per-broker field is technically correct, since each broker could be on a different software version. But I wonder if it might not be more usable to just give back a single list of api versions. This will

Re: [DISCUSS] Compatability and KIPs

2015-01-12 Thread Jay Kreps
Yeah I think this makes sense. Some of the crazy nesting will get better when we move to the new protocol definition I think, but we will always need some kind of if statement that branches for the different behavior and this makes testing difficult. Probably the best thing to do would be to annou

Re: Compatibility + Unknown APIs

2015-01-12 Thread Dana Powers
Perhaps a bit hacky, but you could also reserve a specific correlationId (maybe -1) to represent errors and send back to the client an UnknownAPIResponse like: Response => -1 UnknownAPIResponse UnknownAPIResponse => originalCorrelationId errorCode The benefit here would be that it does not break

Re: [DISCUSS] Compatability and KIPs

2015-01-12 Thread Guozhang Wang
+1 on version evolving with any protocol / data format / functionality changes, and I am wondering if we have a standard process of deprecating old versions? Today with just a couple of versions for the protocol (e.g. offset commit) the code on the server side is already pretty nested and complicat

[jira] [Updated] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1819: Attachment: KAFKA-1819_2015-01-12_17:01:53.patch > Cleaner gets confused about deleted and re-create

[jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274530#comment-14274530 ] Gwen Shapira commented on KAFKA-1819: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29210/ --- (Updated Jan. 13, 2015, 1:01 a.m.) Review request for kafka. Bugs: KAFKA-1819

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Gwen Shapira
> On Jan. 12, 2015, 10:53 p.m., Joel Koshy wrote: > > core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala, line 251 > > > > > > Rather than an arbitrary sleep can we just use waitUntilTrue and the > > condition

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Gwen Shapira
> On Jan. 12, 2015, 10:53 p.m., Joel Koshy wrote: > > core/src/test/scala/unit/kafka/log/LogCleanerIntegrationTest.scala, line 68 > > > > > > Actually, now that you have the check above in DeleteTopicTest, do you > > t

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: KAFKA-1476_2015-01-12_16:31:20.patch > Get a list of consumer groups > -

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274481#comment-14274481 ] Onur Karaman commented on KAFKA-1476: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 29831: Patch for KAFKA-1476

2015-01-12 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/ --- (Updated Jan. 13, 2015, 12:31 a.m.) Review request for kafka. Bugs: KAFKA-147

Re: Review Request 29831: Patch for KAFKA-1476

2015-01-12 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/ --- (Updated Jan. 13, 2015, 12:22 a.m.) Review request for kafka. Bugs: KAFKA-147

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274473#comment-14274473 ] Onur Karaman commented on KAFKA-1476: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: KAFKA-1476_2015-01-12_16:22:26.patch > Get a list of consumer groups > -

Build failed in Jenkins: Kafka-trunk #364

2015-01-12 Thread Apache Jenkins Server
See Changes: [neha.narkhede] KAFKA-1070 Auto assign broker id; reviewed by Neha Narkhede [neha.narkhede] KAFKA-1836 metadata.fetch.timeout.ms set to zero blocks forever; reviewed by Neha Narkhede and Ewen Cheslack-Postava

[jira] [Updated] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: KAFKA-1476.patch > Get a list of consumer groups > - > >

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274460#comment-14274460 ] Onur Karaman commented on KAFKA-1476: - Created reviewboard https://reviews.apache.org/

Review Request 29831: Patch for KAFKA-1476

2015-01-12 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29831/ --- Review request for kafka. Bugs: KAFKA-1476 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274455#comment-14274455 ] Neha Narkhede commented on KAFKA-1476: -- [~balaji.sesha...@dish.com], [~onurkaraman] T

[jira] [Commented] (KAFKA-1476) Get a list of consumer groups

2015-01-12 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274451#comment-14274451 ] Onur Karaman commented on KAFKA-1476: - I have some stuff for sub-task KAFKA-1826 that

[jira] [Updated] (KAFKA-1836) metadata.fetch.timeout.ms set to zero blocks forever

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1836: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patches! Pushed

Re: Compatibility + Unknown APIs

2015-01-12 Thread Jay Kreps
Yeah I totally agree--using the closed socket to indicate "not supported" does work since any network error could lead to that. Arguably we should have a request level error. We discussed this at the time we were defining the protocols for 0.8 and the conclusion was not to do that. The reasoning w

[jira] [Updated] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1070: - Resolution: Fixed Status: Resolved (was: Patch Available) [~sriharsha] Thanks for your ef

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29210/#review67717 --- Minor comments core/src/test/scala/unit/kafka/admin/DeleteTopicTes

[jira] [Updated] (KAFKA-1835) Kafka new producer needs options to make blocking behavior explicit

2015-01-12 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1835: - Fix Version/s: 0.8.2 > Kafka new producer needs options to make blocking behavior explicit > -

Re: Review Request 29692: Patch for kafka-1841

2015-01-12 Thread Jun Rao
> On Jan. 12, 2015, 7:47 p.m., Joel Koshy wrote: > > core/src/main/scala/kafka/server/KafkaApis.scala, line 88 > > > > > > We should probably also change OffsetCommitRequest.responseFor . The > > issue is that if you g

[jira] [Updated] (KAFKA-1835) Kafka new producer needs options to make blocking behavior explicit

2015-01-12 Thread Paul Pearcy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pearcy updated KAFKA-1835: --- Affects Version/s: 0.9.0 0.8.3 Status: Patch Available (was: Ope

[jira] [Updated] (KAFKA-1835) Kafka new producer needs options to make blocking behavior explicit

2015-01-12 Thread Paul Pearcy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pearcy updated KAFKA-1835: --- Attachment: KAFKA-1835-New-producer--blocking_v0.patch > Kafka new producer needs options to make bloc

[jira] [Commented] (KAFKA-1841) OffsetCommitRequest API - timestamp field is not versioned

2015-01-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274301#comment-14274301 ] Jun Rao commented on KAFKA-1841: Updated reviewboard https://reviews.apache.org/r/29692/di

[jira] [Updated] (KAFKA-1841) OffsetCommitRequest API - timestamp field is not versioned

2015-01-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1841: --- Attachment: kafka-1841_2015-01-12_14:30:24.patch > OffsetCommitRequest API - timestamp field is not versioned

Re: Review Request 29692: Patch for kafka-1841

2015-01-12 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29692/ --- (Updated Jan. 12, 2015, 10:30 p.m.) Review request for kafka. Bugs: kafka-184

Re: Compatibility + Unknown APIs

2015-01-12 Thread Gwen Shapira
> I think #1 may be tricky in practice. The response format is always > dictated by the request format so how do I know if the bytes I got back are > a valid response to the given request or are the UnknownRequestResponse? On the other hand, from the client developer perspective, having to figure

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Neha Narkhede
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23702/#review67715 --- Ship it! Ship It! core/src/main/scala/kafka/server/BrokerMetadata

Kafka Cluster Monitoring and Documentation of Internals (JMX Metrics) of Rejected Events

2015-01-12 Thread Bhavesh Mistry
Hi Kafka Team, I am trying to find out Kafka Internal and how a message can be corrupted or lost at brokers side. I have refer to following documentations for monitoring: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Internals http://kafka.apache.org/documentation.html#monitoring I am

Re: Compatibility + Unknown APIs

2015-01-12 Thread Jay Kreps
Yeah I agree there isn't a good way to auto-detect the presence of a given API. I think #1 may be tricky in practice. The response format is always dictated by the request format so how do I know if the bytes I got back are a valid response to the given request or are the UnknownRequestResponse?

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Neha Narkhede
> On Jan. 9, 2015, 11:39 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataFileHandler.scala, line 48 > > > > > > Does java serialization for the Properties object allow you to read > > ju

Re: Compatibility + Unknown APIs

2015-01-12 Thread Gwen Shapira
We ran into similar difficulties, both when trying to get Kafka to use new APIs when available and when testing the wire protocol. +1 for all three suggestions. #1 sounds like the bare minimum, but I'm not sure how much it will complicate the clients (now we expect either a response or an Unknown

Compatibility + Unknown APIs

2015-01-12 Thread Dana Powers
Hi all -- continuing on the compatibility discussion: I've found that it is very difficult to identify when a server does not recognize an api (I'm using kafka-python to submit wire-protocol requests). For example, when I send a ConsumerMetadataRequest to an 0.8.1.1 server, I get a closed socket

Re: Review Request 29692: Patch for kafka-1841

2015-01-12 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29692/#review67667 --- Thanks for patching this. Looks good overall. clients/src/main/jav

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29210/ --- (Updated Jan. 12, 2015, 7:17 p.m.) Review request for kafka. Bugs: KAFKA-1819

[jira] [Updated] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1819: Attachment: KAFKA-1819_2015-01-12_11:17:53.patch > Cleaner gets confused about deleted and re-create

[jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273999#comment-14273999 ] Gwen Shapira commented on KAFKA-1819: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
> On Jan. 9, 2015, 11:39 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataFileHandler.scala, line 48 > > > > > > Does java serialization for the Properties object allow you to read > > ju

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Neha Narkhede
> On Jan. 9, 2015, 11:39 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataFileHandler.scala, line 48 > > > > > > Does java serialization for the Properties object allow you to read > > ju

[jira] [Commented] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273946#comment-14273946 ] Sriharsha Chintalapani commented on KAFKA-1070: --- [~nehanarkhede] updated pat

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
> On Jan. 9, 2015, 11:39 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataFileHandler.scala, line 48 > > > > > > Does java serialization for the Properties object allow you to read > > ju

[jira] [Closed] (KAFKA-1857) Kafka Broker ids are removed ( with zookeeper , Storm )

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede closed KAFKA-1857. Assignee: (was: Neha Narkhede) > Kafka Broker ids are removed ( with zookeeper , Storm ) > -

[jira] [Resolved] (KAFKA-1857) Kafka Broker ids are removed ( with zookeeper , Storm )

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-1857. -- Resolution: Invalid Please send such questions on the mailing list. It has higher visibility and

[jira] [Commented] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273928#comment-14273928 ] Sriharsha Chintalapani commented on KAFKA-1070: --- Updated reviewboard https:/

[jira] [Updated] (KAFKA-1070) Auto-assign node id

2015-01-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1070: -- Attachment: KAFKA-1070_2015-01-12_10:46:54.patch > Auto-assign node id > ---

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
> On Jan. 9, 2015, 11:39 p.m., Neha Narkhede wrote: > > core/src/main/scala/kafka/server/BrokerMetadataFileHandler.scala, line 30 > > > > > > I think the way you modelled this checkpoint file, which makes it > > differ

Re: Review Request 23702: Patch for KAFKA-1070

2015-01-12 Thread Sriharsha Chintalapani
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23702/ --- (Updated Jan. 12, 2015, 6:46 p.m.) Review request for kafka. Bugs: KAFKA-1070

[jira] [Commented] (KAFKA-1856) Add PreCommit Patch Testing

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273923#comment-14273923 ] Neha Narkhede commented on KAFKA-1856: -- Thanks for filing this [~singhashish]! Look f

[jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273921#comment-14273921 ] Gwen Shapira commented on KAFKA-1819: - Ouch. Yes. Good catch! Let me check how this

[jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273918#comment-14273918 ] Neha Narkhede commented on KAFKA-1819: -- [~gwenshap] Thanks for looking into the possi

[jira] [Created] (KAFKA-1858) Make ServerShutdownTest a bit less flaky

2015-01-12 Thread Gwen Shapira (JIRA)
Gwen Shapira created KAFKA-1858: --- Summary: Make ServerShutdownTest a bit less flaky Key: KAFKA-1858 URL: https://issues.apache.org/jira/browse/KAFKA-1858 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1819: Attachment: KAFKA-1819_2015-01-12_10:34:07.patch > Cleaner gets confused about deleted and re-create

[jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics

2015-01-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273906#comment-14273906 ] Gwen Shapira commented on KAFKA-1819: - Updated reviewboard https://reviews.apache.org/

Re: Review Request 29210: Patch for KAFKA-1819

2015-01-12 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29210/ --- (Updated Jan. 12, 2015, 6:34 p.m.) Review request for kafka. Bugs: KAFKA-1819

[jira] [Commented] (KAFKA-725) Broker Exception: Attempt to read with a maximum offset less than start offset

2015-01-12 Thread Diwakar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273861#comment-14273861 ] Diwakar commented on KAFKA-725: --- Neha, we have 6 brokers and 131 partitions per topic(replic

Re: [DISCUSS] 0.8.2-beta2 release

2015-01-12 Thread Neha Narkhede
Joe, Thanks for starting the discussion. It may be ok to do an RC and probably two weeks later, cut the final. After releasing the RC, we can help spread the word and have people try out the new API, which is probably our main concern. -Neha On Mon, Jan 12, 2015 at 8:42 AM, Jun Rao wrote: > Hi

Re: New consumer client

2015-01-12 Thread Jay Kreps
Hey Bhavesh, This seems like a serious issue and not one anyone else has reported. I don't know what you mean by corrupt message, are you saying the CRC check fails? If so, that check is done both by the broker (prior to appending to the log) and the consumer so that implies either a bug in the br

Re: [DISCUSS] Compatability and KIPs

2015-01-12 Thread Jay Kreps
Hey Jun, Good points. I totally agree that the versioning needs to cover both format and behavior if the behavior change is incompatible. I kind of agree about the stable/unstable stuff. What I think this means is not that we would ever evolve the protocol without changing the version, but rathe

Re: Review Request 29301: Patch for KAFKA-1694

2015-01-12 Thread Andrii Biletskyi
> On Jan. 6, 2015, 8:19 a.m., Joe Stein wrote: > > build.gradle, line 211 > > > > > > If we can do this without an upgrade that would be great if we are in > > fact just requiring 1 function. This feature allows us t

[jira] [Updated] (KAFKA-1694) kafka command line and centralized operations

2015-01-12 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1694: Attachment: KAFKA-1694_2015-01-12_18:54:48.patch > kafka command line and centralized operat

[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations

2015-01-12 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273773#comment-14273773 ] Andrii Biletskyi commented on KAFKA-1694: - Updated reviewboard https://reviews.apa

Re: Review Request 29301: Patch for KAFKA-1694

2015-01-12 Thread Andrii Biletskyi
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29301/ --- (Updated Jan. 12, 2015, 4:55 p.m.) Review request for kafka. Bugs: KAFKA-1694

  1   2   >