[jira] [Updated] (KAFKA-1237) Add mirror maker using 08 consumer and 09 producer

[ https://issues.apache.org/jira/browse/KAFKA-1237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1237: - Fix Version/s: 0.9.0 > Add mirror maker using 08 consumer and 09 producer > -

[jira] [Updated] (KAFKA-1237) Add mirror maker using 08 consumer and 09 producer

[ https://issues.apache.org/jira/browse/KAFKA-1237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1237: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the reviews, checke

[jira] [Closed] (KAFKA-1237) Add mirror maker using 08 consumer and 09 producer

[ https://issues.apache.org/jira/browse/KAFKA-1237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede closed KAFKA-1237. > Add mirror maker using 08 consumer and 09 producer >

[jira] [Updated] (KAFKA-1237) Add mirror maker using 08 consumer and 09 producer

[ https://issues.apache.org/jira/browse/KAFKA-1237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1237: - Status: Patch Available (was: Open) > Add mirror maker using 08 consumer and 09 producer > -

Re: protocol definition

Just some minor comments. 1. Struct is not a type. So, it probably should be in the protocol package, instead of protocol.types. 2. Is the comment in Struct.instance() outdated? It seems this is only used for creating an instance of a Field of type struct. 3. When creating a Field in a schema, i

Re: Review Request 17880: Patch for KAFKA-1237

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17880/#review34166 --- Ship it! Ship It! - Jay Kreps On Feb. 9, 2014, 8:02 p.m., Neha N

[jira] [Assigned] (KAFKA-1255) Offset in RecordMetadata is Incorrect with New Producer Ack = -1

[ https://issues.apache.org/jira/browse/KAFKA-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-1255: Assignee: Jay Kreps > Offset in RecordMetadata is Incorrect with New Producer Ack = -1 > --

[jira] [Resolved] (KAFKA-1238) New producer hangs in a loop detecting metadata for auto created topics

[ https://issues.apache.org/jira/browse/KAFKA-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1238. -- Resolution: Fixed > New producer hangs in a loop detecting metadata for auto created topics > -

[jira] [Resolved] (KAFKA-1236) Change producer performance tool to optionally use the new producer

[ https://issues.apache.org/jira/browse/KAFKA-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-1236. -- Resolution: Fixed Fix Version/s: 0.9.0 > Change producer performance tool to optionally use the

Re: Review Request 17879: Patch for KAFKA-1245

> On Feb. 10, 2014, 10:19 p.m., Jun Rao wrote: > > gradle.properties, lines 21-24 > > > > > > Do you really intend to remove those lines? > > Joe Stein wrote: > Yes, that was intentional. > > Those value

[jira] [Issue Comment Deleted] (KAFKA-1232) make TopicCommand more consistent

[ https://issues.apache.org/jira/browse/KAFKA-1232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1232: - Comment: was deleted (was: Updated reviewboard https://reviews.apache.org/r/17918/ against branc

[jira] [Commented] (KAFKA-1233) Integration test for the new producer

[ https://issues.apache.org/jira/browse/KAFKA-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897432#comment-13897432 ] Guozhang Wang commented on KAFKA-1233: -- Updated reviewboard https://reviews.apache.or

Re: Review Request 17918: Patch for KAFKA-1232

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17918/ --- (Updated Feb. 11, 2014, 1:51 a.m.) Review request for kafka. Bugs: KAFKA-1233

Re: Review Request 17918: Patch for KAFKA-1232

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17918/ --- (Updated Feb. 11, 2014, 1:44 a.m.) Review request for kafka. Bugs: KAFKA-1232

[jira] [Commented] (KAFKA-1232) make TopicCommand more consistent

[ https://issues.apache.org/jira/browse/KAFKA-1232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897425#comment-13897425 ] Guozhang Wang commented on KAFKA-1232: -- Updated reviewboard https://reviews.apache.or

Re: Review Request 17921: Misc fixes for perf tools.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17921/#review34155 --- Ship it! - Neha Narkhede On Feb. 10, 2014, 9:49 p.m., Jay Kreps w

Re: Review Request 17921: Misc fixes for perf tools.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17921/#review34154 --- Ship it! - Neha Narkhede On Feb. 10, 2014, 9:49 p.m., Jay Kreps w

[jira] [Updated] (KAFKA-1232) make TopicCommand more consistent

[ https://issues.apache.org/jira/browse/KAFKA-1232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1232: - Attachment: KAFKA-1232_2014-02-10_17:42:18.patch > make TopicCommand more consistent > --

Re: Review Request 17918: Patch for KAFKA-1232

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17918/ --- (Updated Feb. 11, 2014, 1:42 a.m.) Review request for kafka. Summary (updated

[jira] [Commented] (KAFKA-1238) New producer hangs in a loop detecting metadata for auto created topics

[ https://issues.apache.org/jira/browse/KAFKA-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897407#comment-13897407 ] Neha Narkhede commented on KAFKA-1238: -- Got it. Thanks for the explanation. +1 > New

Re: Review Request 17918: Integration test for new producer: Part I

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17918/#review34145 --- core/src/test/scala/integration/kafka/api/ProducerSendTest.scala

Re: Review Request 17921: Misc fixes for perf tools.

> On Feb. 11, 2014, 12:49 a.m., Neha Narkhede wrote: > > 1. The JIRA attached to this rb is incorrect. Yeah I made another, but since you commented on this one, I'll review here. > On Feb. 11, 2014, 12:49 a.m., Neha Narkhede wrote: > > perf/src/main/scala/kafka/perf/ProducerPerformance.scala,

Re: Review Request 17921: Misc fixes for perf tools.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17921/#review34135 --- 1. The JIRA attached to this rb is incorrect. bin/kafka-run-class.

[jira] [Commented] (KAFKA-1238) New producer hangs in a loop detecting metadata for auto created topics

[ https://issues.apache.org/jira/browse/KAFKA-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897361#comment-13897361 ] Jay Kreps commented on KAFKA-1238: -- Yeah arguably the server should just return all the b

[jira] [Commented] (KAFKA-1238) New producer hangs in a loop detecting metadata for auto created topics

[ https://issues.apache.org/jira/browse/KAFKA-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897346#comment-13897346 ] Guozhang Wang commented on KAFKA-1238: -- Added a testcase in integration test for auto

Re: Config for new clients (and server)

Yeah I am aware of how zookeeper behaves, I think it is kind of gross. I think logging it at DEBUG gets you what you want--by default we don't pollute logs, but anyone who wants to log this can enable DEBUG logging on org.apache.kafka.clients.producer.ProducerConfig. If we want this on by default

[jira] [Commented] (KAFKA-1254) remove vestigial sbt

[ https://issues.apache.org/jira/browse/KAFKA-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897303#comment-13897303 ] Jun Rao commented on KAFKA-1254: +1. We can check in after all other sub-jiras in kafka-12

Re: Review Request 17880: Patch for KAFKA-1237

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17880/#review34133 --- Ship it! Minor comments only. build.gradle

[jira] [Commented] (KAFKA-1243) Gradle issues for release

[ https://issues.apache.org/jira/browse/KAFKA-1243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897288#comment-13897288 ] Jun Rao commented on KAFKA-1243: I made a pass of this jira and fixed the following. 1. ad

Re: New Consumer API discussion

WRT to hierarchical topics, I'm referring to KAFKA-1175. I would just like to think through the implications for the Consumer API if and when we do implement hierarchical topics. For example, in the proposal

Re: Review Request 17923: gradle issues for 0.8.1 release

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17923/#review34134 --- Ship it! Ship It! - Neha Narkhede On Feb. 10, 2014, 11:25 p.m.,

[jira] [Commented] (KAFKA-1243) Gradle issues for release

[ https://issues.apache.org/jira/browse/KAFKA-1243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897273#comment-13897273 ] Jun Rao commented on KAFKA-1243: Created reviewboard https://reviews.apache.org/r/17923/

[jira] [Updated] (KAFKA-1243) Gradle issues for release

[ https://issues.apache.org/jira/browse/KAFKA-1243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1243: --- Attachment: KAFKA-1243.patch > Gradle issues for release > - > > Key:

Review Request 17923: gradle issues for 0.8.1 release

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17923/ --- Review request for kafka. Bugs: KAFKA-1243 https://issues.apache.org/jira/b

Re: Config for new clients (and server)

+1 Jun. On Mon, Feb 10, 2014 at 2:17 PM, Sriram Subramanian < srsubraman...@linkedin.com> wrote: > +1 on Jun's suggestion. > > On 2/10/14 2:01 PM, "Jun Rao" wrote: > > >I actually prefer to see those at INFO level. The reason is that the > >config > >system in an application can be complex. Som

Re: New Consumer API discussion

Thanks for the feedback. Mattijs - - Constructors link to http://kafka.apache.org/documentation.html#consumerconfigs for valid configurations, which lists zookeeper.connect rather than metadata.broker.list, the value for BROKER_LIST_CONFIG in ConsumerConfig. Fixed it to just point to ConsumerConf

Re: [jira] [Commented] (KAFKA-123) Rubygem gemspec doesn't build correctly

Please ignore this, wrong ticket for the review. On Mon, Feb 10, 2014 at 1:35 PM, Guozhang Wang (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/KAFKA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897051#comment-13897051] > > Guozhang

[jira] [Commented] (KAFKA-1233) Integration test for the new producer

[ https://issues.apache.org/jira/browse/KAFKA-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897172#comment-13897172 ] Guozhang Wang commented on KAFKA-1233: -- Created reviewboard https://reviews.apache.or

[jira] [Issue Comment Deleted] (KAFKA-123) Rubygem gemspec doesn't build correctly

[ https://issues.apache.org/jira/browse/KAFKA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-123: Comment: was deleted (was: Created reviewboard https://reviews.apache.org/r/17918/ against branch o

Re: Review Request 17918: Integration test for new producer: Part I

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17918/ --- (Updated Feb. 10, 2014, 11:01 p.m.) Review request for kafka. Bugs: KAFKA-123

Re: Review Request 16718: Patch for KAFKA-1180

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16718/#review34128 --- Ship it! core/src/test/scala/unit/kafka/consumer/TopicFilterTest.s

Re: Review Request 17879: Patch for KAFKA-1245

> On Feb. 10, 2014, 10:19 p.m., Jun Rao wrote: > > gradle.properties, lines 21-24 > > > > > > Do you really intend to remove those lines? Yes, that was intentional. Those values are better (more safely) kept globall

Re: Review Request 17879: Patch for KAFKA-1245

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17879/#review34115 --- gradle.properties

Re: Config for new clients (and server)

+1 on Jun's suggestion. On 2/10/14 2:01 PM, "Jun Rao" wrote: >I actually prefer to see those at INFO level. The reason is that the >config >system in an application can be complex. Some configs can be overridden in >different layers and it may not be easy to determine what the final >binding >va

Re: New Consumer API discussion

Couple of very quick thoughts. 1. +1 about renaming commit(...) and commitAsync(...) 2. I'd also like to extend the above for the poll() method as well. poll() and pollWithTimeout(long, TimeUnit)? 3. Have you guys given any thought around how this API would be used with hierarchical topics? 4. Wo

Re: Config for new clients (and server)

I actually prefer to see those at INFO level. The reason is that the config system in an application can be complex. Some configs can be overridden in different layers and it may not be easy to determine what the final binding value is. The logging in Kafka will serve as the source of truth. For r

Re: Review Request 17921: Misc fixes for perf tools.

Ignore this, I attached the review to the wrong ticket. Created a new review request to fix it. On Mon, Feb 10, 2014 at 1:49 PM, Jay Kreps wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apach

[jira] [Commented] (KAFKA-1255) Offset in RecordMetadata is Incorrect with New Producer Ack = -1

[ https://issues.apache.org/jira/browse/KAFKA-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897080#comment-13897080 ] Jay Kreps commented on KAFKA-1255: -- This is surprising as the offset comes through the sa

[jira] [Commented] (KAFKA-1236) Change producer performance tool to optionally use the new producer

[ https://issues.apache.org/jira/browse/KAFKA-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897070#comment-13897070 ] Jay Kreps commented on KAFKA-1236: -- Created reviewboard https://reviews.apache.org/r/1792

[jira] [Issue Comment Deleted] (KAFKA-1255) Offset in RecordMetadata is Incorrect with New Producer Ack = -1

[ https://issues.apache.org/jira/browse/KAFKA-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1255: - Comment: was deleted (was: Created reviewboard https://reviews.apache.org/r/17921/ against branch trunk)

[jira] [Updated] (KAFKA-1255) Offset in RecordMetadata is Incorrect with New Producer Ack = -1

[ https://issues.apache.org/jira/browse/KAFKA-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1255: - Attachment: (was: KAFKA-1255.patch) > Offset in RecordMetadata is Incorrect with New Producer Ack = -

[jira] [Updated] (KAFKA-1236) Change producer performance tool to optionally use the new producer

[ https://issues.apache.org/jira/browse/KAFKA-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1236: - Attachment: KAFKA-1236.patch > Change producer performance tool to optionally use the new producer >

Review Request 17922: Misc fixes for perf tools.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17922/ --- Review request for kafka. Bugs: KAFKA-1236 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-1255) Offset in RecordMetadata is Incorrect with New Producer Ack = -1

[ https://issues.apache.org/jira/browse/KAFKA-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1255: - Attachment: KAFKA-1255.patch > Offset in RecordMetadata is Incorrect with New Producer Ack = -1 > ---

[jira] [Commented] (KAFKA-1255) Offset in RecordMetadata is Incorrect with New Producer Ack = -1

[ https://issues.apache.org/jira/browse/KAFKA-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897064#comment-13897064 ] Jay Kreps commented on KAFKA-1255: -- Created reviewboard https://reviews.apache.org/r/1792

Review Request 17921: Misc fixes for perf tools.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17921/ --- Review request for kafka. Bugs: KAFKA-1255 https://issues.apache.org/jira/b

Re: Review Request 17918: Integration test for new producer: Part I

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17918/ --- (Updated Feb. 10, 2014, 9:40 p.m.) Review request for kafka. Bugs: KAFKA-123

[jira] [Created] (KAFKA-1255) Offset in RecordMetadata is Incorrect with New Producer Ack = -1

Guozhang Wang created KAFKA-1255: Summary: Offset in RecordMetadata is Incorrect with New Producer Ack = -1 Key: KAFKA-1255 URL: https://issues.apache.org/jira/browse/KAFKA-1255 Project: Kafka

[jira] [Updated] (KAFKA-1255) Offset in RecordMetadata is Incorrect with New Producer Ack = -1

[ https://issues.apache.org/jira/browse/KAFKA-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1255: - Attachment: sendwithAckMinusOne sendwithAckOne > Offset in RecordMetadata is Inco

[jira] [Updated] (KAFKA-1236) Change producer performance tool to optionally use the new producer

[ https://issues.apache.org/jira/browse/KAFKA-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1236: - Attachment: KAFKA-1236-v1.patch Attached is a patch that makes the performance test use either the old or

[jira] [Updated] (KAFKA-123) Rubygem gemspec doesn't build correctly

[ https://issues.apache.org/jira/browse/KAFKA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-123: Attachment: KAFKA-123.patch > Rubygem gemspec doesn't build correctly >

Review Request 17918: Integration test for new producer: Part I

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17918/ --- Review request for kafka. Bugs: KAFKA-123 https://issues.apache.org/jira/br

[jira] [Commented] (KAFKA-123) Rubygem gemspec doesn't build correctly

[ https://issues.apache.org/jira/browse/KAFKA-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897051#comment-13897051 ] Guozhang Wang commented on KAFKA-123: - Created reviewboard https://reviews.apache.org/r

Re: New Consumer API discussion

A few items: 1. ConsumerRebalanceCallback a. onPartitionsRevoked would be a better name. b. We should discuss the possibility of splitting this into two interfaces. The motivation would be that in Java 8 single method interfaces can directly take methods which might be more intuitive. c. I

[jira] Subscription: outstanding kafka patches

Issue Subscription Filter: outstanding kafka patches (89 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-1254 remove vestigial sbt https://issues.apache.org/jira/browse/KAFKA-1254 KAFKA-1245 the jar files and pom are not being si

New Consumer API discussion

As mentioned in previous emails, we are also working on a re-implementation of the consumer. I would like to use this email thread to discuss the details of the public API. I would also like us to be picky about this public api now so it is as good as possible and we don't need to break it in the f

[jira] [Commented] (KAFKA-1254) remove vestigial sbt

[ https://issues.apache.org/jira/browse/KAFKA-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13896824#comment-13896824 ] Joel Koshy commented on KAFKA-1254: --- Jun can confirm, but I believe the remaining issues

Re: Config for new clients (and server)

Hey Jun, I think that is reasonable but would object to having it be debug logging? I think logging out a bunch of noise during normal operation in a client library is pretty ugly. Also, is there value in exposing the final configs programmatically? -Jay On Sun, Feb 9, 2014 at 9:23 PM, Jun Rao

Re: gradle and sbt in 0.8.1

Cool, just checking :-) -Jay On Sun, Feb 9, 2014 at 9:36 PM, Jun Rao wrote: > We won't release both sbt and gradle. This is mostly to unblock the > development for the new producer client in trunk. > > For 0.8.1, the plan is to try to get gradle fully working and then remove > sbt. > > Thanks,

Re: The consumer performance test seems to be totally broken

Hmm, I could be wrong... Did you try this on an otherwise empty topic? Try this: 1. Produce 100GB of messages 2. Run consumer performance test with --messages 1 3. Go get some coffee 4. Observe performance tests results with a time taken to consume 100GB of messages but only recording a single mess

[jira] [Commented] (KAFKA-1238) New producer hangs in a loop detecting metadata for auto created topics

[ https://issues.apache.org/jira/browse/KAFKA-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13896775#comment-13896775 ] Jay Kreps commented on KAFKA-1238: -- The way metadata works is the following: 1. When some

Re: Querry regarding setting up Kafka server on offline linux machine.

This seems like a maven issue. Not sure the reason though. You could also just try the binary release for 0.8.0. Thanks, Jun On Mon, Feb 10, 2014 at 1:55 AM, Saurabh Gupta A < saurabh.a.gu...@ericsson.com> wrote: > Hello, > I am new to Kafka, facing below problem while setting up Kafka. > I do

Querry regarding setting up Kafka server on offline linux machine.

Hello, I am new to Kafka, facing below problem while setting up Kafka. I downloaded "kafka-0.8.0-src.gz". Then I followed below steps: > tar xzf kafka-.tgz > cd kafka- > ./sbt update Below is the error I am facing: Please note that I am using linux machine which is offline and I have to install