Re: Review Request 34492: Patch for KAFKA-2210

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/#review87532 --- core/src/main/scala/kafka/server/KafkaConfig.scala

offset storage as kafka with zookeeper 3.4.6

I am trying to migrate the offset storage to kafka (3 brokers of version 0.8.2.1) using the consumer property offsets.storage=kafka. I noticed that a new topic, __consumer_offsets got created. But nothing is being written to this topic, while the consumer offsets continue to reside on zookeeper.

[jira] [Created] (KAFKA-2263) Update "Is it possible to delete a topic" wiki FAQ answer

Stevo Slavic created KAFKA-2263: --- Summary: Update "Is it possible to delete a topic" wiki FAQ answer Key: KAFKA-2263 URL: https://issues.apache.org/jira/browse/KAFKA-2263 Project: Kafka Issue T

Re: offset storage as kafka with zookeeper 3.4.6

> Is it mandatory to use the zookeeper that comes with kafka for offset > storage to be migrated to kafka? If you want to "move" offsets from zookeeper to Kafka then yes you need to have a phase where all consumers in your group set dual commit to true. If you are starting a fresh consumer group th

[jira] [Created] (KAFKA-2264) SESSION_TIMEOUT_MS_CONFIG in ConsumerConfig should be int

Jun Rao created KAFKA-2264: -- Summary: SESSION_TIMEOUT_MS_CONFIG in ConsumerConfig should be int Key: KAFKA-2264 URL: https://issues.apache.org/jira/browse/KAFKA-2264 Project: Kafka Issue Type: Bug

Re: offset storage as kafka with zookeeper 3.4.6

>If you want to "move" offsets from zookeeper to Kafka then yes you >need to have a phase where all consumers in your group set dual commit >to true. If you are starting a fresh consumer group then you can >turn off dual-commit. I followed these steps to move the offsets from zookeeper to kafka: 1.

Re: Review Request 35201: Fix KAFKA-2253

> On June 11, 2015, 1:07 a.m., Jun Rao wrote: > > core/src/main/scala/kafka/server/DelayedOperation.scala, lines 264-266 > > > > > > Not sure if we need this check. Since all writes to watchersForKey are > > sync-ed,

Re: Review Request 35201: Fix KAFKA-2253

> On June 11, 2015, 1:07 a.m., Jun Rao wrote: > > core/src/main/scala/kafka/server/DelayedOperation.scala, lines 264-266 > > > > > > Not sure if we need this check. Since all writes to watchersForKey are > > sync-ed,

RE: [DISCUSS] KIP-25 System test improvements

Hi Geoffrey, Thanks for the writeup. Couple of questions: - Is it possible to configure suites using ducktape? For example: assume all the tests in system_tests have been migrated to ducktape. Can I run a subset of all tests grouped by functional areas i.e. replication, broker failure etc. - Du

[jira] [Updated] (KAFKA-2136) Client side protocol changes to return quota delays

[ https://issues.apache.org/jira/browse/KAFKA-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2136: --- Labels: quotas (was: ) > Client side protocol changes to return quota delays > --

[jira] [Updated] (KAFKA-2171) System Test for Quotas

[ https://issues.apache.org/jira/browse/KAFKA-2171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2171: --- Labels: quotas (was: ) > System Test for Quotas > -- > >

[jira] [Updated] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2084: --- Labels: quotas (was: ) > byte rate metrics per client ID (producer and consumer) > --

[jira] [Updated] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)

[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2241: --- Labels: quotas (was: ) > AbstractFetcherThread.shutdown() should not block on > ReadableByte

[jira] [Updated] (KAFKA-2191) Measured rate should not be infinite

[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2191: Labels: quotas (was: ) > Measured rate should not be infinite > > >

[jira] [Updated] (KAFKA-2205) Generalize TopicConfigManager to handle multiple entity configs

[ https://issues.apache.org/jira/browse/KAFKA-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2205: --- Labels: quotas (was: ) > Generalize TopicConfigManager to handle multiple entity configs > --

RE: [DISCUSS] KIP-4 - Command line and centralized administrative operations (Thread 2)

I've made two changes to the document: - Removed the TMR evolution piece since we agreed to retain this. - Added two new API's to the admin client spec. (Alter and Describe config). Please review. Aditya From: Ashish Singh [asi...@cloudera.com] Sent: Frid

Re: Review Request 35201: Fix KAFKA-2253

> On June 11, 2015, 1:07 a.m., Jun Rao wrote: > > core/src/main/scala/kafka/server/DelayedOperation.scala, lines 264-266 > > > > > > Not sure if we need this check. Since all writes to watchersForKey are > > sync-ed,

Re: Review Request 35231: Fix KAFKA-1740

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35231/#review87583 --- core/src/main/scala/kafka/coordinator/ConsumerCoordinator.scala

Re: [DISCUSS] KIP-25 System test improvements

Hi Aditya, (1) There are a currently a few different ways to target a specific test or subset of tests. If for example tests were organized like the current system tests, where suites are grouped by directory, you could run cd ducktape replication_testsuite/ You can also target tests in a part

RE: [DISCUSS] KIP-4 - Command line and centralized administrative operations (Thread 2)

Andrii, Do we need a new voting thread for this KIP? The last round of votes had 3 binding +1's but there's been a fair amount of discussion since then. Aditya From: Aditya Auradkar Sent: Thursday, June 11, 2015 10:32 AM To: dev@kafka.apache.org Subject:

Re: [VOTE] KIP-25 System test improvements

+1 -Ewen On Thu, Jun 11, 2015 at 1:47 AM, Joe Stein wrote: > +1 > > ~ Joestein > On Jun 10, 2015 10:21 PM, "Neha Narkhede" wrote: > > > +1. Thanks Geoff! > > > > > > > > > > > > On Wed, Jun 10, 2015 at 6:20 PM -0700, "Gwen Shapira" < > > gshap...@cloudera.com> wrote: > > > > > > > > > > > > >

[jira] [Created] (KAFKA-2265) creating a topic with large number of partitions takes a long time

Jun Rao created KAFKA-2265: -- Summary: creating a topic with large number of partitions takes a long time Key: KAFKA-2265 URL: https://issues.apache.org/jira/browse/KAFKA-2265 Project: Kafka Issue T

[jira] [Commented] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper

[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582433#comment-14582433 ] Joel Koshy commented on KAFKA-1367: --- [~singhashish] - yes that is a good summary. Broker

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations (Thread 2)

Discussion aside, was there any significant material change besides the additions below? If so, then we can avoid the overhead of another vote unless someone wants to down-vote these changes. Joel On Thu, Jun 11, 2015 at 06:36:36PM +, Aditya Auradkar wrote: > Andrii, > > Do we need a new vot

Re: Review Request 34789: Patch for KAFKA-2168

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34789/ --- (Updated June 11, 2015, 9:10 p.m.) Review request for kafka. Bugs: KAFKA-2168

[jira] [Updated] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-2168: --- Attachment: KAFKA-2168_2015-06-11_14:09:59.patch > New consumer poll() can block other calls l

[jira] [Commented] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582524#comment-14582524 ] Jason Gustafson commented on KAFKA-2168: Updated reviewboard https://reviews.apach

[jira] [Commented] (KAFKA-2168) New consumer poll() can block other calls like position(), commit(), and close() indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582528#comment-14582528 ] Jay Kreps commented on KAFKA-2168: -- Hey [~guozhang], have you had a chance to look at thi

Re: [DISCUSS] KIP-23 - Add JSON/CSV output and looping options to ConsumerGroupCommand

Hi Guys, This has been lying around for quite some time. Should I start a voting thread on this? On Thu, May 7, 2015 at 12:20 PM, Ashish Singh wrote: > Had to change the title of the page and that surprisingly changed the link > as well. KIP-23 is now available at here >

Pending review requests

Hey Guys, I have a few JIRAs in "patch available" state for some time. I will really appreciate if someone can review them. 1. https://issues.apache.org/jira/browse/KAFKA-2132 2. https://issues.apache.org/jira/browse/KAFKA-2005 3. https://issues.apache.org/jira/browse/KAFKA-1722 4. https://issues

[jira] [Commented] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper

[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582532#comment-14582532 ] Ashish K Singh commented on KAFKA-1367: --- [~jjkoshy] thanks for confirming. I will ge

Re: Review Request 34789: Patch for KAFKA-2168

> On June 9, 2015, 7:58 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java, > > lines 797-798 > > > > > > Hmm, seekToBegining() is supposed to be a blocking call. Basic

Re: Review Request 34789: Patch for KAFKA-2168

> On June 3, 2015, 6:44 a.m., Ewen Cheslack-Postava wrote: > > clients/src/main/java/org/apache/kafka/clients/consumer/internals/BrokerResponse.java, > > line 15 > > > > > > The classes named XResponse may be a bit con

Re: [DISCUSS] KIP-23 - Add JSON/CSV output and looping options to ConsumerGroupCommand

Maybe bring it up at the next KIP call, to make sure everyone is aware? On Thu, Jun 11, 2015 at 2:17 PM, Ashish Singh wrote: > Hi Guys, > > This has been lying around for quite some time. Should I start a voting > thread on this? > > On Thu, May 7, 2015 at 12:20 PM, Ashish Singh wrote: > >> Had

[jira] [Updated] (KAFKA-2132) Move Log4J appender to a separate module

[ https://issues.apache.org/jira/browse/KAFKA-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2132: Summary: Move Log4J appender to a separate module (was: Move Log4J appender to clients module) > M

Re: [DISCUSS] KIP-23 - Add JSON/CSV output and looping options to ConsumerGroupCommand

Jun, Can we add this as part of next KIP's agenda? On Thu, Jun 11, 2015 at 3:00 PM, Gwen Shapira wrote: > Maybe bring it up at the next KIP call, to make sure everyone is aware? > > On Thu, Jun 11, 2015 at 2:17 PM, Ashish Singh wrote: > > Hi Guys, > > > > This has been lying around for quite s

Re: Review Request 33614: Patch for KAFKA-2132

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33614/#review87638 --- Hey Ashish, I've left a few minor comments. Thanks! log4j-appende

[jira] [Updated] (KAFKA-2005) Generate html report for system tests

[ https://issues.apache.org/jira/browse/KAFKA-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2005: --- Resolution: Fixed Fix Version/s: 0.8.3 Status: Resolved (was: Patch Available) Thanks for t

[jira] [Commented] (KAFKA-2005) Generate html report for system tests

[ https://issues.apache.org/jira/browse/KAFKA-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582646#comment-14582646 ] Ashish K Singh commented on KAFKA-2005: --- Thanks [~junrao] for reviewing and committi

[jira] [Updated] (KAFKA-2249) KafkaConfig does not preserve original Properties

[ https://issues.apache.org/jira/browse/KAFKA-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2249: --- Status: In Progress (was: Patch Available) > KafkaConfig does not preserve original Properties >

Re: Review Request 35347: Patch for KAFKA-2249

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35347/#review87643 --- Thanks a lot for cleaning this up! A few comments below. core/src/

Re: Review Request 33620: Patch for KAFKA-1690

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33620/#review87646 --- clients/src/main/java/org/apache/kafka/common/network/SSLTransportL

Jenkins build is back to normal : Kafka-trunk #507

See

[jira] [Commented] (KAFKA-2238) KafkaMetricsConfig cannot be configured in broker (KafkaConfig)

[ https://issues.apache.org/jira/browse/KAFKA-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582679#comment-14582679 ] Aditya Auradkar commented on KAFKA-2238: [~junrao][~nehanarkhede] Can one of you r

[jira] [Commented] (KAFKA-2238) KafkaMetricsConfig cannot be configured in broker (KafkaConfig)

[ https://issues.apache.org/jira/browse/KAFKA-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582698#comment-14582698 ] Gwen Shapira commented on KAFKA-2238: - [~aauradkar] I made significant changes to Kafk

Re: Review Request 35347: Patch for KAFKA-2249

> On June 11, 2015, 11:22 p.m., Jun Rao wrote: > > core/src/main/scala/kafka/server/KafkaConfig.scala, line 485 > > > > > > Is there a particular reason to change this to a long? It is used as LONG everywhere in the c

[jira] [Commented] (KAFKA-2238) KafkaMetricsConfig cannot be configured in broker (KafkaConfig)

[ https://issues.apache.org/jira/browse/KAFKA-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582722#comment-14582722 ] Jun Rao commented on KAFKA-2238: [~auradkar], took a look at your patch. Currently, the Ka

Re: Review Request 35261: Patch for KAFKA-2232

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35261/#review87661 --- Thanks for the patch. A few comments below. clients/src/main/java/

Re: [VOTE] KIP-25 System test improvements

+1 Thanks, Jun On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson wrote: > Hi Kafka, > > After a few rounds of discussion on KIP-25, there doesn't seem to be > opposition, so I'd like to propose a vote. > > Thanks, > Geoff > > On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson > wrote: > > >

Re: [VOTE] KIP-25 System test improvements

+1 Thanks, Jun On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson wrote: > Hi Kafka, > > After a few rounds of discussion on KIP-25, there doesn't seem to be > opposition, so I'd like to propose a vote. > > Thanks, > Geoff > > On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson > wrote: > > >

[jira] [Resolved] (KAFKA-2263) Update "Is it possible to delete a topic" wiki FAQ answer

[ https://issues.apache.org/jira/browse/KAFKA-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-2263. Resolution: Fixed Thanks for reporting this. Updated the wiki. > Update "Is it possible to delete a topic"

Re: [Discussion] New Consumer API / Protocol

Guozhang, Perhaps we can discuss this in our KIP hangout next week? Thanks, Jun On Tue, Jun 9, 2015 at 1:12 PM, Guozhang Wang wrote: > This email is to kick-off some discussion around the changes we want to > make on the new consumer APIs as well as their semantics. Here are a > not-comprehen

[jira] [Created] (KAFKA-2266) Client Selector can drop idle connections without notifying NetworkClient

Jason Gustafson created KAFKA-2266: -- Summary: Client Selector can drop idle connections without notifying NetworkClient Key: KAFKA-2266 URL: https://issues.apache.org/jira/browse/KAFKA-2266 Project:

Review Request 35371: Patch for KAFKA-2266

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35371/ --- Review request for kafka. Bugs: KAFKA-2266 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-2266) Client Selector can drop idle connections without notifying NetworkClient

[ https://issues.apache.org/jira/browse/KAFKA-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-2266: --- Attachment: KAFKA-2266.patch > Client Selector can drop idle connections without notifying Net

[jira] [Updated] (KAFKA-2266) Client Selector can drop idle connections without notifying NetworkClient

[ https://issues.apache.org/jira/browse/KAFKA-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-2266: --- Status: Patch Available (was: Open) > Client Selector can drop idle connections without notif

[jira] [Commented] (KAFKA-2266) Client Selector can drop idle connections without notifying NetworkClient

[ https://issues.apache.org/jira/browse/KAFKA-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582762#comment-14582762 ] Jason Gustafson commented on KAFKA-2266: Created reviewboard https://reviews.apach

Re: Review Request 35371: Patch for KAFKA-2266

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35371/#review87667 --- Thanks for the patch. Good catch! A minor comment below. clients/s

Re: Review Request 33614: Patch for KAFKA-2132

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33614/#review87670 --- Thanks for the patch. A few comments below. build.gradle

[jira] [Updated] (KAFKA-2132) Move Log4J appender to a separate module

[ https://issues.apache.org/jira/browse/KAFKA-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2132: --- Status: In Progress (was: Patch Available) > Move Log4J appender to a separate module > -

Re: [DISCUSS] KIP-23 - Add JSON/CSV output and looping options to ConsumerGroupCommand

Thanks for submitting the KIP, Ashish! Few questions. 1. Can you specify more details around how you expect csv output to be used. Same for json. 2. If we add these options, would you still need the old format. If csv/json offers more convenience, should we have a plan to phase out the old format?