[jira] [Commented] (KAFKA-2397) leave group request

[ https://issues.apache.org/jira/browse/KAFKA-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696602#comment-14696602 ] Ewen Cheslack-Postava commented on KAFKA-2397: -- [~becket_qin] I was more worr

Re: [copycat] How to upgrade connectors

Yes, I think that makes sense. As I see it, the tradeoffs are: 1. Complexity - adding these APIs increases the number of things connector developers need to implement just to get started. In a lot of cases, the first version of a connector might literally only have some connection string as a sett

Re: [copycat] support for "batch" jobs

The JDBC connector I started implementing just handles this manually, and isn't much code (and could be made into a simple utility): https://github.com/confluentinc/copycat-jdbc/blob/master/src/main/java/io/confluent/copycat/jdbc/JdbcSourceTask.java#L152 Given the current APIs, sources can just ha

Re: Copycat (KIP-26. PR-99) - plan on moving forward

For obvious reasons, I definitely won't object, but I will probably quickly have a follow-up patch to add a very basic system test for Copycat. I was trying to get it in with this patch since I think it is a good idea to encourage people to write system tests with major additions like this, but I t

Re: [Copycat] How will copycat serialize its metadata

I'm not sure the existing discussion is clear about how the format of offset data is decided. One possibility is that we choose one fixed format and that is what we use internally to store offsets no matter what serializer you choose. This would be similar to how the __offsets topic is currently ha

Re: [DISCUSS] Client-side Assignment for New Consumer

On Thu, Aug 13, 2015 at 11:07 PM, Neha Narkhede wrote: > Becket, > > As you say, the metadata hash addresses the concern you originally raised > about large topic subscriptions. Can you please list other problems you are > raising more clearly? It is more helpful to know problems that the proposa

[jira] [Commented] (KAFKA-2433) Remove documentation on dead configuration item: replica.lag.max.messages

[ https://issues.apache.org/jira/browse/KAFKA-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696934#comment-14696934 ] Stephen Powis commented on KAFKA-2433: -- You're totally right. Uploading a patch over

[jira] [Updated] (KAFKA-2433) Remove documentation on dead configuration item: replica.lag.max.messages

[ https://issues.apache.org/jira/browse/KAFKA-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Powis updated KAFKA-2433: - Attachment: documentation-update.diff updated fixed patch. > Remove documentation on dead configu

[jira] [Updated] (KAFKA-2433) Remove documentation on dead configuration item: replica.lag.max.messages

[ https://issues.apache.org/jira/browse/KAFKA-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Powis updated KAFKA-2433: - Attachment: (was: documentation-update.diff) > Remove documentation on dead configuration item

[DISCUSSION] Kafka 0.8.2.2 release?

Hi, Everyone, Since the release of Kafka 0.8.2.1, a number of people have reported an issue with snappy compression ( https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue has since been fixed in trunk (ju

Re: [DISCUSSION] Kafka 0.8.2.2 release?

I think this is a good idea as the change is minimal on our side and it has been tested in production for some time by the reporter. Best, Ismael On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao wrote: > Hi, Everyone, > > Since the release of Kafka 0.8.2.1, a number of people have reported an > issue w

Re: [DISCUSSION] Kafka 0.8.2.2 release?

+1 for 0.8.2.2 release On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma wrote: > I think this is a good idea as the change is minimal on our side and it has > been tested in production for some time by the reporter. > > Best, > Ismael > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao wrote: > > > Hi, E

[jira] [Updated] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira updated KAFKA-1387: Reviewer: Flavio Junqueira (was: Jun Rao) > Kafka getting stuck creating ephemeral node it

Re: [DISCUSSION] Kafka 0.8.2.2 release?

Just to clarify. Will KAFKA-2189 be the only patch in the release? On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy wrote: > +1 for 0.8.2.2 release > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma wrote: > > > I think this is a good idea as the change is minimal on our side and it > has > > be

[jira] [Updated] (KAFKA-2433) Remove documentation on dead configuration item: replica.lag.max.messages

[ https://issues.apache.org/jira/browse/KAFKA-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Powis updated KAFKA-2433: - Affects Version/s: (was: 0.8.2.1) (was: 0.8.1.2)

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697002#comment-14697002 ] Flavio Junqueira commented on KAFKA-1387: - I'm actually really sorry that this iss

Re: Copycat (KIP-26. PR-99) - plan on moving forward

Yes, I think the rename (other than being fun) will prevent some confusion with existing coordinator. On Aug 14, 2015 1:09 AM, "Ewen Cheslack-Postava" wrote: > For obvious reasons, I definitely won't object, but I will probably quickly > have a follow-up patch to add a very basic system test for

Re: [DISCUSSION] Kafka 0.8.2.2 release?

Will be nice to include Kafka-2308 and fix two critical snappy issues in the maintenance release. Gwen On Aug 14, 2015 6:16 AM, "Grant Henke" wrote: > Just to clarify. Will KAFKA-2189 be the only patch in the release? > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy > wrote: > > > +1 for 0

[jira] [Assigned] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)

[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Olson reassigned KAFKA-2434: --- Assignee: Andrew Olson > remove roundrobin identical topic constraint in consumer coordinator

[jira] [Created] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)

Andrew Olson created KAFKA-2434: --- Summary: remove roundrobin identical topic constraint in consumer coordinator (old API) Key: KAFKA-2434 URL: https://issues.apache.org/jira/browse/KAFKA-2434 Project: K

[jira] [Commented] (KAFKA-2196) remove roundrobin identical topic constraint in consumer coordinator

[ https://issues.apache.org/jira/browse/KAFKA-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697189#comment-14697189 ] Andrew Olson commented on KAFKA-2196: - > Doesn't this [1] code also need to be updated

[jira] [Commented] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)

[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697192#comment-14697192 ] Andrew Olson commented on KAFKA-2434: - I will submit a patch for this change shortly.

[jira] [Resolved] (KAFKA-1477) add authentication layer and initial JKS x509 implementation for brokers, producers and consumer for network communication

[ https://issues.apache.org/jira/browse/KAFKA-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani resolved KAFKA-1477. --- Resolution: Duplicate > add authentication layer and initial JKS x509 implemen

[jira] [Created] (KAFKA-2435) More optimally balanced partition assignment strategy

Andrew Olson created KAFKA-2435: --- Summary: More optimally balanced partition assignment strategy Key: KAFKA-2435 URL: https://issues.apache.org/jira/browse/KAFKA-2435 Project: Kafka Issue Type:

Review Request 37480: Patch for KAFKA-2434

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37480/ --- Review request for kafka. Bugs: KAFKA-2434 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)

[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697264#comment-14697264 ] Andrew Olson commented on KAFKA-2434: - Created reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)

[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Olson updated KAFKA-2434: Status: Patch Available (was: Open) > remove roundrobin identical topic constraint in consumer coor

[jira] [Updated] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)

[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Olson updated KAFKA-2434: Attachment: KAFKA-2434.patch > remove roundrobin identical topic constraint in consumer coordinator

Review Request 37481: Patch for KAFKA-2435

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37481/ --- Review request for kafka. Bugs: KAFKA-2435 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-2435) More optimally balanced partition assignment strategy

[ https://issues.apache.org/jira/browse/KAFKA-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697266#comment-14697266 ] Andrew Olson commented on KAFKA-2435: - Created reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-2435) More optimally balanced partition assignment strategy

[ https://issues.apache.org/jira/browse/KAFKA-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Olson updated KAFKA-2435: Status: Patch Available (was: Open) > More optimally balanced partition assignment strategy > -

[jira] [Updated] (KAFKA-2435) More optimally balanced partition assignment strategy

[ https://issues.apache.org/jira/browse/KAFKA-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Olson updated KAFKA-2435: Attachment: KAFKA-2435.patch > More optimally balanced partition assignment strategy > -

Re: [DISCUSS] Client-side Assignment for New Consumer

Ewen, I have no problem with adding an explicit version. As you mentioned, I was thinking that each protocol would manage that in its name field, but the explicit field is probably clearer for implementors. Becket, I was also a little concerned that the consumers would have some difficulty agreein

[GitHub] kafka pull request: Log the real exception which triggered a recon...

GitHub user jaikiran opened a pull request: https://github.com/apache/kafka/pull/138 Log the real exception which triggered a reconnect The commit here improves the logging in SimpleConsumer to log the real reason why a reconnect was attempted. Relates to https://issues.apache.org/

[jira] [Commented] (KAFKA-2221) Log the entire cause which caused a reconnect in the SimpleConsumer

[ https://issues.apache.org/jira/browse/KAFKA-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697401#comment-14697401 ] ASF GitHub Bot commented on KAFKA-2221: --- GitHub user jaikiran opened a pull request:

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697427#comment-14697427 ] Abhishek Nigam commented on KAFKA-1387: --- Thanks a lot for digging into this. Not sur

Re: [copycat] How to upgrade connectors

I think this is a great area to think about. What about the simpler informal scheme of just having cc ignore config it doesn't know about and instructing connector developers to maintain compatibility when renaming params (i.e. support both old and new) as we do for Kafka config. The steps would

Re: [copycat] support for "batch" jobs

I thought batch was dead? :-) Yeah I think this would be really useful. Kafka kind of allows you to unify batch and streams since you produce or consume your stream on your own schedule so you would want the ingress/egress to work the same. Ewen, rather than sleeping, I think the use case is that

Re: [DISCUSS] Client-side Assignment for New Consumer

Neha and Ewen, About the metadata change frequency. I guess it really depends on how frequent the metadata change might occur. If we run Kafka as a service, I can see that happens from time to time. As I can imagine people will create some topic, test and maybe delete the topic in some automated t

Re: [copycat] support for "batch" jobs

On Fri, Aug 14, 2015 at 10:57 AM, Jay Kreps wrote: > I thought batch was dead? :-) > > Yeah I think this would be really useful. Kafka kind of allows you to unify > batch and streams since you produce or consume your stream on your own > schedule so you would want the ingress/egress to work the s

[jira] [Commented] (KAFKA-2367) Add Copycat runtime data API

[ https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697499#comment-14697499 ] Jay Kreps commented on KAFKA-2367: -- I'm more negative on using Avro. Here's my thinking.

[jira] [Commented] (KAFKA-2402) Broker should create zkpath /isr_change_notification if it does not exist.

[ https://issues.apache.org/jira/browse/KAFKA-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697520#comment-14697520 ] ASF GitHub Bot commented on KAFKA-2402: --- Github user becketqin closed the pull reque

[GitHub] kafka pull request: KAFKA-2402: Create IsrChangeNotificationPath w...

Github user becketqin closed the pull request at: https://github.com/apache/kafka/pull/108 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[jira] [Updated] (KAFKA-2402) Broker should create zkpath /isr_change_notification if it does not exist.

[ https://issues.apache.org/jira/browse/KAFKA-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2402: --- Resolution: Fixed Status: Resolved (was: Patch Available) Fixed as part of KAFKA-2406 > Brok

[jira] [Commented] (KAFKA-2367) Add Copycat runtime data API

[ https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697567#comment-14697567 ] Ismael Juma commented on KAFKA-2367: I wasn't aware that it was so difficult to upgrad

[jira] [Commented] (KAFKA-1695) Authenticate connection to Zookeeper

[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697588#comment-14697588 ] Parth Brahmbhatt commented on KAFKA-1695: - [~fpj] When you have time can you pleas

Re: [DISCUSS] Client-side Assignment for New Consumer

On Fri, Aug 14, 2015 at 10:59 AM, Jiangjie Qin wrote: > Neha and Ewen, > > About the metadata change frequency. I guess it really depends on how > frequent the metadata change might occur. If we run Kafka as a service, I > can see that happens from time to time. As I can imagine people will creat

[jira] [Commented] (KAFKA-2172) Round-robin partition assignment strategy too restrictive

[ https://issues.apache.org/jira/browse/KAFKA-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697662#comment-14697662 ] Andrew Olson commented on KAFKA-2172: - [~jjkoshy] I've implemented a new assignment al

[jira] [Reopened] (KAFKA-1782) Junit3 Misusage

[ https://issues.apache.org/jira/browse/KAFKA-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson reopened KAFKA-1782: It looks like this patch broke some of the tests (e.g. ConsumerTest). We probably need to go th

[jira] [Commented] (KAFKA-2388) subscribe(topic)/unsubscribe(topic) should either take a callback to allow user to handle exceptions or it should be synchronous.

[ https://issues.apache.org/jira/browse/KAFKA-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697688#comment-14697688 ] ASF GitHub Bot commented on KAFKA-2388: --- GitHub user hachikuji opened a pull request

[GitHub] kafka pull request: KAFKA-2388 [WIP]; refactor KafkaConsumer subsc...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/139 KAFKA-2388 [WIP]; refactor KafkaConsumer subscribe API You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-2388 Alternativel

[jira] [Assigned] (KAFKA-2388) subscribe(topic)/unsubscribe(topic) should either take a callback to allow user to handle exceptions or it should be synchronous.

[ https://issues.apache.org/jira/browse/KAFKA-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson reassigned KAFKA-2388: -- Assignee: Jason Gustafson (was: Onur Karaman) > subscribe(topic)/unsubscribe(topic) sh

[jira] [Commented] (KAFKA-2388) subscribe(topic)/unsubscribe(topic) should either take a callback to allow user to handle exceptions or it should be synchronous.

[ https://issues.apache.org/jira/browse/KAFKA-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697692#comment-14697692 ] Jason Gustafson commented on KAFKA-2388: [~onurkaraman] I submitted a WIP patch fo

[jira] [Work started] (KAFKA-2388) subscribe(topic)/unsubscribe(topic) should either take a callback to allow user to handle exceptions or it should be synchronous.

[ https://issues.apache.org/jira/browse/KAFKA-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2388 started by Jason Gustafson. -- > subscribe(topic)/unsubscribe(topic) should either take a callback to allow

Re: [DISCUSS] Client-side Assignment for New Consumer

> > So if you're just subscribing to one or a couple of topics, why not just > compute the hash by filtering out everything but the topics you are > subscribed to? The problem there is if you ever add/remove subscriptions > and want to support rolling upgrades. If the group was subscribed to topic

[GitHub] kafka pull request: KAFKA-1782: Follow up - add missing @Test anno...

GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/140 KAFKA-1782: Follow up - add missing @Test annotations. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafka kafka-1782-followup Alternati

[jira] [Commented] (KAFKA-1782) Junit3 Misusage

[ https://issues.apache.org/jira/browse/KAFKA-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697899#comment-14697899 ] ASF GitHub Bot commented on KAFKA-1782: --- GitHub user ewencp opened a pull request:

[jira] [Updated] (KAFKA-1782) Junit3 Misusage

[ https://issues.apache.org/jira/browse/KAFKA-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-1782: - Status: Patch Available (was: Reopened) > Junit3 Misusage > --- > >

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697907#comment-14697907 ] Guozhang Wang commented on KAFKA-1387: -- Thanks [~fpj], this is very helpful. Just to

[jira] [Updated] (KAFKA-2366) Initial patch for Copycat

[ https://issues.apache.org/jira/browse/KAFKA-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2366: Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request 99

[GitHub] kafka pull request: KAFKA-2366: Copycat

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/99 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[jira] [Commented] (KAFKA-2366) Initial patch for Copycat

[ https://issues.apache.org/jira/browse/KAFKA-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697920#comment-14697920 ] ASF GitHub Bot commented on KAFKA-2366: --- Github user asfgit closed the pull request

[jira] [Commented] (KAFKA-2366) Initial patch for Copycat

[ https://issues.apache.org/jira/browse/KAFKA-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697923#comment-14697923 ] Gwen Shapira commented on KAFKA-2366: - Merged to trunk. Thank you for your contributio

[jira] [Commented] (KAFKA-2367) Add Copycat runtime data API

[ https://issues.apache.org/jira/browse/KAFKA-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697942#comment-14697942 ] Gwen Shapira commented on KAFKA-2367: - The fact that Avro brings in bazillion things d

Re: Review Request 33049: Patch for KAFKA-2084

> On Aug. 14, 2015, 1:57 a.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/metrics/stats/Rate.java, > > lines 69-77 > > > > > > This is probably not the right place to throw QuotaViolationExcept

[jira] [Commented] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14698000#comment-14698000 ] Aditya A Auradkar commented on KAFKA-2084: -- Updated reviewboard https://reviews.a

Re: Review Request 33049: Patch for KAFKA-2084

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/ --- (Updated Aug. 15, 2015, 12:43 a.m.) Review request for kafka, Joel Koshy and Ju

Re: Review Request 33049: Patch for KAFKA-2084

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/ --- (Updated Aug. 15, 2015, 12:43 a.m.) Review request for kafka, Joel Koshy and Ju

[jira] [Updated] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-2084: - Attachment: KAFKA-2084_2015-08-14_17:43:00.patch > byte rate metrics per client ID (produc

Re: Review Request 33049: Patch for KAFKA-2084

> On Aug. 14, 2015, 1:57 a.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/metrics/stats/Rate.java, > > lines 69-77 > > > > > > This is probably not the right place to throw QuotaViolationExcept

Re: [DISCUSS] Client-side Assignment for New Consumer

I think metadata churn is an interesting problem and it would be nice if we had some approach to deal with it. I wonder if it would be sufficient to have a setting on the client (or the server if we use centralized assignment) which dampens the rate of rebalancing. Basically the effect would be to

Re: [Copycat] How will copycat serialize its metadata

Yeah, I missed the option to match serialization of offsets to data, which solves the configuration overhead. It still doesn't give us the ability to easily evolve the metadata messages or to use them in monitoring tools. And I am still not clear of the benefits of using user-defined serializatio

Re: [DISCUSS] Client-side Assignment for New Consumer

Jason, The way this is handled today (in the old consumer) is by grouping several rebalance events into one rebalance operation (similar to how we handle log flushes using "group commit"). The idea is to batch several rebalance events while one rebalance operation is in progress. This is naturall

Re: Review Request 33049: Patch for KAFKA-2084

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/#review95496 --- Ship it! Minor edits that I will take care of on check-in. core/s

[jira] [Resolved] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-2084. --- Resolution: Fixed Thanks for the patches - committed to trunk. > byte rate metrics per client ID (pro

[jira] [Updated] (KAFKA-2433) Remove documentation on dead configuration item: replica.lag.max.messages

[ https://issues.apache.org/jira/browse/KAFKA-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2433: Resolution: Fixed Fix Version/s: 0.8.3 Status: Resolved (was: Patch Available) Th

Re: [copycat] support for "batch" jobs

For sink side: I'm a bit more comfortable with "batch mode" than with "run once and it will do something every hour" because the former puts scheduling firmly in the user hands (and their cron), the latter means that connector developers need to figure out schedules. For source side: I'm not convi

Re: [copycat] How to upgrade connectors

The "ignore new configs" plan is good, IMO. I just don't know if its feasible in current Copycat: I'm not sure CC can ignore configuration on the connector behalf. Also, this is something we will want to log very clearly and I'm not sure this is doable outside the connector. Regarding complex impl

Re: [Copycat] How will copycat serialize its metadata

On Fri, Aug 14, 2015 at 6:35 PM, Gwen Shapira wrote: > Yeah, I missed the option to match serialization of offsets to data, which > solves the configuration overhead. > > It still doesn't give us the ability to easily evolve the metadata messages > or to use them in monitoring tools. > > And I am

Re: [DISCUSS] Client-side Assignment for New Consumer

Ewen, I agree that if there is a churn in metadata, the consumers need several rounds of rebalances to succeed. The difference I am thinking is that with coordinator as single source of truth, we can let the consumer finish one round of rebalance, work for a while and start the next round of rebal

[GitHub] kafka pull request: KAFKA-1782: Follow up - add missing @Test anno...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/140 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Updated] (KAFKA-1782) Junit3 Misusage

[ https://issues.apache.org/jira/browse/KAFKA-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1782: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[jira] [Commented] (KAFKA-1782) Junit3 Misusage

[ https://issues.apache.org/jira/browse/KAFKA-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14698109#comment-14698109 ] ASF GitHub Bot commented on KAFKA-1782: --- Github user asfgit closed the pull request