Re: KIP-122: Add a tool to Reset Consumer Group Offsets

@Vahid: make sense to add "new lag" info IMO, I will update the KIP. @Becket: 1. About deleting, I think ConsumerGroupCommand already has an option to delete Group information by topic. From delete docs: "Pass in groups to delete topic partition offsets and ownership information over the entire c

[jira] [Updated] (KAFKA-4806) KafkaConsumer: ConsumerConfig gets logged twice.

[ https://issues.apache.org/jira/browse/KAFKA-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Ebert updated KAFKA-4806: --- Fix Version/s: 0.10.2.1 > KafkaConsumer: ConsumerConfig gets logged twice. > -

[jira] [Updated] (KAFKA-4569) Transient failure in org.apache.kafka.clients.consumer.KafkaConsumerTest.testWakeupWithFetchDataAvailable

[ https://issues.apache.org/jira/browse/KAFKA-4569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4569: --- Fix Version/s: 0.10.2.1 > Transient failure in > org.apache.kafka.clients.consumer.KafkaConsumerTest.

[jira] [Commented] (KAFKA-4569) Transient failure in org.apache.kafka.clients.consumer.KafkaConsumerTest.testWakeupWithFetchDataAvailable

[ https://issues.apache.org/jira/browse/KAFKA-4569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885611#comment-15885611 ] Ismael Juma commented on KAFKA-4569: A few comments: 1. We are using a MockTime witho

[jira] [Comment Edited] (KAFKA-4569) Transient failure in org.apache.kafka.clients.consumer.KafkaConsumerTest.testWakeupWithFetchDataAvailable

[ https://issues.apache.org/jira/browse/KAFKA-4569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885611#comment-15885611 ] Ismael Juma edited comment on KAFKA-4569 at 2/27/17 11:28 AM: --

[jira] [Comment Edited] (KAFKA-4569) Transient failure in org.apache.kafka.clients.consumer.KafkaConsumerTest.testWakeupWithFetchDataAvailable

[ https://issues.apache.org/jira/browse/KAFKA-4569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885611#comment-15885611 ] Ismael Juma edited comment on KAFKA-4569 at 2/27/17 11:27 AM: --

[jira] [Commented] (KAFKA-4806) KafkaConsumer: ConsumerConfig gets logged twice.

[ https://issues.apache.org/jira/browse/KAFKA-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885638#comment-15885638 ] ASF GitHub Bot commented on KAFKA-4806: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-4806) KafkaConsumer: ConsumerConfig gets logged twice.

[ https://issues.apache.org/jira/browse/KAFKA-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4806. Resolution: Fixed Fix Version/s: 0.10.3.0 Issue resolved by pull request 2600 [https://github

[GitHub] kafka pull request #2600: KAFKA-4806: Prevent double logging of ConsumerConf...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2600 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Reopened] (KAFKA-4779) Failure in kafka/tests/kafkatest/tests/core/security_rolling_upgrade_test.py

[ https://issues.apache.org/jira/browse/KAFKA-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reopened KAFKA-4779: > Failure in kafka/tests/kafkatest/tests/core/security_rolling_upgrade_test.py > ---

[jira] [Commented] (KAFKA-4779) Failure in kafka/tests/kafkatest/tests/core/security_rolling_upgrade_test.py

[ https://issues.apache.org/jira/browse/KAFKA-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885716#comment-15885716 ] Ismael Juma commented on KAFKA-4779: The test failed again, this time with a different

[jira] [Created] (KAFKA-4808) send of null key to a compacted topic should throw error back to user

Ismael Juma created KAFKA-4808: -- Summary: send of null key to a compacted topic should throw error back to user Key: KAFKA-4808 URL: https://issues.apache.org/jira/browse/KAFKA-4808 Project: Kafka

[jira] [Commented] (KAFKA-4808) send of null key to a compacted topic should throw error back to user

[ https://issues.apache.org/jira/browse/KAFKA-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885759#comment-15885759 ] Ismael Juma commented on KAFKA-4808: cc [~norwood] > send of null key to a compacted

Re: [DISCUSS] KIP-128: Add ByteArrayConverter for Kafka Connect

It's a different interface that's being implemented. The functionality is the same (since it's just a simple pass through), but we intentionally named Converters differently than Serializers since they do more work than Serializers (besides the normal serialization they also need to convert between

Re: [VOTE] KIP-81: Bound Fetch memory usage in the consumer

Apologies for the late response. Thanks Jason for the suggestion. Yes you are right, the Coordinator connection is "tagged" with a different id, so we could retrieve it in NetworkReceive to make the distinction. However, currently the coordinator connection are made different by using: Integer.MAX

Re: [DISCUSS] KIP-126 - Allow KafkaProducer to batch based on uncompressed size

Hi Becket, Seems like an interesting idea. I had couple of questions : 1) How do we decide when the batch should be split? 2) What do you mean by slowly lowering the "actual" compression ratio? An example would really help here. Thanks, Mayuresh On Fri, Feb 24, 2017 at 3:17 PM, Becket Qin wrot

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Hi Ismael, Joel, Becket Would you mind taking a look at this. We require 2 more binding votes for the KIP to pass. Thanks, Mayuresh On Thu, Feb 23, 2017 at 10:57 AM, Dong Lin wrote: > +1 (non-binding) > > On Wed, Feb 22, 2017 at 10:52 PM, Manikumar > wrote: > > > +1 (non-binding) > > > > On

[jira] [Commented] (KAFKA-4808) send of null key to a compacted topic should throw error back to user

[ https://issues.apache.org/jira/browse/KAFKA-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886288#comment-15886288 ] Mayuresh Gharat commented on KAFKA-4808: [~ijuma] I was thinking if throwing "INVA

[jira] [Assigned] (KAFKA-4808) send of null key to a compacted topic should throw error back to user

[ https://issues.apache.org/jira/browse/KAFKA-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat reassigned KAFKA-4808: -- Assignee: Mayuresh Gharat > send of null key to a compacted topic should throw error ba

[jira] [Commented] (KAFKA-4808) send of null key to a compacted topic should throw error back to user

[ https://issues.apache.org/jira/browse/KAFKA-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886297#comment-15886297 ] Ismael Juma commented on KAFKA-4808: [~mgharat], I think a separate error code would b

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Hi Mayuresh, Sorry for the delay. The updated KIP states that there is no compatibility impact, but that doesn't seem right. The fact that we changed the type of Session.principal to `Principal` means that any code that expects it to be `KafkaPrincipal` will break. Either because of declared types

Re: [DISCUSS] KIP-126 - Allow KafkaProducer to batch based on uncompressed size

Hi Becket, Thanks for the expatiation. Regarding : 1) The batch would be split when an RecordTooLargeException is received. Lets say we sent the batch over the wire and received a RecordTooLargeException, how do we split it as once we add the message to the batch we loose the message level granul

Re: [DISCUSS] KIP-126 - Allow KafkaProducer to batch based on uncompressed size

Hey Mayuresh, 1) The batch would be split when an RecordTooLargeException is received. 2) Not lower the actual compression ratio, but lower the estimated compression ratio "according to" the Actual Compression Ratio(ACR). An example, let's start with Estimated Compression Ratio (ECR) = 1.0. Say t

[jira] [Resolved] (KAFKA-4744) Streams_bounce test failing occassionally

[ https://issues.apache.org/jira/browse/KAFKA-4744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4744. -- Resolution: Fixed Issue resolved by pull request 2601 [https://github.com/apache/kafka/pull/2601

[GitHub] kafka pull request #2601: KAFKA-4744: Increased timeout for bounce test

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2601 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4744) Streams_bounce test failing occassionally

[ https://issues.apache.org/jira/browse/KAFKA-4744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886354#comment-15886354 ] ASF GitHub Bot commented on KAFKA-4744: --- Github user asfgit closed the pull request

[jira] [Commented] (KAFKA-4808) send of null key to a compacted topic should throw error back to user

[ https://issues.apache.org/jira/browse/KAFKA-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886365#comment-15886365 ] Mayuresh Gharat commented on KAFKA-4808: [~ijuma] Sure we can provide a better err

Re: [DISCUSS] KIP-128: Add ByteArrayConverter for Kafka Connect

Thanks Ewen, "use the corresponding serializer internally and just add in the extra conversion steps for the data API" sounds good to me. Guozhang On Mon, Feb 27, 2017 at 8:24 AM, Ewen Cheslack-Postava wrote: > It's a different interface that's being implemented. The functionality is > the sa

Kafka MirrorMaker issues

Hi Kafka Gurus :) I am facing issues with KafkaMirror, I am using Kafka 0.10.1.1 and trying to use mirroring to create backup of kafka logs or perhaps this might be a great idea to do it, please let me know if its not. my consumer.properties: bootstrap.servers=localhost:9092 group.id=mirror pro

[jira] [Updated] (KAFKA-4809) sanity_checks/test_verifiable_producer.py should test 0.10.x and not 0.8

[ https://issues.apache.org/jira/browse/KAFKA-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated KAFKA-4809: --- Description: {{sanity_checks/test_verifiable_producer.py}} should test the latest 0.10.0, 0.10

[jira] [Created] (KAFKA-4809) sanity_checks/test_verifiable_producer.py should test 0.10.1 and 0.10.2 and not 0.8

Colin P. McCabe created KAFKA-4809: -- Summary: sanity_checks/test_verifiable_producer.py should test 0.10.1 and 0.10.2 and not 0.8 Key: KAFKA-4809 URL: https://issues.apache.org/jira/browse/KAFKA-4809

[jira] [Updated] (KAFKA-4809) sanity_checks/test_verifiable_producer.py should test 0.10.x and not 0.8

[ https://issues.apache.org/jira/browse/KAFKA-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated KAFKA-4809: --- Summary: sanity_checks/test_verifiable_producer.py should test 0.10.x and not 0.8 (was: sanit

[GitHub] kafka pull request #2602: KAFKA-4809: sanity_checks/test_verifiable_producer...

GitHub user cmccabe opened a pull request: https://github.com/apache/kafka/pull/2602 KAFKA-4809: sanity_checks/test_verifiable_producer.py should test 0.1… …0.x and not 0.8 You can merge this pull request into a Git repository by running: $ git pull https://github.com/cmcca

[jira] [Commented] (KAFKA-4809) sanity_checks/test_verifiable_producer.py should test 0.10.x and not 0.8

[ https://issues.apache.org/jira/browse/KAFKA-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886429#comment-15886429 ] ASF GitHub Bot commented on KAFKA-4809: --- GitHub user cmccabe opened a pull request:

[GitHub] kafka pull request #2603: MINOR: Minor reduce unnecessary calls to time.mill...

GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/2603 MINOR: Minor reduce unnecessary calls to time.millisecond (part 2) Avoid calling time.milliseconds more often than necessary. Cleaning and committing logic can use the timestamp at the start of

[jira] [Commented] (KAFKA-3990) Kafka New Producer may raise an OutOfMemoryError

[ https://issues.apache.org/jira/browse/KAFKA-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886466#comment-15886466 ] Adrian McCague commented on KAFKA-3990: --- For what it's worth witnessing this as well

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

Hi, Dong, For RAID5, I am not sure the rebuild cost is a big concern. If a disk fails, typically an admin has to bring down the broker, replace the failed disk with a new one, trigger the RAID rebuild, and bring up the broker. This way, there is no performance impact at runtime due to rebuild. The

Re: [DISCUSS] KIP-82 - Add Record Headers

a few comments on the KIP as it is now: 1. instead of add(Header) + add (Iterable) i suggest we use add + addAll. this is more in line with how java collections work and may therefor be more intuitive 2. common user code dealing with headers will want get("someKey") / set("someKey"), or equivalen

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

Unfortunately RAID-5/6 is not typically advised anymore due to failure issues, as Dong mentions, e.g.: http://www.zdnet.com/article/why-raid-6-stops-working-in-2019/ Eno > On 27 Feb 2017, at 21:16, Jun Rao wrote: > > Hi, Dong,

[jira] [Commented] (KAFKA-4809) sanity_checks/test_verifiable_producer.py should test 0.10.x and not 0.8

[ https://issues.apache.org/jira/browse/KAFKA-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886610#comment-15886610 ] Ewen Cheslack-Postava commented on KAFKA-4809: -- The reason we want to make su

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

Hi, Eno, Thanks for the pointers. Doesn't RAID-10 have a similar issue during rebuild? In both cases, all data on existing disks have to be read during rebuild? RAID-10 seems to still be used widely. Jun On Mon, Feb 27, 2017 at 1:38 PM, Eno Thereska wrote: > Unfortunately RAID-5/6 is not typic

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

RAID-10's code is much simpler (just stripe plus mirror) and under failure the recovery is much faster since it just has to read from a mirror, not several disks to reconstruct the data. Of course, the price paid is that mirroring is more expensive in terms of storage space. E.g., see discussi

[jira] [Commented] (KAFKA-4794) Add access to OffsetStorageReader from SourceConnector

[ https://issues.apache.org/jira/browse/KAFKA-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886680#comment-15886680 ] ASF GitHub Bot commented on KAFKA-4794: --- GitHub user fhussonnois opened a pull reque

[GitHub] kafka pull request #2604: KAFKA-4794: Add access to OffsetStorageReader from...

GitHub user fhussonnois opened a pull request: https://github.com/apache/kafka/pull/2604 KAFKA-4794: Add access to OffsetStorageReader from SourceConnector This a first attempt to implement Add access to OffsetStorageReader from Source Connector. I am not sure if I did it right.

Jenkins build is back to normal : kafka-trunk-jdk8 #1306

See

[jira] [Updated] (KAFKA-4809) docker/run_tests.sh should be able to run sanity_checks/test_verifiable_producer.py against version 0.8.2.2

[ https://issues.apache.org/jira/browse/KAFKA-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated KAFKA-4809: --- Description: docker/run_tests.sh should be able to run sanity_checks/test_verifiable_producer.

[jira] [Commented] (KAFKA-4809) docker/run_tests.sh should be able to run sanity_checks/test_verifiable_producer.py against version 0.8.2.2

[ https://issues.apache.org/jira/browse/KAFKA-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886715#comment-15886715 ] Colin P. McCabe commented on KAFKA-4809: Thanks for the background, [~ewencp]. I

[jira] [Updated] (KAFKA-4809) docker/run_tests.sh should be able to run sanity_checks/test_verifiable_producer.py against version 0.8.2.2

[ https://issues.apache.org/jira/browse/KAFKA-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated KAFKA-4809: --- Description: {{docker/run_tests.sh }}should be able to run {{sanity_checks/test_verifiable_pro

[jira] [Updated] (KAFKA-4809) docker/run_tests.sh should be able to run sanity_checks/test_verifiable_producer.py against version 0.8.2.2

[ https://issues.apache.org/jira/browse/KAFKA-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated KAFKA-4809: --- Summary: docker/run_tests.sh should be able to run sanity_checks/test_verifiable_producer.py a

[jira] [Updated] (KAFKA-4809) docker/run_tests.sh should set up /opt/kafka-dev to be the source directory

[ https://issues.apache.org/jira/browse/KAFKA-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated KAFKA-4809: --- Summary: docker/run_tests.sh should set up /opt/kafka-dev to be the source directory (was: do

[jira] [Updated] (KAFKA-4809) docker/run_tests.sh should set up /opt/kafka-dev the same way Vagrant does

[ https://issues.apache.org/jira/browse/KAFKA-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated KAFKA-4809: --- Summary: docker/run_tests.sh should set up /opt/kafka-dev the same way Vagrant does (was: doc

[jira] [Updated] (KAFKA-4809) docker/run_tests.sh should set up /opt/kafka-dev to be the source directory

[ https://issues.apache.org/jira/browse/KAFKA-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe updated KAFKA-4809: --- Description: {{docker/run_tests.sh }} should be able to run {{sanity_checks/test_verifiable_pr

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Hi Ismael, Yeah. I agree that it might break the clients if the user is using the kafkaPrincipal directly. But since KafkaPrincipal is also a Java Principal and I think, it would be a right thing to do replace the kafkaPrincipal with Java Principal at this stage than later. We can mention in the

[GitHub] kafka-site pull request #48: Filled in og meta tags

GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka-site/pull/48 Filled in og meta tags Set og meta tags so sharing links to places like LinkedIn, Facebook, Twitter gives people good defaults. You can merge this pull request into a Git repository by runnin

[jira] [Commented] (KAFKA-4778) OOM on kafka-streams instances with high numbers of unreaped Record classes

[ https://issues.apache.org/jira/browse/KAFKA-4778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886778#comment-15886778 ] Guozhang Wang commented on KAFKA-4778: -- [~peoplemerge] What I was looking for is actu

Re: [DISCUSS] KIP-126 - Allow KafkaProducer to batch based on uncompressed size

> > Lets say we sent the batch over the wire and received a > RecordTooLargeException, how do we split it as once we add the message to > the batch we loose the message level granularity. We will have to > decompress, do deep iteration and split and again compress. right? This > looks like a perfor

[jira] [Created] (KAFKA-4810) SchemaBuilder should be more lax about checking that fields are unset if they are being set to the same value

Ewen Cheslack-Postava created KAFKA-4810: Summary: SchemaBuilder should be more lax about checking that fields are unset if they are being set to the same value Key: KAFKA-4810 URL: https://issues.apache.o

[GitHub] kafka pull request #2605: Kafka 4738:Remove generic type of class ClientStat...

GitHub user sharad-develop opened a pull request: https://github.com/apache/kafka/pull/2605 Kafka 4738:Remove generic type of class ClientState Remove generic type of class ClientState. Also removed Generic type from TaskAssignor as it referred to same generic type of ClientState Y

Re: Consumption on a explicitly (dynamically) created topic has a 5 minute delay

> On Feb 26, 2017, at 10:36 PM, Jaikiran Pai wrote: > > An update on one of the workaround I tried - I added some logic in our > consumption part to wait for the KafkaConsumer.partitionsFor() to return the > topic name in the list, before actually considering the KafkaConsumer ready > to con

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Breaking clients without a deprecation period is something we only do as a last resort. Is there strong justification for doing it here? Ismael On Mon, Feb 27, 2017 at 11:28 PM, Mayuresh Gharat < gharatmayures...@gmail.com> wrote: > Hi Ismael, > > Yeah. I agree that it might break the clients if

Re: [DISCUSS] KIP-82 - Add Record Headers

If I understand correctly, the suggestion is to let headers be mutable on the producer side basically until after they've passed through the interceptors. That sounds like a reasonable compromise to me. @Becket 3. It might be useful to have headers at MessageSet level as well so we can > avoid de

Re: Consumption on a explicitly (dynamically) created topic has a 5 minute delay

Yes, that was going to be my last resort attempt :) I'm going to give that a try today and see how it goes. Although it isn't that great of a solution, I don't mind doing it since the logic resides at one single place in our application. -Jaikiran On Tuesday, February 28, 2017, James Cheng wrote

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

Hi Jun, In addition to the Eno's reference of why rebuild time with RAID-5 is more expensive, another concern is that RAID-5 will fail if more than one disk fails. JBOD is still works with 1+ disk failure and has better performance with one disk failure. These seems like good argument for using JB

[jira] [Created] (KAFKA-4811) ReplicaFetchThread may fail to create due to existing metric

Jun Rao created KAFKA-4811: -- Summary: ReplicaFetchThread may fail to create due to existing metric Key: KAFKA-4811 URL: https://issues.apache.org/jira/browse/KAFKA-4811 Project: Kafka Issue Type: B

Re: [VOTE] KIP-118: Drop Support for Java 7 in Kafka 0.11

Thanks to everyone who voted and provided feedback. +1 (binding) from me too. The vote has passed with 4 binding votes (Grant, Jason, Guozhang, Ismael) and 11 non-binding votes (Bill, Damian, Eno, Edoardo, Mickael, Bharat, Onur, Vahid, Colin, Apurva, Tom). There were no 0 or -1 votes. I have upda

[jira] [Commented] (KAFKA-4811) ReplicaFetchThread may fail to create due to existing metric

[ https://issues.apache.org/jira/browse/KAFKA-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887119#comment-15887119 ] Jun Rao commented on KAFKA-4811: In AbstractFetcherManager, we only create a new ReplicaFe

[jira] [Updated] (KAFKA-4811) ReplicaFetchThread may fail to create due to existing metric

[ https://issues.apache.org/jira/browse/KAFKA-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4811: --- Labels: newbie (was: ) > ReplicaFetchThread may fail to create due to existing metric > -

[jira] [Updated] (KAFKA-4811) ReplicaFetchThread may fail to create due to existing metric

[ https://issues.apache.org/jira/browse/KAFKA-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4811: --- Component/s: replication > ReplicaFetchThread may fail to create due to existing metric >

[jira] [Updated] (KAFKA-3959) __consumer_offsets wrong number of replicas at startup

[ https://issues.apache.org/jira/browse/KAFKA-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3959: --- Resolution: Fixed Assignee: Onur Karaman (was: Ewen Cheslack-Postava) Status: Resolved

[jira] [Updated] (KAFKA-4586) Add purgeDataBefore() API in AdminClient

[ https://issues.apache.org/jira/browse/KAFKA-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4586: --- Fix Version/s: 0.10.3.0 > Add purgeDataBefore() API in AdminClient > -

[jira] [Updated] (KAFKA-4586) Add purgeDataBefore() API in AdminClient

[ https://issues.apache.org/jira/browse/KAFKA-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4586: --- Status: Patch Available (was: Open) > Add purgeDataBefore() API in AdminClient >

[DISCUSS] 0.10.3.0/0.11.0.0 release planning

Hi all, With 0.10.2.0 out of the way, I would like to volunteer to be the release manager for our next time-based release. See https://cwiki.apache.org/c onfluence/display/KAFKA/Time+Based+Release+Plan if you missed previous communication on time-based releases or need a reminder. I put together

Re: [VOTE] KIP-118: Drop Support for Java 7 in Kafka 0.11

+1 On Mon, Feb 27, 2017 at 6:38 PM, Ismael Juma wrote: > Thanks to everyone who voted and provided feedback. +1 (binding) from me > too. > > The vote has passed with 4 binding votes (Grant, Jason, Guozhang, Ismael) > and 11 non-binding votes (Bill, Damian, Eno, Edoardo, Mickael, Bharat, > Onur,

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

Hi Ismael, Thanks for volunteering on the new release. I think 0.11.0.0 makes a lot of sense given the new big features we are intended to include. Thanks, Jiangjie (Becket) Qin On Mon, Feb 27, 2017 at 7:47 PM, Ismael Juma wrote: > Hi all, > > With 0.10.2.0 out of the way, I would like to vo

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

+1 for major version bump. A good bit of deprecated code I would like to see removed especially on old consumer side plus a few other settings defaults changed such as the brief discussion on mirrormaker options a few months back. Just be good to continue to make the new user experience a lot more

Re: [VOTE] KIP-107: Add purgeDataBefore() API in AdminClient

Hi Jun and everyone, I would like to change the KIP in the following way. Currently, if any replica if offline, the purge result for a partition will be NotEnoughReplicasException and its low_watermark will be 0. The motivation for this approach is that we want to guarantee that the data before pu

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

+1 for 0.11.0.0 Guozhang On Mon, Feb 27, 2017 at 8:14 PM, Becket Qin wrote: > Hi Ismael, > > Thanks for volunteering on the new release. > > I think 0.11.0.0 makes a lot of sense given the new big features we are > intended to include. > > Thanks, > > Jiangjie (Becket) Qin > > On Mon, Feb 27, 2

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

+1 on 0.11.0.0. Can we also include KIP-54 to the list? The PR for this KIP is ready for review. Thanks. --Vahid From: Ismael Juma To: dev@kafka.apache.org Date: 02/27/2017 07:47 PM Subject:[DISCUSS] 0.10.3.0/0.11.0.0 release planning Sent by:isma...@gmail.com Hi

Re: [VOTE] KIP-81: Bound Fetch memory usage in the consumer

Hey Mickael, The suggestion to add something to Node makes sense. I could imagine for example adding a flag to indicate that the connection has a higher "priority," meaning that we can allocate outside of the memory pool if necessary. That would still be generic even if the only use case is the co

[jira] [Commented] (KAFKA-4722) Add application.id to StreamThread name

[ https://issues.apache.org/jira/browse/KAFKA-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887301#comment-15887301 ] Matthias J. Sax commented on KAFKA-4722: [~sharad.develop] I left already a commen

[jira] [Commented] (KAFKA-4623) Change Default unclean.leader.election.enabled from True to False

[ https://issues.apache.org/jira/browse/KAFKA-4623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887302#comment-15887302 ] Matthias J. Sax commented on KAFKA-4623: [~sharad.develop]: [~ewencp] left already

[jira] [Commented] (KAFKA-4706) Unify StreamsKafkaClient instances

[ https://issues.apache.org/jira/browse/KAFKA-4706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887304#comment-15887304 ] Matthias J. Sax commented on KAFKA-4706: [~sharad.develop] You did close your PR.

[jira] [Commented] (KAFKA-4811) ReplicaFetchThread may fail to create due to existing metric

[ https://issues.apache.org/jira/browse/KAFKA-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887308#comment-15887308 ] huxi commented on KAFKA-4811: - [~junrao] what about the port change? Do we also need to consid

[jira] [Commented] (KAFKA-4738) Remove generic type of class ClientState

[ https://issues.apache.org/jira/browse/KAFKA-4738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887309#comment-15887309 ] Matthias J. Sax commented on KAFKA-4738: [~sharad.develop] Are you working on this

[jira] [Assigned] (KAFKA-4811) ReplicaFetchThread may fail to create due to existing metric

[ https://issues.apache.org/jira/browse/KAFKA-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huxi reassigned KAFKA-4811: --- Assignee: huxi > ReplicaFetchThread may fail to create due to existing metric > -

[GitHub] kafka pull request #2606: kafka4811: ReplicaFetchThread may fail to create d...

GitHub user amethystic opened a pull request: https://github.com/apache/kafka/pull/2606 kafka4811: ReplicaFetchThread may fail to create due to existing metric Have fetcherThreadMap keyed off brokerId + fetcherId instead of broker + fetcherId, but did not consider the case where por

[jira] [Resolved] (KAFKA-1523) Implement transaction manager module

[ https://issues.apache.org/jira/browse/KAFKA-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-1523. Resolution: Unresolved This work has been superseded by KIP-98: https://cwiki.apache.org/co

[jira] [Resolved] (KAFKA-1524) Implement transactional producer

[ https://issues.apache.org/jira/browse/KAFKA-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-1524. Resolution: Unresolved This work has been superseded by KIP-98: https://cwiki.apache.org/co

[jira] [Resolved] (KAFKA-1525) DumpLogSegments should print transaction IDs

[ https://issues.apache.org/jira/browse/KAFKA-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-1525. Resolution: Unresolved This work has been superseded by KIP-98: https://cwiki.apache.org/co

[jira] [Resolved] (KAFKA-1527) SimpleConsumer should be transaction-aware

[ https://issues.apache.org/jira/browse/KAFKA-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-1527. Resolution: Unresolved This work has been superseded by KIP-98: https://cwiki.apache.org/co

[jira] [Resolved] (KAFKA-1526) Producer performance tool should have an option to enable transactions

[ https://issues.apache.org/jira/browse/KAFKA-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-1526. Resolution: Unresolved This work has been superseded by KIP-98: https://cwiki.apache.org/co

[jira] [Resolved] (KAFKA-1601) ConsoleConsumer/SimpleConsumerPerformance should be transaction-aware

[ https://issues.apache.org/jira/browse/KAFKA-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-1601. Resolution: Unresolved This work has been superseded by KIP-98: https://cwiki.apache.org/co

[jira] [Resolved] (KAFKA-1541) Add transactional request definitions to clients package

[ https://issues.apache.org/jira/browse/KAFKA-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-1541. Resolution: Unresolved This work has been superseded by KIP-98: https://cwiki.apache.org/co

[jira] [Resolved] (KAFKA-1604) System Test for Transaction Management

[ https://issues.apache.org/jira/browse/KAFKA-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-1604. Resolution: Unresolved This work has been superseded by KIP-98: https://cwiki.apache.org/co

[jira] [Resolved] (KAFKA-1569) Tool for performance and correctness of transactions end-to-end

[ https://issues.apache.org/jira/browse/KAFKA-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-1569. Resolution: Unresolved This work has been superseded by KIP-98: https://cwiki.apache.org/co

Re: [DISCUSS] KIP-82 - Add Record Headers

So on mutable, and headers just at message level seems we're all agreed then. On Radai's comments. 1) agreed - kip updated. 2) Now i totally get the nasty code having this would create as noted by your example. And obviously we want an API which means boiler plate code and most common interact