Re: [DISCUSS] KIP-427: Add AtMinIsr topic partition category (new metric & TopicCommand option)

Hey Kevin, Thanks for the update. The KIP suggests that AtMinIsr is better than UnderReplicatedPartition as indicator for alerting. However, in most case where min_isr = replica_set_size - 1, these two metrics are exactly the same, where planned maintenance can easily cause positive AtMinIsr valu

Re: Speeding up integration tests

It's an idea that has come up before and worth exploring eventually. However, I'd first try to optimize the server startup/shutdown process. If we measure where the time is going, maybe some opportunities will present themselves. Ismael On Wed, Feb 27, 2019, 3:09 AM Viktor Somogyi-Vass wrote: >

[jira] [Resolved] (KAFKA-7962) StickyAssignor: throws NullPointerException during assignments if topic is deleted

[ https://issues.apache.org/jira/browse/KAFKA-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian resolved KAFKA-7962. Resolution: Fixed Reviewer: Vahid Hashemian Fix Version/s: 2.3.0 > StickyAs

[jira] [Created] (KAFKA-8014) Extend Connect integration tests to add and remove workers dynamically

Konstantine Karantasis created KAFKA-8014: - Summary: Extend Connect integration tests to add and remove workers dynamically Key: KAFKA-8014 URL: https://issues.apache.org/jira/browse/KAFKA-8014

Re: [DISCUSS] KIP-430 - Return Authorized Operations in Describe Responses

Hi All, While implementing KIP-430, we have added supportedOperations() method to kafka.security.auth.ResourceType public API. This will be used to maintain supported operations for a resourceType. Updated the KIP

Build failed in Jenkins: kafka-trunk-jdk8 #3422

See Changes: [rajinisivaram] Address flakiness of CustomQuotaCallbackTest#testCustomQuotaCallback -- [...truncated 4.63 MB...] kafka.zk.KafkaZkClientTest > testController

[jira] [Created] (KAFKA-8013) Avoid buffer underflow when reading a Struct from a partially correct buffer

Konstantine Karantasis created KAFKA-8013: - Summary: Avoid buffer underflow when reading a Struct from a partially correct buffer Key: KAFKA-8013 URL: https://issues.apache.org/jira/browse/KAFKA-8013

[jira] [Resolved] (KAFKA-7936) Flaky Test ConsumerBounceTest#testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup

[ https://issues.apache.org/jira/browse/KAFKA-7936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-7936. Resolution: Duplicate > Flaky Test > ConsumerBounceTest#testRollingBrokerRestartsWithSmall

Build failed in Jenkins: kafka-trunk-jdk11 #323

See Changes: [rajinisivaram] Address flakiness of CustomQuotaCallbackTest#testCustomQuotaCallback -- [...truncated 2.31 MB...] org.apache.kafka.connect.runtime.WorkerCon

Re: [DISCUSS] KIP-433: Provide client API version to authorizer

HI Colin, Overlooked the IDEMPOTENT_WRITE ACL. This along with client.min.version should solve the cases proposed in the KIP. Can we turn this KIP into adding min.client.version config to broker and it could be part of the dynamic config . Thanks, Harsha On Wed, Feb 27, 2019, at 12:

Re: [VOTE] KIP-307: Allow to define custom processor names with KStreams DSL

Florian, So we have re-vote and KIP-307 is excepted with the changes. I've looked in GitHub and I don't see the 5 separate PRs you mentioned on this thread. Can you push those PRs and ping us when they are ready for review? Also, this is an important KIP we'd like to get in sooner rather tha

Re: [DISCUSS] KIP-426: Persist Broker Id to Zookeeper

Hi Colin, What we want to is to preserve the broker.id so that we can do an offline rebuild of a broker. In our cases going through online Kafka replication to bring up, a failed node will put producer latencies at risk given the new broker will put all the other leaders busy with

Re: Speeding up integration tests

Hi Colin. << wrote: > On Wed, Feb 27, 2019, at 10:02, Ron Dagostino wrote: > > Hi everyone. Maybe providing the option to run it both ways -- start > your > > own cluster vs. using one that is pre-started -- might be useful? Don't > > know how it would work or if it would be useful, but it is s

Re: [DISCUSS] KIP-433: Provide client API version to authorizer

On Tue, Feb 26, 2019, at 16:33, Harsha wrote: > Hi Colin, > > "> I think Ismael and Gwen here bring up a good point. The version of the > > request is a technical detail that isn't really related to > > authorization. There are a lot of other technical details like this > > like the siz

Re: [DISCUSS] KIP-426: Persist Broker Id to Zookeeper

Hi Li, > The mechanism simplifies deployment because the same configuration can be > used across all brokers, however, in a large system where disk failure is > a norm, the meta file could often get lost, causing a new broker id being > allocated. This is problematic because new broker id h

Re: Speeding up integration tests

On Wed, Feb 27, 2019, at 10:02, Ron Dagostino wrote: > Hi everyone. Maybe providing the option to run it both ways -- start your > own cluster vs. using one that is pre-started -- might be useful? Don't > know how it would work or if it would be useful, but it is something to > think about. > >

Re: Speeding up integration tests

Hi everyone. Maybe providing the option to run it both ways -- start your own cluster vs. using one that is pre-started -- might be useful? Don't know how it would work or if it would be useful, but it is something to think about. Also, while the argument against using a pre-started cluster due

Re: contributors list

Done. On 2/27/19 1:21 AM, Jing Chen wrote: > hi there, > could anyone add me to the contributor list of kafka project, then I could > assign tickets to myself on jira. > My apache jira username is jingc. > > Thanks > Jing > signature.asc Description: OpenPGP digital signature

contributors list

hi there, could anyone add me to the contributor list of kafka project, then I could assign tickets to myself on jira. My apache jira username is jingc. Thanks Jing

[jira] [Created] (KAFKA-8012) NullPointerException while truncating at high watermark can crash replica fetcher thread

Colin Hicks created KAFKA-8012: -- Summary: NullPointerException while truncating at high watermark can crash replica fetcher thread Key: KAFKA-8012 URL: https://issues.apache.org/jira/browse/KAFKA-8012 Pr

Build failed in Jenkins: kafka-trunk-jdk8 #3421

See Changes: [bbejeck] KAFKA-7918: Inline generic parameters Pt. II: RocksDB Bytes Store and -- [...truncated 2.31 MB...] org.apache.kafka.streams.test.OutputVerifierTest

Re: [VOTE] KIP-307: Allow to define custom processor names with KStreams DSL

+1(binding) -Bill On 2019/02/27 02:01:41, "Matthias J. Sax" wrote: > +1 (binding) > > > -Matthias > > On 1/25/19 9:49 AM, Guozhang Wang wrote: > > Fair enough; could you summarize the open questions on the DISCUSS thread > > then? > > > > On Fri, Jan 25, 2019 at 9:46 AM Matthias J. Sax > >

Re: Speeding up integration tests

Hi, while I am also extremely annoyed at times by the amount of coffee I have to drink before tests finish I think the argument about flaky tests is valid! The current setup has the benefit that every test case runs on a pristine cluster, if we changed this we'd need to go through all tests and en

[jira] [Created] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest.

Bill Bejeck created KAFKA-8011: -- Summary: Flaky Test RegexSourceIntegrationTest. Key: KAFKA-8011 URL: https://issues.apache.org/jira/browse/KAFKA-8011 Project: Kafka Issue Type: Improvement

Re: Speeding up integration tests

Hey Viktor, I am all up for the idea of speeding up the tests. Running the `:core:integrationTest` command takes an absurd amount of time as is and is continuously going to go up if we don't do anything about it. Having said that, I am very scared that your proposal might significantly increase th

[jira] [Created] (KAFKA-8010) kafka-configs.sh does not allow setting config with an equal in the value

Mickael Maison created KAFKA-8010: - Summary: kafka-configs.sh does not allow setting config with an equal in the value Key: KAFKA-8010 URL: https://issues.apache.org/jira/browse/KAFKA-8010 Project: Ka

Re: [VOTE] KIP-430 - Return Authorized Operations in Describe Responses

The vote has passed with 5 binding votes (Harsha, Gwen, Manikumar, Colin, me) and 2 non-binding votes (Satish, Mickael). I will update the KIP page. Many thanks to everyone for the feedback and votes. Regards, Rajini On Tue, Feb 26, 2019 at 6:41 PM Rajini Sivaram wrote: > Thanks Colin, I hav

Speeding up integration tests

Hi Folks, I've been observing lately that unit tests usually take 2.5 hours to run and a very big portion of these are the core tests where a new cluster is spun up for every test. This takes most of the time. I ran a test (TopicCommandWithAdminClient with 38 test inside) through the profiler and

Re: [VOTE] 2.2.0 RC0

+1 (non-binding) - Ran testAll/releaseTarGzAll successfully with NO failures. - Ran through quickstart of core/streams on builds generated from 2.2.0-rc0 tag - Ran few internal apps targeting to topics on 3 node cluster. Thanks for running the release Matthias! On Tue, Feb 26, 2019 at 8:17 PM Ad

Jenkins build is back to normal : kafka-2.0-jdk8 #231

See

[jira] [Created] (KAFKA-8009) Uppgrade Jenkins job Gradle version from 4.10.2 to 5.x

Dejan Stojadinović created KAFKA-8009: - Summary: Uppgrade Jenkins job Gradle version from 4.10.2 to 5.x Key: KAFKA-8009 URL: https://issues.apache.org/jira/browse/KAFKA-8009 Project: Kafka

Jenkins build is back to normal : kafka-1.0-jdk7 #260

See

[jira] [Created] (KAFKA-8008) Clients unable to connect and replicas are not able to connect to each other

Abhi created KAFKA-8008: --- Summary: Clients unable to connect and replicas are not able to connect to each other Key: KAFKA-8008 URL: https://issues.apache.org/jira/browse/KAFKA-8008 Project: Kafka Iss