[GitHub] [kafka] cmccabe commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-28 Thread GitBox
cmccabe commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r479548121 ## File path: core/src/test/scala/unit/kafka/server/ClientQuotasRequestTest.scala ## @@ -17,6 +17,8 @@ package kafka.server +import java.util Review c

[GitHub] [kafka] cmccabe commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-28 Thread GitBox
cmccabe commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r479548642 ## File path: core/src/test/scala/unit/kafka/server/ClientQuotasRequestTest.scala ## @@ -166,14 +168,11 @@ class ClientQuotasRequestTest extends BaseRequest

[GitHub] [kafka] cmccabe commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-28 Thread GitBox
cmccabe commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r479551061 ## File path: core/src/test/scala/unit/kafka/admin/UserScramCredentialsCommandTest.scala ## @@ -0,0 +1,135 @@ +/** + * Licensed to the Apache Software Foun

[GitHub] [kafka] cmccabe commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-28 Thread GitBox
cmccabe commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r479552810 ## File path: core/src/test/scala/integration/kafka/server/DynamicBrokerReconfigurationTest.scala ## @@ -1047,8 +1047,8 @@ class DynamicBrokerReconfigurati

[GitHub] [kafka] cmccabe commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-28 Thread GitBox
cmccabe commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r479553827 ## File path: core/src/test/scala/integration/kafka/api/SaslScramSslEndToEndAuthorizationTest.scala ## @@ -42,7 +42,18 @@ class SaslScramSslEndToEndAuthori

[GitHub] [kafka] cmccabe commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-28 Thread GitBox
cmccabe commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r479554324 ## File path: core/src/test/scala/integration/kafka/api/SaslClientsWithInvalidCredentialsTest.scala ## @@ -248,4 +250,25 @@ class SaslClientsWithInvalidCre

[GitHub] [kafka] cmccabe commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-28 Thread GitBox
cmccabe commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r479554911 ## File path: core/src/test/scala/integration/kafka/api/EndToEndAuthorizationTest.scala ## @@ -545,6 +558,16 @@ abstract class EndToEndAuthorizationTest ex

[GitHub] [kafka] chia7712 commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-683284825 > If it's meant to be called by every request, then maybe we should have the delayed actions in a separate class instead of ReplicaManager. Other classes could, in theory, add th

[GitHub] [kafka] jeqo opened a new pull request #9228: KAFKA-10445: Align IQ SessionStore API with Instant-based methods as ReadOnlyWindowStore

2020-08-29 Thread GitBox
jeqo opened a new pull request #9228: URL: https://github.com/apache/kafka/pull/9228 [KIP-666](https://cwiki.apache.org/confluence/display/KAFKA/KIP-666%3A+Add+Instant-based+methods+to+ReadOnlySessionStore) proposal ### Committer Checklist (excluded from commit message) - [ ] Ver

[GitHub] [kafka] junrao commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
junrao commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479661386 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,48 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479677889 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -180,6 +181,11 @@ class KafkaApis(val requestChannel: RequestChannel, case Ap

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479678653 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -585,6 +591,23 @@ class ReplicaManager(val config: KafkaConfig,

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479678761 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -562,6 +564,10 @@ class ReplicaManager(val config: KafkaConfig, * Append message

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479678812 ## File path: core/src/main/scala/kafka/server/KafkaServer.scala ## @@ -134,6 +134,8 @@ class KafkaServer(val config: KafkaConfig, time: Time = Time.SYSTEM,

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479679930 ## File path: core/src/main/scala/kafka/server/DelayedOperation.scala ## @@ -100,41 +99,22 @@ abstract class DelayedOperation(override val delayMs: Long,

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479680637 ## File path: core/src/main/scala/kafka/server/KafkaServer.scala ## @@ -134,6 +134,8 @@ class KafkaServer(val config: KafkaConfig, time: Time = Time.SYSTE

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479683036 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -562,6 +564,10 @@ class ReplicaManager(val config: KafkaConfig, * Append mess

[GitHub] [kafka] junrao commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
junrao commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479684163 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -85,6 +85,8 @@ object LogAppendInfo { * @param validBytes The number of valid bytes * @param

[GitHub] [kafka] ijuma opened a new pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
ijuma opened a new pull request #9229: URL: https://github.com/apache/kafka/pull/9229 Use a caching BufferSupplier per request handler thread so that decompression buffers are cached if supported by the underlying CompressionType. This reduces allocations significantly for LZ4 when the

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479686292 ## File path: core/src/main/scala/kafka/server/KafkaServer.scala ## @@ -134,6 +134,8 @@ class KafkaServer(val config: KafkaConfig, time: Time = Time.SYSTEM,

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479686398 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -85,6 +85,8 @@ object LogAppendInfo { * @param validBytes The number of valid bytes * @param o

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479686517 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -85,6 +85,8 @@ object LogAppendInfo { * @param validBytes The number of valid bytes * @para

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689381 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689457 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -37,7 +37,10 @@ class ActionQueue { * picks up an action to complete. */ def

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689653 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or m

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689664 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689694 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -37,7 +37,10 @@ class ActionQueue { * picks up an action to complete. */

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689838 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689694 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -37,7 +37,10 @@ class ActionQueue { * picks up an action to complete. */

[GitHub] [kafka] ijuma commented on pull request #9220: KAFKA-10433 Reuse the ByteBuffer in validating compressed records

2020-08-29 Thread GitBox
ijuma commented on pull request #9220: URL: https://github.com/apache/kafka/pull/9220#issuecomment-683342367 @chia7712 here's what I had in mind: https://github.com/apache/kafka/pull/9229 What do you think? This is

[GitHub] [kafka] ijuma commented on a change in pull request #9223: KAFKA-10438 Lazy initialization of record header to reduce memory usa…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #9223: URL: https://github.com/apache/kafka/pull/9223#discussion_r479690701 ## File path: clients/src/test/java/org/apache/kafka/common/record/DefaultRecordTest.java ## @@ -270,7 +270,7 @@ public void testInvalidNumHeadersPartial() t

[GitHub] [kafka] ijuma commented on pull request #9223: KAFKA-10438 Lazy initialization of record header to reduce memory usa…

2020-08-29 Thread GitBox
ijuma commented on pull request #9223: URL: https://github.com/apache/kafka/pull/9223#issuecomment-683342810 Good change. Any thoughts on test for this? This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479691180 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or m

[GitHub] [kafka] chia7712 commented on pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
chia7712 commented on pull request #9229: URL: https://github.com/apache/kafka/pull/9229#issuecomment-683343524 This patch makes each request (handler) thread have a ```BufferSupplier``` to simplify concurrency handling (by contrast, #9220 offers a thread-safe BufferSupplier). This

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479692892 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479692892 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] chia7712 commented on pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
chia7712 commented on pull request #9229: URL: https://github.com/apache/kafka/pull/9229#issuecomment-683347067 Could we use ThreadLocal to keep those thread resources, like BufferSupplier and ActionQueue, to simplify the method arguments? The cost of ThreadLocal is low and it is easy to a

[GitHub] [kafka] ijuma commented on pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
ijuma commented on pull request #9229: URL: https://github.com/apache/kafka/pull/9229#issuecomment-683352094 In my opinion, thread locals are most useful when one doesn't control the code. For cases like this, being explicit makes it easier to reason about and also test. Even if it's a bit

[GitHub] [kafka] ijuma commented on pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
ijuma commented on pull request #9229: URL: https://github.com/apache/kafka/pull/9229#issuecomment-683352558 @chia7712 One option would be for me to introduce a `RequestContext` case class and add the `BufferSupplier` as one of the fields. It would be easy to extend this class with request

[GitHub] [kafka] junrao commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
junrao commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479706303 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -37,7 +37,10 @@ class ActionQueue { * picks up an action to complete. */ de

[GitHub] [kafka] chia7712 commented on pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
chia7712 commented on pull request #9229: URL: https://github.com/apache/kafka/pull/9229#issuecomment-683369221 > One option would be for me to introduce a RequestContext case class and add the BufferSupplier as one of the fields. It would be easy to extend this class with request bound el

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479716882 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or m

[GitHub] [kafka] dielhennr edited a comment on pull request #9101: KAFKA-10325: KIP-649 implementation

2020-08-29 Thread GitBox
dielhennr edited a comment on pull request #9101: URL: https://github.com/apache/kafka/pull/9101#issuecomment-683076305 Hey @jsancio , I added some work in progress to this branch including new APIs for this feature and functionality using them. Fitting user and client-id into the

[GitHub] [kafka] dielhennr edited a comment on pull request #9101: KAFKA-10325: KIP-649 implementation

2020-08-29 Thread GitBox
dielhennr edited a comment on pull request #9101: URL: https://github.com/apache/kafka/pull/9101#issuecomment-683076305 Hey @jsancio , I added some work in progress to this branch including new APIs for this feature and functionality using them. Fitting user and client-id into the

[GitHub] [kafka] dielhennr edited a comment on pull request #9101: KAFKA-10325: KIP-649 implementation

2020-08-29 Thread GitBox
dielhennr edited a comment on pull request #9101: URL: https://github.com/apache/kafka/pull/9101#issuecomment-683076305 Hey @jsancio , I added some work in progress to this branch including new APIs for this feature and functionality using them. Fitting user and client-id into the

[GitHub] [kafka] dielhennr edited a comment on pull request #9101: KAFKA-10325: KIP-649 implementation

2020-08-30 Thread GitBox
dielhennr edited a comment on pull request #9101: URL: https://github.com/apache/kafka/pull/9101#issuecomment-683076305 Hey @jsancio , I added some work in progress to this branch including new APIs for this feature. Fitting user and client-id into the `DescribeConfigs` API was awkw

[GitHub] [kafka] dielhennr edited a comment on pull request #9101: KAFKA-10325: KIP-649 implementation

2020-08-30 Thread GitBox
dielhennr edited a comment on pull request #9101: URL: https://github.com/apache/kafka/pull/9101#issuecomment-683076305 Hey @jsancio , I added some work in progress to this branch including new APIs for this feature. Fitting user and client-id into the `DescribeConfigs` API was awkw

[GitHub] [kafka] dielhennr edited a comment on pull request #9101: KAFKA-10325: KIP-649 implementation

2020-08-30 Thread GitBox
dielhennr edited a comment on pull request #9101: URL: https://github.com/apache/kafka/pull/9101#issuecomment-683076305 Hey @jsancio , I added some work in progress to this branch including new APIs for this feature. Fitting user and client-id into the `DescribeConfigs` API was awkw

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-30 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479737358 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] chia7712 closed pull request #9220: KAFKA-10433 Reuse the ByteBuffer in validating compressed records

2020-08-30 Thread GitBox
chia7712 closed pull request #9220: URL: https://github.com/apache/kafka/pull/9220 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] chia7712 commented on pull request #9220: KAFKA-10433 Reuse the ByteBuffer in validating compressed records

2020-08-30 Thread GitBox
chia7712 commented on pull request #9220: URL: https://github.com/apache/kafka/pull/9220#issuecomment-683400353 close as there is a better approach (#9229) This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] chia7712 commented on pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-30 Thread GitBox
chia7712 commented on pull request #9229: URL: https://github.com/apache/kafka/pull/9229#issuecomment-683400436 @ijuma I have closed #9220 and assign https://issues.apache.org/jira/browse/KAFKA-10433 to you. This is an autom

[GitHub] [kafka] chia7712 commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-30 Thread GitBox
chia7712 commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-683400677 > Could you do some perf tests so that we know the performance of fetch requests doesn't change noticeably? @junrao Are there any suggested official benchmark tools?

[GitHub] [kafka] chia7712 commented on a change in pull request #9223: KAFKA-10438 Lazy initialization of record header to reduce memory usa…

2020-08-30 Thread GitBox
chia7712 commented on a change in pull request #9223: URL: https://github.com/apache/kafka/pull/9223#discussion_r479750019 ## File path: clients/src/test/java/org/apache/kafka/common/record/DefaultRecordTest.java ## @@ -270,7 +270,7 @@ public void testInvalidNumHeadersPartial(

[GitHub] [kafka] soarez edited a comment on pull request #9064: KAFKA-10205: Documentation and handling of non deterministic Topologies

2020-08-30 Thread GitBox
soarez edited a comment on pull request #9064: URL: https://github.com/apache/kafka/pull/9064#issuecomment-683421515 @mjsax rebased and fixed the error, now `./gradlew :streams:unitTest` seems to run without errors over here. --

[GitHub] [kafka] soarez commented on pull request #9064: KAFKA-10205: Documentation and handling of non deterministic Topologies

2020-08-30 Thread GitBox
soarez commented on pull request #9064: URL: https://github.com/apache/kafka/pull/9064#issuecomment-683421515 @mjsax rebased and fix the error, now `./gradlew :streams:unitTest` seems to run without errors over here. This i

[GitHub] [kafka] chia7712 commented on pull request #9223: KAFKA-10438 Lazy initialization of record header to reduce memory usa…

2020-08-30 Thread GitBox
chia7712 commented on pull request #9223: URL: https://github.com/apache/kafka/pull/9223#issuecomment-683425417 > Any thoughts on test for this? The JMH is attached. This is an automated message from the Apache Git Ser

[GitHub] [kafka] aakashgupta96 commented on pull request #5068: KAFKA-6886 Externalize secrets from Connect configs

2020-08-30 Thread GitBox
aakashgupta96 commented on pull request #5068: URL: https://github.com/apache/kafka/pull/5068#issuecomment-683433765 Hi @rayokota Was this feature merged? I was also planning to submit a KIP around same then I saw your proposal. Can you please update whether it is merged or are you

[GitHub] [kafka] chia7712 commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-30 Thread GitBox
chia7712 commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-683434648 > Could you do some perf tests so that we know the performance of fetch requests doesn't change noticeably? the result of ```Benchmark.test_producer_and_consumer``` is atta

[GitHub] [kafka] chia7712 opened a new pull request #9230: KAFKA-10446 add lz4 and zstd to compression type of benchmark_test.py

2020-08-30 Thread GitBox
chia7712 opened a new pull request #9230: URL: https://github.com/apache/kafka/pull/9230 issue: https://issues.apache.org/jira/browse/KAFKA-10446 Both "lz4" and "zstd" are popular and important compressions supported by kafka. They are worth being benchmark. ### Committer Chec

[GitHub] [kafka] rayokota commented on pull request #5068: KAFKA-6886 Externalize secrets from Connect configs

2020-08-30 Thread GitBox
rayokota commented on pull request #5068: URL: https://github.com/apache/kafka/pull/5068#issuecomment-683443295 @aakashgupta96 yes it was merged This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [kafka] omkreddy commented on pull request #9062: KAFKA-8098: fix the flaky test by disabling the auto commit to avoid member rejoining

2020-08-30 Thread GitBox
omkreddy commented on pull request #9062: URL: https://github.com/apache/kafka/pull/9062#issuecomment-683453793 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] junrao commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-30 Thread GitBox
junrao commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479802860 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -85,6 +92,9 @@ object LogAppendInfo { * @param validBytes The number of valid bytes * @param

[GitHub] [kafka] ijuma opened a new pull request #9231: KAFKA-10447: Migrate tools module to JUnit 5

2020-08-30 Thread GitBox
ijuma opened a new pull request #9231: URL: https://github.com/apache/kafka/pull/9231 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade n

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-30 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479829275 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -68,6 +68,13 @@ object LogAppendInfo { offsetsMonotonic = false, -1L, recordErrors, errorMe

[GitHub] [kafka] soarez commented on a change in pull request #9000: KAFKA-10036 Improve handling and documentation of Suppliers

2020-08-30 Thread GitBox
soarez commented on a change in pull request #9000: URL: https://github.com/apache/kafka/pull/9000#discussion_r479832011 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamUtil.java ## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] soarez commented on pull request #9000: KAFKA-10036 Improve handling and documentation of Suppliers

2020-08-30 Thread GitBox
soarez commented on pull request #9000: URL: https://github.com/apache/kafka/pull/9000#issuecomment-683488002 Squashed and rebased as there were conflicts. Please take another look @mjsax . This is an automated message from

[GitHub] [kafka] LMnet commented on a change in pull request #8955: KAFKA-10020: Create a new version of a scala Serdes without name clash (KIP-616)

2020-08-30 Thread GitBox
LMnet commented on a change in pull request #8955: URL: https://github.com/apache/kafka/pull/8955#discussion_r479877121 ## File path: streams/streams-scala/src/main/scala/org/apache/kafka/streams/scala/serialization/Serdes.scala ## @@ -0,0 +1,93 @@ +/* + * Copyright (C) 2018 L

[GitHub] [kafka] chia7712 commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-31 Thread GitBox
chia7712 commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-683602732 @junrao the result of ```benchmark_test.py``` is attached (see description) The main regression ({"records_per_sec": 3653635.3672, "mb_per_sec": 348.4378} -> {"records_per_

[GitHub] [kafka] cadonna commented on pull request #9177: KAFKA-9924: Add RocksDB metric num-entries-active-mem-table

2020-08-31 Thread GitBox
cadonna commented on pull request #9177: URL: https://github.com/apache/kafka/pull/9177#issuecomment-683631452 @vvcephei Thank you very much for taking care of the conflicts and merging the PR! This is an automated message f

[GitHub] [kafka] cadonna commented on pull request #9191: KAFKA-10355: Throw error when source topic was deleted

2020-08-31 Thread GitBox
cadonna commented on pull request #9191: URL: https://github.com/apache/kafka/pull/9191#issuecomment-683673488 This is the implementation for KIP-662. This is an automated message from the Apache Git Service. To respond to th

[GitHub] [kafka] ijuma commented on a change in pull request #9231: KAFKA-10447: Migrate tools module to JUnit 5 and mockito

2020-08-31 Thread GitBox
ijuma commented on a change in pull request #9231: URL: https://github.com/apache/kafka/pull/9231#discussion_r480023644 ## File path: tools/src/test/java/org/apache/kafka/trogdor/coordinator/CoordinatorTest.java ## @@ -62,17 +62,18 @@ import java.util.List; import java.util.

[GitHub] [kafka] cadonna opened a new pull request #9232: KAFKA-9924: Add remaining property-based RocksDB metrics as described in KIP-607

2020-08-31 Thread GitBox
cadonna opened a new pull request #9232: URL: https://github.com/apache/kafka/pull/9232 This commit adds the remaining property-based RocksDB metrics as described in KIP-607, except for num-entries-active-mem-table, which was added in PR #9177. ### Committer Checklist (excluded from

[GitHub] [kafka] cadonna commented on pull request #9232: KAFKA-9924: Add remaining property-based RocksDB metrics as described in KIP-607

2020-08-31 Thread GitBox
cadonna commented on pull request #9232: URL: https://github.com/apache/kafka/pull/9232#issuecomment-683729711 Call for review: @guozhangwang @vvcephei This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] rondagostino opened a new pull request #9233: Testing

2020-08-31 Thread GitBox
rondagostino opened a new pull request #9233: URL: https://github.com/apache/kafka/pull/9233 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of test

[GitHub] [kafka] rondagostino closed pull request #9233: Testing

2020-08-31 Thread GitBox
rondagostino closed pull request #9233: URL: https://github.com/apache/kafka/pull/9233 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] jeqo commented on pull request #9138: KAFKA-9929: Support backward iterator on WindowStore

2020-08-31 Thread GitBox
jeqo commented on pull request #9138: URL: https://github.com/apache/kafka/pull/9138#issuecomment-683830757 @ableegoldman key ordering is added to `InMemoryWindowStore` now This is an automated message from the Apache Git Ser

[GitHub] [kafka] lct45 commented on a change in pull request #9157: Update for KIP-450 to handle early records

2020-08-31 Thread GitBox
lct45 commented on a change in pull request #9157: URL: https://github.com/apache/kafka/pull/9157#discussion_r480194412 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamSlidingWindowAggregate.java ## @@ -0,0 +1,386 @@ +/* + * Licensed to th

[GitHub] [kafka] junrao commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-31 Thread GitBox
junrao commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-683859366 @chia7712 : Thanks for the performance results. It seems that the average across multiple runs doesn't change much? Also, 1 failure in the latest system test run. http://

[GitHub] [kafka] chia7712 commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-31 Thread GitBox
chia7712 commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-683863356 > It seems that the average across multiple runs doesn't change much? yep. I didn't observe obvious regression caused by this patch. > Also, 1 failure in the latest s

[GitHub] [kafka] asdaraujo commented on a change in pull request #8730: KAFKA-10048: Possible data gap for a consumer after a failover when u…

2020-08-31 Thread GitBox
asdaraujo commented on a change in pull request #8730: URL: https://github.com/apache/kafka/pull/8730#discussion_r480265451 ## File path: connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorConnectorsIntegrationTest.java ## @@ -367,14 +406,37 @@ public void test

[GitHub] [kafka] asdaraujo commented on a change in pull request #8730: KAFKA-10048: Possible data gap for a consumer after a failover when u…

2020-08-31 Thread GitBox
asdaraujo commented on a change in pull request #8730: URL: https://github.com/apache/kafka/pull/8730#discussion_r480266459 ## File path: connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorConnectorsIntegrationTest.java ## @@ -128,10 +136,23 @@ public void setu

[GitHub] [kafka] asdaraujo commented on a change in pull request #8730: KAFKA-10048: Possible data gap for a consumer after a failover when u…

2020-08-31 Thread GitBox
asdaraujo commented on a change in pull request #8730: URL: https://github.com/apache/kafka/pull/8730#discussion_r480267122 ## File path: connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorConnectorsIntegrationTest.java ## @@ -128,10 +136,23 @@ public void setu

[GitHub] [kafka] asdaraujo commented on a change in pull request #8730: KAFKA-10048: Possible data gap for a consumer after a failover when u…

2020-08-31 Thread GitBox
asdaraujo commented on a change in pull request #8730: URL: https://github.com/apache/kafka/pull/8730#discussion_r480267347 ## File path: connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorConnectorsIntegrationTest.java ## @@ -190,24 +211,19 @@ public void clos

[GitHub] [kafka] asdaraujo commented on pull request #8730: KAFKA-10048: Possible data gap for a consumer after a failover when u…

2020-08-31 Thread GitBox
asdaraujo commented on pull request #8730: URL: https://github.com/apache/kafka/pull/8730#issuecomment-683909598 @mimaison Thanks for the feedback. I've refactored the tests. Could you please give it another review. This is

[GitHub] [kafka] rondagostino commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-31 Thread GitBox
rondagostino commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r480272048 ## File path: tests/kafkatest/services/kafka/kafka.py ## @@ -575,10 +577,21 @@ def set_unclean_leader_election(self, topic, value=True, node=None):

[GitHub] [kafka] ableegoldman commented on pull request #9039: KAFKA-5636: SlidingWindows (KIP-450)

2020-08-31 Thread GitBox
ableegoldman commented on pull request #9039: URL: https://github.com/apache/kafka/pull/9039#issuecomment-683916587 test this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] ableegoldman commented on pull request #9039: KAFKA-5636: SlidingWindows (KIP-450)

2020-08-31 Thread GitBox
ableegoldman commented on pull request #9039: URL: https://github.com/apache/kafka/pull/9039#issuecomment-683917229 Retest this please This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [kafka] rondagostino commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-31 Thread GitBox
rondagostino commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r480282139 ## File path: core/src/test/scala/unit/kafka/utils/JaasTestUtils.scala ## @@ -169,6 +169,18 @@ object JaasTestUtils { jaasFile } + // Return

[GitHub] [kafka] rondagostino commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-31 Thread GitBox
rondagostino commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r480301973 ## File path: core/src/test/scala/unit/kafka/admin/UserScramCredentialsCommandTest.scala ## @@ -0,0 +1,135 @@ +/** + * Licensed to the Apache Software

[GitHub] [kafka] rondagostino commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-31 Thread GitBox
rondagostino commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r480324560 ## File path: core/src/test/scala/integration/kafka/server/DynamicBrokerReconfigurationTest.scala ## @@ -1047,8 +1047,8 @@ class DynamicBrokerReconfig

[GitHub] [kafka] rondagostino commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-31 Thread GitBox
rondagostino commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r480329570 ## File path: core/src/test/scala/integration/kafka/api/SaslScramSslEndToEndAuthorizationTest.scala ## @@ -42,7 +42,18 @@ class SaslScramSslEndToEndAu

[GitHub] [kafka] rondagostino commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-31 Thread GitBox
rondagostino commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r480370634 ## File path: core/src/test/scala/integration/kafka/api/EndToEndAuthorizationTest.scala ## @@ -545,6 +558,16 @@ abstract class EndToEndAuthorizationTe

[GitHub] [kafka] rondagostino commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-31 Thread GitBox
rondagostino commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r480370463 ## File path: core/src/test/scala/integration/kafka/api/SaslClientsWithInvalidCredentialsTest.scala ## @@ -248,4 +250,25 @@ class SaslClientsWithInval

[GitHub] [kafka] ijuma commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing

2020-08-31 Thread GitBox
ijuma commented on a change in pull request #9226: URL: https://github.com/apache/kafka/pull/9226#discussion_r480364696 ## File path: Jenkinsfile ## @@ -0,0 +1,200 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreem

[GitHub] [kafka] mumrah commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing

2020-08-31 Thread GitBox
mumrah commented on a change in pull request #9226: URL: https://github.com/apache/kafka/pull/9226#discussion_r480388916 ## File path: Jenkinsfile ## @@ -0,0 +1,200 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agree

[GitHub] [kafka] mumrah commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing

2020-08-31 Thread GitBox
mumrah commented on a change in pull request #9226: URL: https://github.com/apache/kafka/pull/9226#discussion_r480391239 ## File path: Jenkinsfile ## @@ -0,0 +1,200 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agree

[GitHub] [kafka] lbradstreet commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing

2020-08-31 Thread GitBox
lbradstreet commented on a change in pull request #9226: URL: https://github.com/apache/kafka/pull/9226#discussion_r480392178 ## File path: Jenkinsfile ## @@ -0,0 +1,200 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license

[GitHub] [kafka] mumrah commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing

2020-08-31 Thread GitBox
mumrah commented on a change in pull request #9226: URL: https://github.com/apache/kafka/pull/9226#discussion_r480393840 ## File path: Jenkinsfile ## @@ -0,0 +1,200 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agree

[GitHub] [kafka] vvcephei opened a new pull request #9234: MINOR: Record all poll invocations

2020-08-31 Thread GitBox
vvcephei opened a new pull request #9234: URL: https://github.com/apache/kafka/pull/9234 Record the `pollSensor` after every invocation to poll, rather than just when we get records back so that we can accurately gauge how often we're invoking Consumer#poll. ### Committer Checklist

<    14   15   16   17   18   19   20   21   22   23   >