[jira] [Updated] (KAFKA-10434) Remove deprecated methods on WindowStore

2020-08-29 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Esteban Quilcate Otoya updated KAFKA-10434: - Labels: needs-kip (was: kip-required) > Remove deprecated metho

[GitHub] [kafka] chia7712 commented on pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on pull request #8657: URL: https://github.com/apache/kafka/pull/8657#issuecomment-683284825 > If it's meant to be called by every request, then maybe we should have the delayed actions in a separate class instead of ReplicaManager. Other classes could, in theory, add th

[GitHub] [kafka] jeqo opened a new pull request #9228: KAFKA-10445: Align IQ SessionStore API with Instant-based methods as ReadOnlyWindowStore

2020-08-29 Thread GitBox
jeqo opened a new pull request #9228: URL: https://github.com/apache/kafka/pull/9228 [KIP-666](https://cwiki.apache.org/confluence/display/KAFKA/KIP-666%3A+Add+Instant-based+methods+to+ReadOnlySessionStore) proposal ### Committer Checklist (excluded from commit message) - [ ] Ver

[GitHub] [kafka] junrao commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
junrao commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479661386 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,48 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479677889 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -180,6 +181,11 @@ class KafkaApis(val requestChannel: RequestChannel, case Ap

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479678653 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -585,6 +591,23 @@ class ReplicaManager(val config: KafkaConfig,

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479678761 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -562,6 +564,10 @@ class ReplicaManager(val config: KafkaConfig, * Append message

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479678812 ## File path: core/src/main/scala/kafka/server/KafkaServer.scala ## @@ -134,6 +134,8 @@ class KafkaServer(val config: KafkaConfig, time: Time = Time.SYSTEM,

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479679930 ## File path: core/src/main/scala/kafka/server/DelayedOperation.scala ## @@ -100,41 +99,22 @@ abstract class DelayedOperation(override val delayMs: Long,

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479680637 ## File path: core/src/main/scala/kafka/server/KafkaServer.scala ## @@ -134,6 +134,8 @@ class KafkaServer(val config: KafkaConfig, time: Time = Time.SYSTE

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479683036 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -562,6 +564,10 @@ class ReplicaManager(val config: KafkaConfig, * Append mess

[GitHub] [kafka] junrao commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
junrao commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479684163 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -85,6 +85,8 @@ object LogAppendInfo { * @param validBytes The number of valid bytes * @param

[GitHub] [kafka] ijuma opened a new pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
ijuma opened a new pull request #9229: URL: https://github.com/apache/kafka/pull/9229 Use a caching BufferSupplier per request handler thread so that decompression buffers are cached if supported by the underlying CompressionType. This reduces allocations significantly for LZ4 when the

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479686292 ## File path: core/src/main/scala/kafka/server/KafkaServer.scala ## @@ -134,6 +134,8 @@ class KafkaServer(val config: KafkaConfig, time: Time = Time.SYSTEM,

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479686398 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -85,6 +85,8 @@ object LogAppendInfo { * @param validBytes The number of valid bytes * @param o

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479686517 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -85,6 +85,8 @@ object LogAppendInfo { * @param validBytes The number of valid bytes * @para

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689381 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689457 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -37,7 +37,10 @@ class ActionQueue { * picks up an action to complete. */ def

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689653 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or m

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689664 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689694 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -37,7 +37,10 @@ class ActionQueue { * picks up an action to complete. */

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689838 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479689694 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -37,7 +37,10 @@ class ActionQueue { * picks up an action to complete. */

[GitHub] [kafka] ijuma commented on pull request #9220: KAFKA-10433 Reuse the ByteBuffer in validating compressed records

2020-08-29 Thread GitBox
ijuma commented on pull request #9220: URL: https://github.com/apache/kafka/pull/9220#issuecomment-683342367 @chia7712 here's what I had in mind: https://github.com/apache/kafka/pull/9229 What do you think? This is

[GitHub] [kafka] ijuma commented on a change in pull request #9223: KAFKA-10438 Lazy initialization of record header to reduce memory usa…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #9223: URL: https://github.com/apache/kafka/pull/9223#discussion_r479690701 ## File path: clients/src/test/java/org/apache/kafka/common/record/DefaultRecordTest.java ## @@ -270,7 +270,7 @@ public void testInvalidNumHeadersPartial() t

[GitHub] [kafka] ijuma commented on pull request #9223: KAFKA-10438 Lazy initialization of record header to reduce memory usa…

2020-08-29 Thread GitBox
ijuma commented on pull request #9223: URL: https://github.com/apache/kafka/pull/9223#issuecomment-683342810 Good change. Any thoughts on test for this? This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479691180 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or m

[GitHub] [kafka] chia7712 commented on pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
chia7712 commented on pull request #9229: URL: https://github.com/apache/kafka/pull/9229#issuecomment-683343524 This patch makes each request (handler) thread have a ```BufferSupplier``` to simplify concurrency handling (by contrast, #9220 offers a thread-safe BufferSupplier). This

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479692892 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479692892 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] chia7712 commented on pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
chia7712 commented on pull request #9229: URL: https://github.com/apache/kafka/pull/9229#issuecomment-683347067 Could we use ThreadLocal to keep those thread resources, like BufferSupplier and ActionQueue, to simplify the method arguments? The cost of ThreadLocal is low and it is easy to a

[GitHub] [kafka] ijuma commented on pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
ijuma commented on pull request #9229: URL: https://github.com/apache/kafka/pull/9229#issuecomment-683352094 In my opinion, thread locals are most useful when one doesn't control the code. For cases like this, being explicit makes it easier to reason about and also test. Even if it's a bit

[GitHub] [kafka] ijuma commented on pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
ijuma commented on pull request #9229: URL: https://github.com/apache/kafka/pull/9229#issuecomment-683352558 @chia7712 One option would be for me to introduce a `RequestContext` case class and add the `BufferSupplier` as one of the fields. It would be easy to extend this class with request

[GitHub] [kafka] junrao commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
junrao commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479706303 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -37,7 +37,10 @@ class ActionQueue { * picks up an action to complete. */ de

[GitHub] [kafka] chia7712 commented on pull request #9229: MINOR: Reduce allocations in requests via buffer caching

2020-08-29 Thread GitBox
chia7712 commented on pull request #9229: URL: https://github.com/apache/kafka/pull/9229#issuecomment-683369221 > One option would be for me to introduce a RequestContext case class and add the BufferSupplier as one of the fields. It would be easy to extend this class with request bound el

[GitHub] [kafka] ijuma commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-08-29 Thread GitBox
ijuma commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r479716882 ## File path: core/src/main/scala/kafka/server/ActionQueue.scala ## @@ -0,0 +1,46 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or m

[GitHub] [kafka] dielhennr edited a comment on pull request #9101: KAFKA-10325: KIP-649 implementation

2020-08-29 Thread GitBox
dielhennr edited a comment on pull request #9101: URL: https://github.com/apache/kafka/pull/9101#issuecomment-683076305 Hey @jsancio , I added some work in progress to this branch including new APIs for this feature and functionality using them. Fitting user and client-id into the

[GitHub] [kafka] dielhennr edited a comment on pull request #9101: KAFKA-10325: KIP-649 implementation

2020-08-29 Thread GitBox
dielhennr edited a comment on pull request #9101: URL: https://github.com/apache/kafka/pull/9101#issuecomment-683076305 Hey @jsancio , I added some work in progress to this branch including new APIs for this feature and functionality using them. Fitting user and client-id into the

[GitHub] [kafka] dielhennr edited a comment on pull request #9101: KAFKA-10325: KIP-649 implementation

2020-08-29 Thread GitBox
dielhennr edited a comment on pull request #9101: URL: https://github.com/apache/kafka/pull/9101#issuecomment-683076305 Hey @jsancio , I added some work in progress to this branch including new APIs for this feature and functionality using them. Fitting user and client-id into the