[
https://issues.apache.org/jira/browse/KAFKA-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kowshik Prakasam updated KAFKA-13701:
-------------------------------------
Description:
Certain background work such as UnifiedLog.flush() need not support
concurrency. Today in the existing KafkaScheduler, we are not able to pin
background work to specific threads. As a result we are unable to prevent
concurrent UnifiedLog.flush() calls, so we have to ensure UnifiedLog.flush()
implementation is thread safe by modifying the code at subtle areas (ex: [PR
#11814|https://github.com/apache/kafka/pull/11814]). The code will be simpler
if instead KafkaScheduler (or alike) could support pinning of certain
background work to specific threads, for example the UnifiedLog.flush()
operation for the same topic-partition will go to the same thread. This will
ensure strict ordering of flush() calls, thereby enabling us to write simpler
code eventually.
Subsequently, in this Jira we will modify the code to schedule
UnifiedLog.flush() only to specific background threads always (based on
topic-partition).
was:Certain background work such as UnifiedLog.flush() need not support
concurrency. Today in the existing KafkaScheduler, we are not able to pin
background work to specific threads. As a result we are unable to prevent
concurrent UnifiedLog.flush() calls, so we have to ensure UnifiedLog.flush()
implementation is thread safe by modifying the code at subtle areas (ex: [PR
#11814|https://github.com/apache/kafka/pull/11814]). The code will be simpler
if instead KafkaScheduler (or alike) could support pinning of certain
background work to specific threads, for example the UnifiedLog.flush()
operation for the same topic-partition will go to the same thread. This will
ensure strict ordering of flush() calls, thereby enabling us to write simpler
code eventually.
> Pin background worker threads for certain background work (ex:
> UnifiedLog.flush())
> ----------------------------------------------------------------------------------
>
> Key: KAFKA-13701
> URL: https://issues.apache.org/jira/browse/KAFKA-13701
> Project: Kafka
> Issue Type: Improvement
> Reporter: Kowshik Prakasam
> Priority: Major
>
> Certain background work such as UnifiedLog.flush() need not support
> concurrency. Today in the existing KafkaScheduler, we are not able to pin
> background work to specific threads. As a result we are unable to prevent
> concurrent UnifiedLog.flush() calls, so we have to ensure UnifiedLog.flush()
> implementation is thread safe by modifying the code at subtle areas (ex: [PR
> #11814|https://github.com/apache/kafka/pull/11814]). The code will be simpler
> if instead KafkaScheduler (or alike) could support pinning of certain
> background work to specific threads, for example the UnifiedLog.flush()
> operation for the same topic-partition will go to the same thread. This will
> ensure strict ordering of flush() calls, thereby enabling us to write simpler
> code eventually.
> Subsequently, in this Jira we will modify the code to schedule
> UnifiedLog.flush() only to specific background threads always (based on
> topic-partition).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
