Xuefu Zhang created HIVE-17586:
----------------------------------
Summary: Make HS2 BackgroundOperationPool not fixed
Key: HIVE-17586
URL: https://issues.apache.org/jira/browse/HIVE-17586
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 1.1.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Currently the threadpool for background asynchronous operatons has a fixed size
controled by {{hive.server2.async.exec.threads}}. However, the thread factory
supplied for this threadpool is {{ThreadFactoryWithGarbageCleanup}} which
creates ThreadWithGarbageCleanup. Since this is a fixed threadpool, the thread
is actually never killed, defecting the purpose of garbage cleanup as noted in
the thread class name. On the other hand, since these threads never go away,
significant resources such as threadlocal variables (classloaders, hiveconfs,
etc) are holding up even if there is no operation running. This can lead to
escalated HS2 memory usage.
Ideally, the threadpool should not be fixed, allowing thread to die out so
resources can be reclaimed. The existing config
{{hive.server2.async.exec.threads}} is treated as the max, and we can add a min
for the threadpool {{hive.server2.async.exec.min.threads}}. Default value for
this configure is -1, which keeps the existing behavior.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
