[
https://issues.apache.org/jira/browse/HADOOP-16876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang resolved HADOOP-16876.
--------------------------------------
Resolution: Cannot Reproduce
> KMS delegation tokens are memory expensive
> ------------------------------------------
>
> Key: HADOOP-16876
> URL: https://issues.apache.org/jira/browse/HADOOP-16876
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
> Reporter: Wei-Chiu Chuang
> Priority: Major
> Attachments: Screen Shot 2020-02-20 at 5.04.12 PM.png
>
>
> We recently saw a number of users reporting high memory consumption in KMS.
> Part of the reason being HADOOP-14445. Without that, the number of kms
> delegation tokens that zookeeper stores is proportional to the number of KMS
> servers.
> There are two problems:
> (1) it exceeds zookeeper jute buffer length and operations fail.
> (2) KMS uses more heap memory to store KMS DTs.
> But even with HADOOP-14445, KMS DTs are still expensive. Looking at a heap
> dump from KMS, the majority of the heap is occupied by znode and KMS DT
> objects. With the growing number of encrypted clusters and use cases, this is
> increasingly a problem our users encounter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
