Re: FYI: MetaStore running out of threads

2022-08-31 Thread Rajesh Balamohan
>> In hive, FileUtils.checkFileAccessWithImpersonation can be fixed to use create UGI once to reduce the impact (suspecting this will have 50% impact). Looked closely at the method impl for "FileUtils.checkFileAccessWithImpersonation". It doesn't make 2 connections; 50% impact may not be relevant

Re: FYI: MetaStore running out of threads

2022-08-31 Thread Rajesh Balamohan
W.r.t to connection reuse issues, LLAP had a similar issue (not in HMS) https://issues.apache.org/jira/browse/HIVE-16020. It was making a connection in every task and UGI had to be persisted in the QueryInfo level to reduce the impact. In hive, FileUtils.checkFileAccessWithImpersonation can be fix