[jira] [Created] (HIVE-23196) Reduce number of delete calls to NN during Context::clear

[Rajesh Balamohan (Jira)]

Rajesh Balamohan created HIVE-23196:
---------------------------------------

             Summary: Reduce number of delete calls to NN during Context::clear
                 Key: HIVE-23196
                 URL: https://issues.apache.org/jira/browse/HIVE-23196
             Project: Hive
          Issue Type: Improvement
            Reporter: Rajesh Balamohan


{\{Context::clear()}} ends up deleting same directories (or its subdirs) 
multiple times. It would be good to reduce the number of delete calls to NN for 
latency sensitive queries. This also has an impact on concurrent queries.

{noformat}
2020-04-14T04:22:28,703 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899 
HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting result dir: 
hdfs://nn1:8020/tmp/hive/rbalamohan/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13/-mr-10000
2020-04-14T04:22:28,721 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899 
HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir: 
hdfs://nn1:8020/tmp/hive/rbalamohan/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13
2020-04-14T04:22:28,737 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899 
HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir: 
hdfs://nn1:8020/tmp/hive/rbalamohan/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13/-mr-10000/.hive-staging_hive_2020-04-14_04-22-24_335_8573832618972595103-13
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)