Quanlong Huang created IMPALA-14361:
---------------------------------------
Summary: Improve time spent in TestConcurrentDdls
Key: IMPALA-14361
URL: https://issues.apache.org/jira/browse/IMPALA-14361
Project: IMPALA
Issue Type: Task
Components: Test
Reporter: Quanlong Huang
The majority of the test time is spent in dropping the unique_database. E.g.
TestConcurrentDdls::test_ddls_with_invalidate_metadata takes 90s. But the drop
database statement in the cleanup step takes 70s!
Query profile shows the time is spent in HMS:
{noformat}
Catalog Server Operation: 1m10s
- Got metastoreDdlLock: 63.756us (63.756us)
- Got Metastore client: 90.024us (26.268us)
- Fetched unloaded tables from Metastore: 36.921ms (36.831ms)
- Got Metastore client: 36.930ms (8.590us)
- Got current Metastore event id 357580: 37.249ms (319.465us)
- Dropped database in Metastore: 1m10s (1m10s) <--- Major time here
- Fetched event batch from Metastore: 1m10s (1.203ms)
- Deleted database in catalog cache: 1m10s (204.722us)
- DDL finished: 1m10s (13.245us){noformat}
The test creates 30+ tables and DROP DATABASE CASCADE in HMS will drop them
sequentially thus take time.
I tried adding DROP TABLE statements at the end of each thread but this doesn't
help: The cleanup step finishes quickly but the test itself runs longer.
CC [~rizaon] , [~MikaelSmith]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
