Quanlong Huang created IMPALA-13800:
---------------------------------------
Summary: Add catalogd logs for the progress of file metadata
loading
Key: IMPALA-13800
URL: https://issues.apache.org/jira/browse/IMPALA-13800
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Quanlong Huang
Assignee: Quanlong Huang
When loading file metadata of an Hdfs table, we just have logs when the
parallel load starts and finishes:
{code:java}
I0224 23:59:51.042261 2189 HdfsTable.java:1366] Fetched partition metadata
from the Metastore: default.part_6m
I0225 00:01:46.838979 2189 ParallelFileMetadataLoader.java:217] Loading file
and block metadata for 6000000 paths for table default.part_6m using a thread
pool of size 16
I0225 00:05:38.324651 2189 HdfsTable.java:840] Loaded file and block metadata
for default.part_6m partitions: p=0/q=0, p=0/q=1, p=0/q=10, and 5999997 others.
Time taken: 4m6s
W0225 00:05:45.111688 2189 HdfsTable.java:1388] Time taken on loading table
default.part_6m exceeded warning threshold. Time: 11m43s{code}
After fetching the partition metadata from HMS, no other logs except the JVM
pause logs are shown. It'd be helpful to add more logs to show the progress.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
