[
https://issues.apache.org/jira/browse/IMPALA-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18044021#comment-18044021
]
Quanlong Huang commented on IMPALA-6556:
----------------------------------------
We can add another section in the /operations page to show the in-flight
metadata loading. For each table, I think these info will be helpful:
* Table name
* Query id that triggers the metadata loading.
** In local-catalog mode, GetPartialCatalogObject requests have the query id
(IMPALA-14494) which can be displayed here.
** In the legacy catalog mode, PrioritizeLoad requests don't have the query
ids. It's OK to showing nothing since this mode is disabled by default and is
deprecated.
** If the metadata loading is triggered by DDL/DML requests, they also have
the query ids.
* Submit time / Scheduled time
** Timestamp when the task is put into the table loading queue
([code|https://github.com/apache/impala/blob/ff8bb33b91cd689bfaf2a51a19d0f615edd33d54/fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java#L263]).
** Only useful for async loading (i.e. prioritized and background loads). For
sync loading, it's the same as start time below.
* Start time
** Timestamp when the table loading actually starts, i.e. when
[Table.load()|https://github.com/apache/impala/blob/826c8cf9b083ed951fbfe649dc9551436efceb7e/fe/src/main/java/org/apache/impala/catalog/Table.java#L450]
is actually invoked.
* Waiting time
** Duration between submit time and start time.
* Duration
** Duration since the start time
* Progress
** How many partitions have been loaded. This is only useful for HdfsTable. It
can be tracked in this
[loop|https://github.com/apache/impala/blob/711797e7fbda6f30fc49d91e30ad6ab31a4f4a69/fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java#L193].
* Timeline
** A link to a page (or popup info) showing the string representation of the
loading
[timeline|https://github.com/apache/impala/blob/826c8cf9b083ed951fbfe649dc9551436efceb7e/fe/src/main/java/org/apache/impala/catalog/Table.java#L452C7-L452C25].
> Show what tables have been loading on Catalog WebUI
> ---------------------------------------------------
>
> Key: IMPALA-6556
> URL: https://issues.apache.org/jira/browse/IMPALA-6556
> Project: IMPALA
> Issue Type: New Feature
> Components: Catalog
> Reporter: Juan Yu
> Assignee: Quanlong Huang
> Priority: Major
>
> This helps users to know how many DDLs are running. How many tables have been
> loading.
> So users could know if a query is hung or just waiting for metadata.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
