[
https://issues.apache.org/jira/browse/IMPALA-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18045425#comment-18045425
]
Quanlong Huang edited comment on IMPALA-14490 at 12/16/25 10:01 AM:
--------------------------------------------------------------------
This failed by the same reason of IMPALA-13607 (See
[this|https://issues.apache.org/jira/browse/IMPALA-13607?focusedCommentId=18045419&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-18045419]).
The only difference is here catalogd processes the DROP_TABLE event so the
cache is updated. IMPALA-13607 is due to ALTER_PARTITION event.
Here are the catalogd logs:
[^catalogd.impala-ec2-redhat86-m6i-4xlarge-ondemand-0002.vpc.cloudera.com.jenkins.log.INFO.20251214-022818.2108185]
{noformat}
I20251214 02:28:35.071189 2109323 MetastoreEvents.java:936] EventId: 65504
EventType: DROP_TABLE Target:
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new.
Successfully removed table
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new
I20251214 02:28:35.071295 2109323 MetastoreEvents.java:936] EventId: 65504
EventType: DROP_TABLE Target:
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new.
Scheduling delay: 9ms, Process time: 290ms
I20251214 02:28:35.071345 2109537 CatalogMetastoreServer.java:216] Invoking HMS
API: get_table_req
I20251214 02:28:35.071396 2109323 MetastoreEvents.java:947] EventId: 65504
EventType: DROP_TABLE Target:
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new.
Complete process time: 299ms
I20251214 02:28:35.071444 2109323 EventExecutorService.java:529] Current count
of dispatched events that are being tracked: 0
I20251214 02:28:35.071494 2109323 EventExecutorService.java:554] Current count
of processed events that are tracked: 1, greatest synced event id: 65504
W20251214 02:28:35.071578 2109537 CatalogServiceCatalog.java:4568] Fetching
TABLE failed: TABLE_NOT_FOUND. Could not find TCatalogObject(type:TABLE,
catalog_version:0,
table:TTable(db_name:test_cache_valid_on_nontransactional_table_ddls_dbcsonw,
tbl_name:test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new))
I20251214 02:28:35.075098 2109537 MetastoreServiceHandler.java:1506] Received
exception while executing get_table_req
Java exception follows:
NoSuchObjectException(message:Table
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new
not found)
at
org.apache.impala.catalog.CatalogHmsAPIHelper.getPartialCatalogObjResponse(CatalogHmsAPIHelper.java:221)
at
org.apache.impala.catalog.CatalogHmsAPIHelper.getTableReq(CatalogHmsAPIHelper.java:151)
at
org.apache.impala.catalog.metastore.CatalogMetastoreServiceHandler.get_table_req(CatalogMetastoreServiceHandler.java:116)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.impala.catalog.metastore.CatalogMetastoreServer$TimingInvocationHandler.invoke(CatalogMetastoreServer.java:217)
at com.sun.proxy.$Proxy10.get_table_req(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20366)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20345)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
I20251214 02:28:35.075186 2109537 MetastoreServiceHandler.java:897] Forwarding
the request get_table_req for table
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new
to the backing HiveMetastore service
E20251214 02:28:35.077373 2109537 CatalogMetastoreServer.java:220] Received
exception while executing get_table_req :
Java exception follows:
NoSuchObjectException(message:hive.test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new
table not found)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result.read(ThriftHiveMetastore.java)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_req(ThriftHiveMetastore.java:2760)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_req(ThriftHiveMetastore.java:2747)
at
org.apache.impala.catalog.metastore.MetastoreServiceHandler.get_table_req(MetastoreServiceHandler.java:904)
at
org.apache.impala.catalog.metastore.CatalogMetastoreServiceHandler.get_table_req(CatalogMetastoreServiceHandler.java:122)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.impala.catalog.metastore.CatalogMetastoreServer$TimingInvocationHandler.invoke(CatalogMetastoreServer.java:217)
at com.sun.proxy.$Proxy10.get_table_req(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20366)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20345)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750){noformat}
and the corresponding test output:
{code:python}
custom_cluster/test_metastore_service.py:466: in
test_cache_valid_on_nontransactional_table_ddls
self.__test_non_transactional_table_cache_helper(db_name, tbl_name, False)
custom_cluster/test_metastore_service.py:679: in
__test_non_transactional_table_cache_helper
cur_get_table_response = catalog_hms_client.get_table_req(
/data/jenkins/workspace/impala-asf-master-core-s3-data-cache/repos/Impala/shell/impala_thrift_gen/hive_metastore/ThriftHiveMetastore.py:4381:
in get_table_req
return self.recv_get_table_req()
/data/jenkins/workspace/impala-asf-master-core-s3-data-cache/repos/Impala/shell/impala_thrift_gen/hive_metastore/ThriftHiveMetastore.py:4407:
in recv_get_table_req
raise result.o2
E impala_thrift_gen.hive_metastore.ttypes.NoSuchObjectException:
NoSuchObjectException(message='hive.test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new
table not found'){code}
Note that it doesn't matter if hierarchical event processing is enabled.
was (Author: stiga-huang):
This failed by the same reason of IMPALA-13607 (See
[this|https://issues.apache.org/jira/browse/IMPALA-13607?focusedCommentId=18045419&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-18045419]).
The only difference is here catalogd processes the DROP_TABLE event so the
cache is updated. IMPALA-13607 is due to ALTER_PARTITION event.
Here are the catalogd logs:
[^catalogd.impala-ec2-redhat86-m6i-4xlarge-ondemand-0002.vpc.cloudera.com.jenkins.log.INFO.20251214-022818.2108185]
{noformat}
I20251214 02:28:35.071189 2109323 MetastoreEvents.java:936] EventId: 65504
EventType: DROP_TABLE Target:
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new.
Successfully removed table
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new
I20251214 02:28:35.071295 2109323 MetastoreEvents.java:936] EventId: 65504
EventType: DROP_TABLE Target:
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new.
Scheduling delay: 9ms, Process time: 290ms
I20251214 02:28:35.071345 2109537 CatalogMetastoreServer.java:216] Invoking HMS
API: get_table_req
I20251214 02:28:35.071396 2109323 MetastoreEvents.java:947] EventId: 65504
EventType: DROP_TABLE Target:
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new.
Complete process time: 299ms
I20251214 02:28:35.071444 2109323 EventExecutorService.java:529] Current count
of dispatched events that are being tracked: 0
I20251214 02:28:35.071494 2109323 EventExecutorService.java:554] Current count
of processed events that are tracked: 1, greatest synced event id: 65504
W20251214 02:28:35.071578 2109537 CatalogServiceCatalog.java:4568] Fetching
TABLE failed: TABLE_NOT_FOUND. Could not find TCatalogObject(type:TABLE,
catalog_version:0,
table:TTable(db_name:test_cache_valid_on_nontransactional_table_ddls_dbcsonw,
tbl_name:test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new))
I20251214 02:28:35.075098 2109537 MetastoreServiceHandler.java:1506] Received
exception while executing get_table_req
Java exception follows:
NoSuchObjectException(message:Table
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new
not found)
at
org.apache.impala.catalog.CatalogHmsAPIHelper.getPartialCatalogObjResponse(CatalogHmsAPIHelper.java:221)
at
org.apache.impala.catalog.CatalogHmsAPIHelper.getTableReq(CatalogHmsAPIHelper.java:151)
at
org.apache.impala.catalog.metastore.CatalogMetastoreServiceHandler.get_table_req(CatalogMetastoreServiceHandler.java:116)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.impala.catalog.metastore.CatalogMetastoreServer$TimingInvocationHandler.invoke(CatalogMetastoreServer.java:217)
at com.sun.proxy.$Proxy10.get_table_req(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20366)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20345)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
I20251214 02:28:35.075186 2109537 MetastoreServiceHandler.java:897] Forwarding
the request get_table_req for table
test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new
to the backing HiveMetastore service
E20251214 02:28:35.077373 2109537 CatalogMetastoreServer.java:220] Received
exception while executing get_table_req :
Java exception follows:
NoSuchObjectException(message:hive.test_cache_valid_on_nontransactional_table_ddls_dbcsonw.test_cache_valid_on_nontransactional_table_ddls_tblgxpym_new
table not found)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_req_result.read(ThriftHiveMetastore.java)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:88)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_req(ThriftHiveMetastore.java:2760)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_req(ThriftHiveMetastore.java:2747)
at
org.apache.impala.catalog.metastore.MetastoreServiceHandler.get_table_req(MetastoreServiceHandler.java:904)
at
org.apache.impala.catalog.metastore.CatalogMetastoreServiceHandler.get_table_req(CatalogMetastoreServiceHandler.java:122)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.impala.catalog.metastore.CatalogMetastoreServer$TimingInvocationHandler.invoke(CatalogMetastoreServer.java:217)
at com.sun.proxy.$Proxy10.get_table_req(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20366)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:20345)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750){noformat}
Note that it doesn't matter if hierarchical event processing is enabled.
> test_cache_valid_on_nontransactional_table_ddls() fails with
> NoSuchObjectException
> ----------------------------------------------------------------------------------
>
> Key: IMPALA-14490
> URL: https://issues.apache.org/jira/browse/IMPALA-14490
> Project: IMPALA
> Issue Type: Bug
> Affects Versions: Impala 5.0.0
> Reporter: Laszlo Gaal
> Assignee: Quanlong Huang
> Priority: Blocker
> Attachments:
> catalogd.impala-ec2-redhat86-m6i-4xlarge-ondemand-0002.vpc.cloudera.com.jenkins.log.INFO.20251214-022818.2108185
>
>
> custom_cluster.test_metastore_service.TestMetastoreService.test_cache_valid_on_nontransactional_table_ddls()
> failed with
> {code}
> impala_thrift_gen.hive_metastore.ttypes.NoSuchObjectException:
> NoSuchObjectException(message='hive.test_cache_valid_on_nontransactional_table_ddls_dbhlrot.test_cache_valid_on_nontransactional_table_ddls_tblwqwpi_new
> table not found')
> {code}
> during s3-arm-datacache test.
> Stack trace:{code}
> custom_cluster/test_metastore_service.py:466: in
> test_cache_valid_on_nontransactional_table_ddls
> self.__test_non_transactional_table_cache_helper(db_name, tbl_name, False)
> custom_cluster/test_metastore_service.py:679: in
> __test_non_transactional_table_cache_helper
> cur_get_table_response = catalog_hms_client.get_table_req(
> /data/jenkins/workspace/impala-asf-master-core-s3-arm-data-cache/repos/Impala/shell/impala_thrift_gen/hive_metastore/ThriftHiveMetastore.py:4373:
> in get_table_req
> return self.recv_get_table_req()
> /data/jenkins/workspace/impala-asf-master-core-s3-arm-data-cache/repos/Impala/shell/impala_thrift_gen/hive_metastore/ThriftHiveMetastore.py:4399:
> in recv_get_table_req
> raise result.o2
> E impala_thrift_gen.hive_metastore.ttypes.NoSuchObjectException:
> NoSuchObjectException(message='hive.test_cache_valid_on_nontransactional_table_ddls_dbhlrot.test_cache_valid_on_nontransactional_table_ddls_tblwqwpi_new
> table not found'){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
