Feng Jin created FLINK-36242:
--------------------------------
Summary: Fix unstable test in MaterializedTableITCase
Key: FLINK-36242
URL: https://issues.apache.org/jira/browse/FLINK-36242
Project: Flink
Issue Type: Bug
Components: Table SQL / Gateway, Tests
Reporter: Feng Jin
h3. Error message:
{code:java}
Aug 21 09:32:20 09:32:20.322 [ERROR] Tests run: 18, Failures: 0, Errors: 1,
Skipped: 0, Time elapsed: 69.61 s <<< FAILURE! – in
org.apache.flink.table.gateway.service.MaterializedTableStatementITCase
Aug 21 09:32:20 09:32:20.322 [ERROR]
org.apache.flink.table.gateway.service.MaterializedTableStatementITCase.testDropMaterializedTableWithDeletedRefreshWorkflowInFullMode
– Time elapsed: 0.415 s <<< ERROR!
Aug 21 09:32:20 org.apache.flink.table.gateway.api.utils.SqlGatewayException:
Failed to getTable.
Aug 21 09:32:20 at
org.apache.flink.table.gateway.service.SqlGatewayServiceImpl.getTable(SqlGatewayServiceImpl.java:300)
Aug 21 09:32:20 at
org.apache.flink.table.gateway.AbstractMaterializedTableStatementITCase.after(AbstractMaterializedTableStatementITCase.java:196)
Aug 21 09:32:20 at java.lang.reflect.Method.invoke(Method.java:498)
Aug 21 09:32:20 at
java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
Aug 21 09:32:20 at
java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
Aug 21 09:32:20 at
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
Aug 21 09:32:20 at
java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
Aug 21 09:32:20 at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
Aug 21 09:32:20 Caused by: org.apache.flink.table.api.TableException: Cannot
find table '`test_catalog12`.`test_db`.`users_shops`' in any of the catalogs
[test_catalog6, test_catalog7, test_catalog4, test_catalog5, test_catalog8,
test_catalog9, test_catalog12, test_catalog11, test_catalog2, test_catalog10,
test_catalog3, test_catalog1, default_catalog], nor as a temporary table.
Aug 21 09:32:20 at
org.apache.flink.table.catalog.CatalogManager.lambda$getTableOrError$4(CatalogManager.java:673)
Aug 21 09:32:20 at java.util.Optional.orElseThrow(Optional.java:290)
Aug 21 09:32:20 at
org.apache.flink.table.catalog.CatalogManager.getTableOrError(CatalogManager.java:670)
Aug 21 09:32:20 at
org.apache.flink.table.gateway.service.operation.OperationExecutor.getTable(OperationExecutor.java:297)
Aug 21 09:32:20 at
org.apache.flink.table.gateway.service.SqlGatewayServiceImpl.getTable(SqlGatewayServiceImpl.java:297)
Aug 21 09:32:20 ... 7 more
{code}
{color:#000000}Corresponding test link{color}
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=61530&view=logs&j=0c940707-2659-5648-cbe6-a1ad63045f0a&t=075c2716-8010-5565-fe08-3c4bb45824a4&s=ae4f8708-9994-57d3-c2d7-b892156e7812]
h3. Problem:
{color:#000000}As shown in the error message above, in the test method
afterEach, we will list all materialized tables and then drop them to prevent
any remaining refresh tasks.{color}
{color:#000000}However, since the Drop Materialized Table has already been
deleted, it causes the error mentioned above.{color}
{color:#000000}* Why does listing tables show they exist when dropping tables
results in an error stating they do not exist?{color}
{color:#000000}1. In the test
dropMaterializedTableWithDeletedRefreshWorkflowInFullMode, we manually dropped
the Materialized Table.{color}
{color:#000000}2. Despite manual deletion, background refresh tasks are still
being submitted. This leads to data continuing to be written into the
corresponding table data directory.{color}
{color:#000000}3. TestFileSystemCatalog lists all directories for existing
tables during listTable. If a directory exists, it returns true. However,
during dropTable it checks if both table and schema files exist simultaneously.
This inconsistency caused the mentioned issue.{color}
h3. {color:#000000}Solution:{color}
{color:#000000}1. Fix TestFileSystemCatalog logic for listTable by checking not
only directories but also schema file existence.{color}
{color:#000000}2. To further avoid this problem, change all tables in
MaterializedTableITCase to be manually dropped instead.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
