danhuawang opened a new issue, #10405: URL: https://github.com/apache/gravitino/issues/10405
### Version main branch ### Describe what's wrong 1. Drop table having `"on-cluster": "true"` <img width="1188" height="620" alt="Image" src="https://github.com/user-attachments/assets/b52dd4cf-34c6-443d-b7e8-d2e1cbfa5526"; /> 2. Create the table again <img width="1203" height="987" alt="Image" src="https://github.com/user-attachments/assets/39817a74-77ea-4d1a-b6b8-3caf4b60303b"; /> This issue didn't exist when the table don't have the `"on-cluster": "true"` properties ### Error message and/or stacktrace ``` { "code": 1004, "type": "TableAlreadyExistsException", "message": "Failed to operate table(s) [orders2] operation [CREATE] under schema [sales], reason [Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build)). (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))\n]", "stack": [ "org.apache.gravitino.exceptions.TableAlreadyExistsException: hostportstatuserrornum_hosts_remainingnum_hosts_activeStringUInt16Int64StringUInt64UInt64gmy-cluster-clickhouse-0-2-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-0-0-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#gmy-cluster-clickhouse-0-1-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-1-1-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-1-0-0. my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-1-2-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))Code: 57. DB::Exception: There was an error on [my-cluster-clickhouse-0-2-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local:9001]: Code: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build)). (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))", "", "\tat org.apache.gravitino.catalog.clickhouse.converter.ClickHouseExceptionConverter.toGravitinoException(ClickHouseExceptionConverter.java:44)", "\tat org.apache.gravitino.catalog.jdbc.operation.JdbcTableOperations.create(JdbcTableOperations.java:160)", "\tat org.apache.gravitino.catalog.jdbc.JdbcCatalogOperations.createTable(JdbcCatalogOperations.java:508)", "\tat org.apache.gravitino.catalog.TableOperationDispatcher.lambda$internalCreateTable$25(TableOperationDispatcher.java:599)", "\tat org.apache.gravitino.catalog.CatalogManager$CatalogWrapper.lambda$doWithTableOps$1(CatalogManager.java:153)", "\tat org.apache.gravitino.utils.IsolatedClassLoader.withClassLoader(IsolatedClassLoader.java:86)", "\tat org.apache.gravitino.catalog.CatalogManager$CatalogWrapper.doWithTableOps(CatalogManager.java:148)", "\tat org.apache.gravitino.catalog.TableOperationDispatcher.lambda$internalCreateTable$26(TableOperationDispatcher.java:597)", "\tat org.apache.gravitino.catalog.OperationDispatcher.doWithCatalog(OperationDispatcher.java:117)", "\tat org.apache.gravitino.catalog.TableOperationDispatcher.internalCreateTable(TableOperationDispatcher.java:594)", "\tat org.apache.gravitino.catalog.TableOperationDispatcher.lambda$createTable$5(TableOperationDispatcher.java:177)", "\tat org.apache.gravitino.lock.TreeLockUtils.doWithTreeLock(TreeLockUtils.java:49)", "\tat org.apache.gravitino.catalog.TableOperationDispatcher.createTable(TableOperationDispatcher.java:173)", "\tat org.apache.gravitino.hook.TableHookDispatcher.createTable(TableHookDispatcher.java:78)", "\tat org.apache.gravitino.catalog.TableNormalizeDispatcher.createTable(TableNormalizeDispatcher.java:78)", "\tat org.apache.gravitino.listener.TableEventDispatcher.createTable(TableEventDispatcher.java:136)", "\tat org.apache.gravitino.server.web.rest.TableOperations.lambda$createTable$1(TableOperations.java:151)", "\tat java.base/java.security.AccessController.doPrivileged(AccessController.java:712)", "\tat java.base/javax.security.auth.Subject.doAs(Subject.java:439)", "\tat org.apache.gravitino.utils.PrincipalUtils.doAs(PrincipalUtils.java:44)", "\tat org.apache.gravitino.server.web.Utils.doAs(Utils.java:202)", "\tat org.apache.gravitino.server.web.rest.TableOperations.createTable(TableOperations.java:137)", "\tat org.apache.gravitino.server.web.rest.TableOperations_$$_jvstdb6_9._d3createTable(TableOperations_$$_jvstdb6_9.java)", "\tat jdk.internal.reflect.GeneratedMethodAccessor148.invoke(Unknown Source)", "\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)", "\tat java.base/java.lang.reflect.Method.invoke(Method.java:569)", "\tat org.glassfish.hk2.utilities.reflection.ReflectionHelper.invoke(ReflectionHelper.java:1268)", "\tat org.jvnet.hk2.internal.MethodInterceptorHandler$MethodInvocationImpl.proceed(MethodInterceptorHandler.java:164)", "\tat org.apache.gravitino.server.web.filter.GravitinoInterceptionService$MetadataAuthorizationMethodInterceptor.invoke(GravitinoInterceptionService.java:208)", "\tat org.jvnet.hk2.internal.MethodInterceptorHandler.invoke(MethodInterceptorHandler.java:97)", "\tat org.apache.gravitino.server.web.rest.TableOperations_$$_jvstdb6_9.createTable(TableOperations_$$_jvstdb6_9.java)", "\tat jdk.internal.reflect.GeneratedMethodAccessor54.invoke(Unknown Source)", "\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)", "\tat java.base/java.lang.reflect.Method.invoke(Method.java:569)", "\tat org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)", "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)", "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)", "\tat org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)", "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)", "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)", "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)", "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)", "\tat org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)", "\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)", "\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)", "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:292)", "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:274)", "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:244)", "\tat org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)", "\tat org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)", "\tat org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)", "\tat org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)", "\tat org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)", "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358)", "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:311)", "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)", "\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)", "\tat org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)", "\tat org.apache.gravitino.server.authentication.AuthenticationFilter.lambda$doFilter$0(AuthenticationFilter.java:89)", "\tat java.base/java.security.AccessController.doPrivileged(AccessController.java:712)", "\tat java.base/javax.security.auth.Subject.doAs(Subject.java:439)", "\tat org.apache.gravitino.utils.PrincipalUtils.doAs(PrincipalUtils.java:44)", "\tat org.apache.gravitino.server.authentication.AuthenticationFilter.doFilter(AuthenticationFilter.java:86)", "\tat org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)", "\tat org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)", "\tat org.apache.gravitino.server.web.VersioningFilter.doFilter(VersioningFilter.java:111)", "\tat org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)", "\tat org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)", "\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)", "\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)", "\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)", "\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)", "\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)", "\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)", "\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)", "\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)", "\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)", "\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)", "\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)", "\tat org.eclipse.jetty.server.Server.handle(Server.java:516)", "\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)", "\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)", "\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)", "\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)", "\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)", "\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)", "\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)", "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)", "\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)", "\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)", "\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)", "\tat java.base/java.lang.Thread.run(Thread.java:840)", "Caused by: java.sql.SQLException: hostportstatuserrornum_hosts_remainingnum_hosts_activeStringUInt16Int64StringUInt64UInt64gmy-cluster-clickhouse-0-2-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-0-0-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#gmy-cluster-clickhouse-0-1-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-1-1-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-1-0-0.my-cluster-clickhouse-headl ess.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-1-2-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))Code: 57. DB::Exception: There was an error on [my-cluster-clickhouse-0-2-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local:9001]: Code: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build)). (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))", "", "\tat com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:85)", "\tat com.clickhouse.jdbc.SqlExceptionUtils.create(SqlExceptionUtils.java:31)", "\tat com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:90)", "\tat com.clickhouse.jdbc.internal.ClickHouseStatementImpl.sendRequest(ClickHouseStatementImpl.java:375)", "\tat com.clickhouse.jdbc.internal.ClickHouseStatementImpl.getLastResponse(ClickHouseStatementImpl.java:142)", "\tat com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeLargeUpdate(ClickHouseStatementImpl.java:554)", "\tat com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeUpdate(ClickHouseStatementImpl.java:563)", "\tat org.apache.commons.dbcp2.DelegatingStatement.executeUpdate(DelegatingStatement.java:330)", "\tat org.apache.commons.dbcp2.DelegatingStatement.executeUpdate(DelegatingStatement.java:330)", "\tat org.apache.gravitino.catalog.jdbc.utils.JdbcConnectorUtils.executeUpdate(JdbcConnectorUtils.java:42)", "\tat org.apache.gravitino.catalog.jdbc.operation.JdbcTableOperations.create(JdbcTableOperations.java:157)", "\t... 98 more", "Caused by: java.io.IOException: hostportstatuserrornum_hosts_remainingnum_hosts_activeStringUInt16Int64StringUInt64UInt64gmy-cluster-clickhouse-0-2-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-0-0-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#gmy-cluster-clickhouse-0-1-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-1-1-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-1-0-0.my-cluster-clickhouse-headles s.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))gmy-cluster-clickhouse-1-2-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local)#9xCode: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))Code: 57. DB::Exception: There was an error on [my-cluster-clickhouse-0-2-0.my-cluster-clickhouse-headless.clickhouse-operator-system.svc.cluster.local:9001]: Code: 57. DB::Exception: Table sales.orders2 already exists. (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build)). (TABLE_ALREADY_EXISTS) (version 25.7.8.71 (official build))", "", "\tat com.clickhouse.client.http.ApacheHttpConnectionImpl.checkResponse(ApacheHttpConnectionImpl.java:241)", "\tat com.clickhouse.client.http.ApacheHttpConnectionImpl.post(ApacheHttpConnectionImpl.java:304)", "\tat com.clickhouse.client.http.ClickHouseHttpClient.send(ClickHouseHttpClient.java:195)", "\tat com.clickhouse.client.AbstractClient.execute(AbstractClient.java:280)", "\tat com.clickhouse.client.ClickHouseClientBuilder$Agent.sendOnce(ClickHouseClientBuilder.java:282)", "\tat com.clickhouse.client.ClickHouseClientBuilder$Agent.send(ClickHouseClientBuilder.java:294)", "\tat com.clickhouse.client.ClickHouseClientBuilder$Agent.execute(ClickHouseClientBuilder.java:349)", "\tat com.clickhouse.client.ClickHouseClient.executeAndWait(ClickHouseClient.java:881)", "\tat com.clickhouse.client.ClickHouseRequest.executeAndWait(ClickHouseRequest.java:2154)", "\tat com.clickhouse.jdbc.internal.ClickHouseStatementImpl.sendRequest(ClickHouseStatementImpl.java:366)", "\t... 105 more" ] } ``` ### How to reproduce 1. Create a table ``` { "name": "orders2", "comment": "This is my ClickHouse table", "columns": [ { "name": "order_id", "type": "integer", "comment": "Order ID", "nullable": false }, { "name": "user_id", "type": "integer", "comment": "User ID", "nullable": false }, { "name": "amount", "type": "decimal(18,2)", "comment": "Amount", "nullable": false }, { "name": "created_at", "type": "timestamp", "comment": "Created time", "nullable": false } ], "properties": { "engine": "MergeTree", "on-cluster": "true", "cluster-name": "default" }, "sortOrders": [ { "sortTerm": { "type": "field", "fieldName": [ "order_id" ] }, "direction": "asc", "nullOrdering": "nulls_first" } ], "partitioning": [ ], "indexes": [ { "indexType": "primary_key", "name": "pk_order", "fieldNames": [ [ "order_id" ] ] } ] } ``` 2. Drop the table 3. Recreate the same table ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
