[ https://issues.apache.org/jira/browse/IGNITE-23989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vladimir Dmitrienko resolved IGNITE-23989. ------------------------------------------ Resolution: Cannot Reproduce No longer reproducible. > Tables creation in 1 thread throws "Send with retry timed out" > -------------------------------------------------------------- > > Key: IGNITE-23989 > URL: https://issues.apache.org/jira/browse/IGNITE-23989 > Project: Ignite > Issue Type: Bug > Components: persistence > Affects Versions: 3.0, 3.0.0-beta1 > Environment: 3 nodes (each node is CMG, each node > {color:#067d17}"-Xms4096m"{color}, {color:#067d17}"-Xmx4096m"{color}), each > on separate host. Each host vCPU: 4, Memory: 32GB. > Reporter: Igor > Priority: Major > Labels: ignite-3 > Attachments: > 20241213_053059_906000_TablesAmountCapacityMultiNodeTest_cluster.zip.001, > 20241213_053059_906000_TablesAmountCapacityMultiNodeTest_cluster.zip.002, > 20241213_053059_906000_TablesAmountCapacityMultiNodeTest_cluster.zip.003, > 20241213_053059_906000_TablesAmountCapacityMultiNodeTest_cluster.zip.004, > 20241213_053059_906000_TablesAmountCapacityMultiNodeTest_cluster.zip.005 > > > *Steps to reproduce:* > # Start 3 nodes (each node is CMG, each node > {color:#067d17}"-Xms4096m"{color}, {color:#067d17}"-Xmx4096m"{color}), each > on separate host. Each host vCPU: 4, Memory: 32GB. > # Create 50 tables with 200 columns in 1 thread. > # Assert 50 tables are present in system view. > # Insert 1 row into each. > # Assert rows content is correct in 1 threads. > # Repeat steps 2-5 while amount of tables is 1000. > *Expected:* > 1000 tables are created. > *Actual:* > Exception during 550-599 tables creation at step 2. > {code:java} > org.opentest4j.AssertionFailedError: org.opentest4j.AssertionFailedError: > Execute: Create tables 550 - 599 ==> Unexpected exception thrown: > java.util.concurrent.ExecutionException: java.sql.SQLException: Send with > retry timed out [retryCount = 3, groupId = metastorage_group, traceId = > 0b046f66-664e-4925-ac06-1ad5e17af09c, request = > org.apache.ignite.raft.jraft.rpc.WriteActionRequestImpl(org.apache.ignite.internal.metastorage.command.MultiInvokeCommandImpl), > originCommand = null]. > org.opentest4j.AssertionFailedError: Execute: Create tables 550 - 599 ==> > Unexpected exception thrown: java.util.concurrent.ExecutionException: > java.sql.SQLException: Send with retry timed out [retryCount = 3, groupId = > metastorage_group, traceId = 0b046f66-664e-4925-ac06-1ad5e17af09c, request = > org.apache.ignite.raft.jraft.rpc.WriteActionRequestImpl(org.apache.ignite.internal.metastorage.command.MultiInvokeCommandImpl), > originCommand = null]. > at > app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:152) > at > app//org.junit.jupiter.api.AssertDoesNotThrow.createAssertionFailedError(AssertDoesNotThrow.java:84) > at > app//org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:53) > at > app//org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:40) > at > app//org.junit.jupiter.api.Assertions.assertDoesNotThrow(Assertions.java:3183) > at > app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.lambda$invokeTasks$14(TablesAmountCapacityBaseTest.java:381) > at app//io.qameta.allure.Allure.lambda$step$0(Allure.java:113) > at app//io.qameta.allure.Allure.lambda$step$1(Allure.java:127) > at app//io.qameta.allure.Allure.step(Allure.java:181) > at app//io.qameta.allure.Allure.step(Allure.java:125) > at app//io.qameta.allure.Allure.step(Allure.java:112) > at > app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.step(TablesAmountCapacityBaseTest.java:271) > at > app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.invokeTasks(TablesAmountCapacityBaseTest.java:376) > at > app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.createTablesParallel(TablesAmountCapacityBaseTest.java:129) > at > app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.testTablesAmount(TablesAmountCapacityBaseTest.java:88) > at > app//org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityMultiNodeTest.create1000EmptyTablesAmountOfColumnsEachAndMakeSimpleQueries(TablesAmountCapacityMultiNodeTest.java:97) > at java.base@17.0.10/java.lang.reflect.Method.invoke(Method.java:568) > at > app//io.qameta.allure.junit5.AllureJunit5.interceptTestTemplateMethod(AllureJunit5.java:59) > at > java.base@17.0.10/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > java.base@17.0.10/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) > at > java.base@17.0.10/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) > at java.base@17.0.10/java.lang.Thread.run(Thread.java:842) > Caused by: java.util.concurrent.ExecutionException: java.sql.SQLException: > Send with retry timed out [retryCount = 3, groupId = metastorage_group, > traceId = 0b046f66-664e-4925-ac06-1ad5e17af09c, request = > org.apache.ignite.raft.jraft.rpc.WriteActionRequestImpl(org.apache.ignite.internal.metastorage.command.MultiInvokeCommandImpl), > originCommand = null]. > at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191) > at > org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.lambda$invokeTasks$13(TablesAmountCapacityBaseTest.java:383) > at > org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:49) > ... 19 more > Caused by: java.sql.SQLException: Send with retry timed out [retryCount = 3, > groupId = metastorage_group, traceId = 0b046f66-664e-4925-ac06-1ad5e17af09c, > request = > org.apache.ignite.raft.jraft.rpc.WriteActionRequestImpl(org.apache.ignite.internal.metastorage.command.MultiInvokeCommandImpl), > originCommand = null]. > at > org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57) > at > org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:160) > at > org.apache.ignite.internal.jdbc.JdbcStatement.executeUpdate(JdbcStatement.java:183) > at > org.gridgain.qaaas.sql.generator.TableGeneratorHelper.createTable(TableGeneratorHelper.java:98) > at > org.gridgain.ai3tests.tests.amountcapacity.TablesAmountCapacityBaseTest.lambda$createTablesParallel$0(TablesAmountCapacityBaseTest.java:121) > ... 4 more {code} > Server logs are in attachments. -- This message was sent by Atlassian Jira (v8.20.10#820010)