mananpreetsingh opened a new issue, #8593: URL: https://github.com/apache/seatunnel/issues/8593
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened Hi, [ VTS repo](https://github.com/zilliztech/vts) is using seatunnel for datga migration from milvus to milvus DB. It was working few week ago but suddenly stops working with the new version VTS pushed. Their old tag still works fine so it is something related to seatunnel that is failing migration. Both source and destination is reachable since it creates collection name in destination milvus but does not copy data. Environment config: Milvus set up in Kubernetes Kubernetes cluster: 1.30 Milvus version: 2.5.3 (source and destination) migration config works fine with older versions. ### SeaTunnel Version Hazelcast IMDG version: 5.1.0 ### SeaTunnel Config ```conf env { parallelism = 1 job.mode = "BATCH" } source { Milvus { url="https://xyz:443"; # this is the remote milvus instance token="" database="default" collection="test" batch_size=100 } } sink { Milvus { url="http://milvus:19530"; # This is the local K8 service and vts container running in the same namespace. token="" database="default" batch_size=10 } } ``` ### Running Command ```shell ./bin/seatunnel.sh --config ./migration.config -m local ``` ### Error Exception ```log 2025-01-24 23:27:10,069 INFO [a.s.c.s.m.s.MilvusBufferReader] [BlockingWorker-TaskGroupLocation{jobId=935194012300607489, pipelineId=1, taskGroupId=30000}] - Collection 'test' is loaded. Starting query execution... 2025-01-24 23:27:10,069 INFO [a.s.c.s.m.s.MilvusBufferReader] [BlockingWorker-TaskGroupLocation{jobId=935194012300607489, pipelineId=1, taskGroupId=30000}] - Querying data from Milvus: collection=test, batchSize=100, offset=null, limit=null 2025-01-24 23:27:10,743 ERROR [a.s.m.MultiTableWriterRunnable] [st-multi-table-sink-writer-1] - MultiTableWriterRunnable error org.apache.seatunnel.connectors.seatunnel.milvus.exception.MilvusConnectorException: ErrorCode:[MILVUS-22], ErrorDescription:[Milvus write error] at org.apache.seatunnel.connectors.seatunnel.milvus.sink.MilvusSinkWriter.write(MilvusSinkWriter.java:141) ~[connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.connectors.seatunnel.milvus.sink.MilvusSinkWriter.write(MilvusSinkWriter.java:61) ~[connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.api.sink.multitablesink.MultiTableWriterRunnable.run(MultiTableWriterRunnable.java:62) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:39) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.lang.Thread.run(Thread.java:840) [?:?] Caused by: java.lang.NullPointerException: Cannot invoke "java.lang.Boolean.booleanValue()" because the return value of "org.apache.seatunnel.api.configuration.ReadonlyConfig.get(org.apache.seatunnel.api.configuration.Option)" is null at org.apache.seatunnel.connectors.seatunnel.milvus.sink.utils.MilvusSinkConverter.buildMilvusData(MilvusSinkConverter.java:279) ~[connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.connectors.seatunnel.milvus.sink.writer.MilvusBufferBatchWriter.write(MilvusBufferBatchWriter.java:79) ~[connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.connectors.seatunnel.milvus.sink.MilvusSinkWriter.write(MilvusSinkWriter.java:139) ~[connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] ... 8 more 2025-01-24 23:27:10,747 WARN [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=935194012300607489, pipelineId=1, taskGroupId=30000}] - [localhost]:5801 [seatunnel-503154] [5.1] Exception in org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask@65671f0b java.lang.RuntimeException: java.lang.RuntimeException: org.apache.seatunnel.connectors.seatunnel.milvus.exception.MilvusConnectorException: ErrorCode:[MILVUS-22], ErrorDescription:[Milvus write error] at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:253) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:66) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:75) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:50) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:51) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:73) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:78) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:693) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1018) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:39) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.lang.Thread.run(Thread.java:840) [?:?] Caused by: java.lang.RuntimeException: org.apache.seatunnel.connectors.seatunnel.milvus.exception.MilvusConnectorException: ErrorCode:[MILVUS-22], ErrorDescription:[Milvus write error] at org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.subSinkErrorCheck(MultiTableSinkWriter.java:136) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.write(MultiTableSinkWriter.java:173) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.api.sink.multitablesink.MultiTableSinkWriter.write(MultiTableSinkWriter.java:47) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:249) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] ... 17 more Caused by: org.apache.seatunnel.connectors.seatunnel.milvus.exception.MilvusConnectorException: ErrorCode:[MILVUS-22], ErrorDescription:[Milvus write error] at org.apache.seatunnel.connectors.seatunnel.milvus.sink.MilvusSinkWriter.write(MilvusSinkWriter.java:141) ~[connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.connectors.seatunnel.milvus.sink.MilvusSinkWriter.write(MilvusSinkWriter.java:61) ~[connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.api.sink.multitablesink.MultiTableWriterRunnable.run(MultiTableWriterRunnable.java:62) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] ... 6 more Caused by: java.lang.NullPointerException: Cannot invoke "java.lang.Boolean.booleanValue()" because the return value of "org.apache.seatunnel.api.configuration.ReadonlyConfig.get(org.apache.seatunnel.api.configuration.Option)" is null at org.apache.seatunnel.connectors.seatunnel.milvus.sink.utils.MilvusSinkConverter.buildMilvusData(MilvusSinkConverter.java:279) ~[connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.connectors.seatunnel.milvus.sink.writer.MilvusBufferBatchWriter.write(MilvusBufferBatchWriter.java:79) ~[connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.connectors.seatunnel.milvus.sink.MilvusSinkWriter.write(MilvusSinkWriter.java:139) ~[connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.connectors.seatunnel.milvus.sink.MilvusSinkWriter.write(MilvusSinkWriter.java:61) ~[connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.api.sink.multitablesink.MultiTableWriterRunnable.run(MultiTableWriterRunnable.java:62) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] ... 6 more 2025-01-24 23:27:10,748 INFO [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=935194012300607489, pipelineId=1, taskGroupId=30000}] - [localhost]:5801 [seatunnel-503154] [5.1] taskDone, taskId = 50000, taskGroup = TaskGroupLocation{jobId=935194012300607489, pipelineId=1, taskGroupId=30000} 2025-01-24 23:27:10,748 INFO [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=935194012300607489, pipelineId=1, taskGroupId=30000}] - [localhost]:5801 [seatunnel-503154] [5.1] task 50000 error with exception: [java.lang.RuntimeException: java.lang.RuntimeException: org.apache.seatunnel.connectors.seatunnel.milvus.exception.MilvusConnectorException: ErrorCode:[MILVUS-22], ErrorDescription:[Milvus write error]], cancel other task in taskGroup TaskGroupLocation{jobId=935194012300607489, pipelineId=1, taskGroupId=30000}. 2025-01-24 23:27:10,749 ERROR [a.s.c.s.m.s.MilvusBufferReader] [BlockingWorker-TaskGroupLocation{jobId=935194012300607489, pipelineId=1, taskGroupId=30000}] - Query failed. Batch size: 100. Aborting... java.lang.RuntimeException: java.lang.InterruptedException at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.received(IntermediateBlockingQueue.java:41) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.received(IntermediateQueueFlowLifeCycle.java:46) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.received(IntermediateQueueFlowLifeCycle.java:28) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.SeaTunnelSourceCollector.sendRecordToNext(SeaTunnelSourceCollector.java:195) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.SeaTunnelSourceCollector.collect(SeaTunnelSourceCollector.java:112) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.connectors.seatunnel.milvus.source.MilvusBufferReader.queryIteratorData(MilvusBufferReader.java:103) [connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.connectors.seatunnel.milvus.source.MilvusBufferReader.pollData(MilvusBufferReader.java:62) [connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.connectors.seatunnel.milvus.source.MilvusSourceReader.pollNext(MilvusSourceReader.java:92) [connector-milvus-2.3.8-SNAPSHOT.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:159) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:127) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:132) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:693) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1018) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:39) [seatunnel-starter.jar:2.3.8-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.lang.Thread.run(Thread.java:840) [?:?] Caused by: java.lang.InterruptedException at java.util.concurrent.locks.ReentrantLock$Sync.lockInterruptibly(ReentrantLock.java:159) ~[?:?] at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:372) ~[?:?] at java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:367) ~[?:?] at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:75) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.received(IntermediateBlockingQueue.java:39) ~[seatunnel-starter.jar:2.3.8-SNAPSHOT] ... 19 more ``` ### Zeta or Flink or Spark Version _No response_ ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
