mrtisttt commented on PR #9421: URL: https://github.com/apache/seatunnel/pull/9421#issuecomment-2961194325
> I notice you use chunk splitter of jdbc. It's a not bad choice. But maybe use > > https://github.com/apache/seatunnel/blob/f4178c72f1d217b5b2c177f4b8b1d804fdde20ad/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/DynamicChunkSplitter.java#L58 > > is a better way. Then connector can split partition by itself without user config. Hi @Hisoka-X Thank you sincerely for your review. Certainly, the splitter in ClickHouse refers to the implementation of JDBC's `FixedChunkSplitter` and has undergone specific adaptations, optimizations, and fixes tailored for ClickHouse, making its functionality more stable and reliable. I have indeed noticed the `DynamicChunkSplitter` and reserved extensible approaches when implementing the `FixedChunkSplitter`. However, considering that ClickHouse did not support basic parallel reading previously, and the implementation of both `FixedChunkSplitter` and `DynamicChunkSplitter` would involve significant code development and testing efforts, it has not been implemented for the time being. The `FixedChunkSplitter` has stable functionality and allows users a certain degree of flexibility. On the basis of retaining the `FixedChunkSplitter`, I will submit a PR in the future to support the `DynamicChunkSplitter`. How about this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
