jtuglu1 commented on issue #9780: URL: https://github.com/apache/druid/issues/9780#issuecomment-4158989480
Hi folks (cc @clintropolis @gianm @abhishekrb19 @kfaraz), I'm interested in getting this connector back on the path of getting merged as an contributed extension. We have confidence in our (+ a few other users') ability to continue dev support of this connector. Currently, as it stands, I see the following blockers: 1. The connector is implemented in Scala (we should use Java). This should be straightforward to convert. 2. The connector is based on Spark 2.x, and Spark v4 has recently been released. I will need to port it to V3/V4. 3. The connector does not integrate with the task lock APIs. This is a big pain-point for off-cluster ingestion as it means users can easily build jobs that race with each other and need to rely on careful timing to synchronize their jobs. I will need to add interval locking support to this connector. Any other thoughts here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
