Hi all,

I have a question about the future of custom data sinks in Structured
Streaming. In particular, I want to know how continuous processing and the
Datasource API V2 will impact them.

Right now, it is possible to have custom data sinks via the current
Datasource API (V1) by implementing StreamSinkProvider and Sink traits. I
am wondering if this approach is planned to be the right way or it is
better to wait for the Datasource API V2 and the final implementation of
continuous processing.

As far as I understand SPARK-20928, there will be changes in the Source API
in SS. But what about the Sink API? Is it safe to implement it now?

Best regards,
Anton

Reply via email to