Hi, Start with DataStreamWriter.foreachBatch.
Pozdrawiam, Jacek Laskowski ---- https://about.me/JacekLaskowski "The Internals Of" Online Books <https://books.japila.pl/> Follow me on https://twitter.com/jaceklaskowski <https://twitter.com/jaceklaskowski> On Thu, Jan 7, 2021 at 6:55 PM mhd wrk <mhdwrkoff...@gmail.com> wrote: > I'm trying to convert a spark batch application to a streaming application > and wondering what function (or design pattern) I should use to execute a > series of operations inside the driver upon arrival of each message (a text > file inside an HDFS folder) before starting computation inside executors. > > Thanks, > Mohammad >
