I am assuming it's a bad idea to have DB operations in a map? May be a better way is to chain the pipelines. Is that possible? eg:
workflow A ends -> workflow B starts On Mon, Feb 13, 2017 at 12:38 PM, Abhishek Singh < [email protected]> wrote: > You can keep adding stages, but then your sink is no more a sink - it > would have transformed into a map or a flatmap ! > > On Mon, Feb 13, 2017 at 12:34 PM Mohit Anchlia <[email protected]> > wrote: > >> Is it possible to further add aggregation after the sink task executes? >> Or is the sink the last stage of the workflow? Is this flow possible? >> >> start stream -> transform -> load (sink) -> mark final state as loaded in >> a table after all the load was successful in previous state (sink) >> >
