I am assuming it's a bad idea to have DB operations in a map? May be a better way is to chain the pipelines. Is that possible? eg:
workflow A ends -> workflow B starts On Mon, Feb 13, 2017 at 12:38 PM, Abhishek Singh < abhis...@tetrationanalytics.com> wrote: > You can keep adding stages, but then your sink is no more a sink - it > would have transformed into a map or a flatmap ! > > On Mon, Feb 13, 2017 at 12:34 PM Mohit Anchlia <mohitanch...@gmail.com> > wrote: > >> Is it possible to further add aggregation after the sink task executes? >> Or is the sink the last stage of the workflow? Is this flow possible? >> >> start stream -> transform -> load (sink) -> mark final state as loaded in >> a table after all the load was successful in previous state (sink) >> >
