You can checkpoint & itll stop the lineage to only updates after the checkpoint. Regards Mayur
Mayur Rustagi Ph: +919632149971 h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com https://twitter.com/mayur_rustagi On Wed, Feb 26, 2014 at 1:23 PM, Adrian Mocanu <amoc...@verticalscope.com>wrote: > Hi > > Scenario: Say I’ve been streaming tuples with Spark for 24 hours and one > of the nodes fails. > > The RDD will be recomputed on the other Spark nodes and the streaming > continues. > > > > I’m interested to know how I can skip the first 23 hours and jump in the > stream to the last hour. Is this possible? > > > > -Adrian > > >
