As long as the logic can be run in parallel, yes. You should not however load any logic in driver. All logic should run in executors. On 25 Jun 2015 07:58, <asoni.le...@gmail.com> wrote:
> Any custom script ( python or java or scala ) > > Thanks , > Ashish > > On Jun 24, 2015, at 4:39 PM, ayan guha <guha.a...@gmail.com> wrote: > > What kind of custom logic? > On 25 Jun 2015 01:33, "Ashish Soni" <asoni.le...@gmail.com> wrote: > >> Hi All , >> >> We are looking to use spark as our stream processing framework and it >> would be helpful if experts can weigh if we made a right choice given below >> requirement >> >> Given a stream of data we need to take those event to multiple stage ( >> pipeline processing ) and in those stage customer will define there own >> logic like custom code which we need to load inside a driver program ... >> >> Any idea the best way to do this ... >> >> Ashish >> >
