Re: WorkFlow Processing - Spark

2015-06-24 Thread ayan guha
As long as the logic can be run in parallel, yes. You should not however load any logic in driver. All logic should run in executors. On 25 Jun 2015 07:58, wrote: > Any custom script ( python or java or scala ) > > Thanks , > Ashish > > On Jun 24, 2015, at 4:39 PM, ayan guha wrote: > > What kind

Re: WorkFlow Processing - Spark

2015-06-24 Thread asoni . learn
Any custom script ( python or java or scala ) Thanks , Ashish > On Jun 24, 2015, at 4:39 PM, ayan guha wrote: > > What kind of custom logic? > >> On 25 Jun 2015 01:33, "Ashish Soni" wrote: >> Hi All , >> >> We are looking to use spark as our stream processing framework and it would >> be h

Re: WorkFlow Processing - Spark

2015-06-24 Thread ayan guha
What kind of custom logic? On 25 Jun 2015 01:33, "Ashish Soni" wrote: > Hi All , > > We are looking to use spark as our stream processing framework and it > would be helpful if experts can weigh if we made a right choice given below > requirement > > Given a stream of data we need to take those e