Re: WorkFlow Processing - Spark

2015-06-24 Thread ayan guha
As long as the logic can be run in parallel, yes. You should not however load any logic in driver. All logic should run in executors. On 25 Jun 2015 07:58, wrote: > Any custom script ( python or java or scala ) > > Thanks , > Ashish > > On Jun 24, 2015, at 4:39 PM, ayan guha wrote: > > What kind

Re: WorkFlow Processing - Spark

2015-06-24 Thread asoni . learn
Any custom script ( python or java or scala ) Thanks , Ashish > On Jun 24, 2015, at 4:39 PM, ayan guha wrote: > > What kind of custom logic? > >> On 25 Jun 2015 01:33, "Ashish Soni" wrote: >> Hi All , >> >> We are looking to use spark as our stream processing framework and it would >> be h

Re: WorkFlow Processing - Spark

2015-06-24 Thread ayan guha
What kind of custom logic? On 25 Jun 2015 01:33, "Ashish Soni" wrote: > Hi All , > > We are looking to use spark as our stream processing framework and it > would be helpful if experts can weigh if we made a right choice given below > requirement > > Given a stream of data we need to take those e

WorkFlow Processing - Spark

2015-06-24 Thread Ashish Soni
Hi All , We are looking to use spark as our stream processing framework and it would be helpful if experts can weigh if we made a right choice given below requirement Given a stream of data we need to take those event to multiple stage ( pipeline processing ) and in those stage customer will defi