Re: spark and matlab

2014-08-27 Thread Jaonary Rabarisoa
forgot the second point, I found the answer myself inside the source code PipedRDD :) On Wed, Aug 27, 2014 at 1:36 PM, Jaonary Rabarisoa wrote: > Thank you Matei. > > I found a solution using pipe and matlab engine (an executable that can > call matlab behind the scene and uses stdin and stdou

Re: spark and matlab

2014-08-27 Thread Jaonary Rabarisoa
Thank you Matei. I found a solution using pipe and matlab engine (an executable that can call matlab behind the scene and uses stdin and stdout to communicate). I just need to fix two other issues : - how can I handle my dependencies ? My matlab script need other matlab files that need to be pre

Re: spark and matlab

2014-08-25 Thread Matei Zaharia
Have you tried the pipe() operator? It should work if you can launch your script from the command line. Just watch out for any environment variables needed (you can pass them to pipe() as an optional argument if there are some). On August 25, 2014 at 12:41:29 AM, Jaonary Rabarisoa (jaon...@gmail

spark and matlab

2014-08-25 Thread Jaonary Rabarisoa
Hi all, Is there someone that tried to pipe RDD into matlab script ? I'm trying to do something similiar if one of you could point some hints. Best regards, Jao