Running Javascript from scala spark

2015-05-26 Thread marcos rebelo
and depending on the structure I will do some calculation. Did someone make it work and can get me a simple snippet that works? Thanks for any support Best Regards Marcos Rebelo

Re: Running Javascript from scala spark

2015-05-26 Thread marcos rebelo
thing similar? Best Regards Marcos Rebelo On Tue, May 26, 2015 at 8:03 PM, Marcelo Vanzin wrote: > Is it just me or does that look completely unrelated to > Spark-the-Apache-project? > > On Tue, May 26, 2015 at 10:55 AM, Ted Yu wrote: > >> Have you looked at https:

Re: Running Javascript from scala spark

2015-05-26 Thread marcos rebelo
e the Javascripts to better match the business rules at the time. Yesterday I +- did all this work. Just the Javascript is missing, I need to do it today. Best Regards Marcos Rebelo On Wed, May 27, 2015 at 12:44 AM, ayan guha wrote: > Yes you are in right mailing list, for sure :) > >

Saving calculation to single local file

2015-06-05 Thread marcos rebelo
Hi all I'm running spark in a single local machine, no hadoop, just reading and writing in local disk. I need to have a single file as output of my calculation. if I do "rdd.saveAsTextFile(...)" all runs ok but I get allot of files. Since I need a single file I was considering to do something li

Re: Saving calculation to single local file

2015-06-05 Thread marcos rebelo
is huge since one task will be doing the >> whole writing. >> >> Thanks >> Best Regards >> >> On Fri, Jun 5, 2015 at 3:46 PM, marcos rebelo wrote: >> >>> Hi all >>> >>> I'm running spark in a single local machine, no hadoop, just

Importing a github project on sbt

2017-01-16 Thread marcos rebelo
:: [warn] [warn] Note: Unresolved dependencies path: [warn] default:aws-stepfunctions_2.11:1.0 [warn] +- de.home24:importerwithstepfunction_2.11:0.1-SNAPSHOT Clearly I'm missing something. Can you direct me to the solution or to documentation? I will work someth

Scala Developers

2017-01-25 Thread marcos rebelo
the perfect article to the client, collect all the data on the site, … We are responsible to build Recommenders, Site Search, … The technology is always changing, so you can learn on the job. Do you see yourself working as a Data Engineer? Contact me, I promise a great team. Best Regards Marcos

Fast Unit Tests

2018-05-01 Thread marcos rebelo
Hey all, We are using Scala and SQL heavily, but I have a problem with VERY SLOW Unit Tests. Is there a way to do fast Unit Tests on Spark? How are you guys going around it? Best Regards Marcos Rebelo