In addition to Scalding and Scrunch, there is Scoobi. Unlike the others, it
is only Scala (it doesn't wrap a Java framework). All three have fairly
similar APIs and aren't too different from Spark. For example, instead of
RDD you have DList (distributed list) or PCollection (parallel collection)
- or in Scalding's case, Pipe, because Cascading had to get cute with its
names.


On Mon, Jul 7, 2014 at 8:12 PM, Sean Owen <so...@cloudera.com> wrote:

> On Tue, Jul 8, 2014 at 1:05 AM, Nabeel Memon <nm3...@gmail.com> wrote:
>
>> For Scala API on map/reduce (hadoop engine) there's a library called
>> "Scalding". It's built on top of Cascading. If you have a huge dataset or
>> if you consider using map/reduce engine for your job, for any reason, you
>> can try Scalding.
>>
>
> PS Crunch also has a Scala API called Scrunch. And Crunch can run its jobs
> on Spark too, not just M/R.
>
>
>


-- 
Daniel Siegmann, Software Developer
Velos
Accelerating Machine Learning

440 NINTH AVENUE, 11TH FLOOR, NEW YORK, NY 10001
E: daniel.siegm...@velos.io W: www.velos.io

Reply via email to