I do not know Hadoop too well, but I will try to point the difference. The main idea is to implement MapReduce in a dynamic language. The aim is to use higher-order functions and abstract types. Advantages and disadvantages of the dynamic over static language are well known, so you can decide yourself if this implementation fits the problem being solved. In short it is: - Abstract implementation of algorithms on the data types - Much smaller volume of the source code - Skiped the compilation step - Expressiveness syntax
Sisyphus is a young project, so not really suitable for production use, the more that it lacks of user-friendly interface. However, I think that is excellent to learn MapReduce. A small volume of the source code implies faster understanding of the system and algorithms used in it, which encourages for repairing, improving, and modifying the system. Perhaps Sisyphus will even indicate areas in which Clojure could support Sisyphus natively. For example, serialization and deserialization of functions or moving several higher-order functions to the core of Clojure (eg, merge-sort or a lazy-group-by). What distinguishes the Sisyphus from MapReduce are: - The requesting user process becomes a master - System can not deal with errors on record-level (is skipping the whole split for a worker) - Lack of redundancy of tasks for faster processing - Operating on abstract data, rather than strings - System can not optimize performance basing on location of data and the nodes - Lack of a convenient interface for scheduling and viewing the status of tasks being performed - It operates in terms of number of records, rather than the volume of data (eg, when determinig the size of split or the size of sorting buffers) For now, enough on this topic. This is a good text on the WIKI, so thanks for your question. On 26 Sty, 01:18, kovas boguta <kovas.bog...@gmail.com> wrote: > Hi Olek, > > Could you explain how this differs from Hadoop in concept and in > execution? Thanks. > > > > > > > > On Tue, Jan 25, 2011 at 2:26 PM, Olek <aleksander.nas...@gmail.com> wrote: > > Hi! > > > It is nice to announce that Sisyphus - the google's mapreduce > > implemented in Clojure - has been released. > > Here are the sources:https://github.com/njoanna/Sisyphus > > > Some comments are still in polish but they will be gradually replaced > > with english version. > > > Right now there are 5 tasks (zadanie 1 - 5): > > > 1. Count number of occurences of a particular word in a set of text > > documents. The result is a pair of: searched word, its number of > > occurences. > > > 2. Look up for a sequence of chars in a set of text documents. The > > result is a set of pairs of: name of a file, line where the sequence > > has occured. > > > 3. Sort words in a set of text documents. The results is a set of > > sorted unique words. > > > 4. Create an index of words which length is higher then given value. > > The result is a sorted list of pairs of: words, list of files where > > the word occures. > > > 5. Create statistic of used words. The result is a sorted list of > > pairs: word, numer of occurences. > > > Hope you enjoy that. > > > Bye! > > > -- > > You received this message because you are subscribed to the Google > > Groups "Clojure" group. > > To post to this group, send email to clojure@googlegroups.com > > Note that posts from new members are moderated - please be patient with > > your first post. > > To unsubscribe from this group, send email to > > clojure+unsubscr...@googlegroups.com > > For more options, visit this group at > >http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
