On Mon, Feb 2, 2015 at 7:12 AM, Marshall Bockrath-Vandegrift < llas...@gmail.com> wrote:
> On Friday, January 30, 2015 at 5:00:31 PM UTC-5, Jason Wolfe wrote: > > Thanks for the recommendation. For now we're looking for a simple >> low-level interface to MR, but we're also keeping an eye on parkour and >> pigpen for more complex tasks down the road. Can you explain why I might >> prefer parkour to pigpen or vice-versa? >> > > Parkour actually *is* a low-level interface to MR. It just exposes that > interface though relatively Clojure-idiomatic and composable abstractions, > which can make it look higher-level than it really is. At Parkour's core > is the support required for MR tasks to invoke a regular Clojure var-bound > function in place of the `.run` method of a `Mapper` or `Reducer` class. > Everything else in Parkour is built to make using that primitive, low-level > interface more composable, convenient, and pleasant; but ultimately nothing > *replaces* that interface -- your Parkour MR Clojure task code runs in > exactly the way equivalent raw Hadoop MR Java task code would. > > Parkour's documentation includes a "motivation" document describing the > project motivation in the face of the Clojure-Hadoop integration projects > which existed when I started Parkour (including clojure-hadoop): > https://github.com/damballa/parkour/blob/master/doc/motivation.md . It > doesn't yet cover PigPen, although I certainly should add a section. I > honestly haven't evaluated PigPen in detail, but the approach of compiling > Clojure code to Pig seems excessively complex to me, to the point of only > be worth it for organizations which have already made a significant > investment in Pig. > Ah, that's very interesting -- thanks for the explanation. I think when I saw the similarity to pigpen syntax I just assumed parkour was also compiling to Pig. I'll definitely have to look into parkour further. Cheers, Jason > > -Marshall > > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to a topic in the > Google Groups "Clojure" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/clojure/bqGU3VRNFhY/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
