This sounds exciting. The next question is how do you plan to do it? Would a physical plan be translated to a Storm job (or jobs)? Would it need a different physical plan? Or would you just have the connection at the language layer and all the planning separate? Do you envision needing extensions/changes to the language to support Storm? Feel free to add a page to Pig's wiki with your thoughts on an approach.
Alan. On Jul 23, 2013, at 9:52 AM, Pradeep Gollakota wrote: > Hi Pig Developers, > > I wanted to reach out to you all and ask for you opinion on something. > > As a Pig user, I have come to love Pig as a framework. Pig provides a great > set of abstractions that make working with large datasets easy. Currently > Pig is only backed by hadoop. However, with the new rise of Twitter Storm > as a distributed real time processing engine, Pig users are missing out on > a great opportunity to be able to work with Pig in Storm. As a user of Pig, > Hadoop and Storm, and keeping with the Pig philosophy of "Pigs live > anywhere," I'd like to get your thoughts on starting the implementation of > a Pig backend for Storm. > > Thanks > Pradeep
