Thanks Alan, this was helpful. On Thu, Jan 24, 2013 at 9:46 AM, Alan Gates <[email protected]> wrote:
> You might want to look at webhcat's code. It produces a servlet that it > embeds in a jetty server. You may be able to copy paste this to get what > you want. > > The code of interest is in the hcat repository under webhcat/svr. > > Alan. > > On Jan 24, 2013, at 9:42 AM, Prashant Kommireddi wrote: > > > Thanks Alan. We are trying to plug Pig into our existing app server. > > We have already done this for Java MR. The difficulty we are facing is > > with the fact that we can use JobClient.submitJob and jobtracker's job > > end notification to run jobs async, whereas PigServer.executeBatch > > blocks until pig job is complete. > > > > Sent from my iPhone > > > > On Jan 24, 2013, at 9:31 AM, Alan Gates <[email protected]> wrote: > > > >> If you're looking for an app server for Pig I'd take a look at a couple > of other projects already out there that can do this: > >> > >> 1) webhcat (fka Templeton, now part of the HCatalog project). It > provides a REST API that launches Pig, Hive, or MR jobs and allows you to > manage them, get results, etc. It's in HCatalog 0.5, which is in the > release candidate state. You can go to > http://people.apache.org/~travis/hcatalog-0.5.0-incubating-candidate-1/and > pick up the release candidate. > >> > >> 2) Oozie. Oozie's a workflow engine for Hadoop, but it also supports > submission of single Pig or MR jobs via REST. It may be a little > heavyweight for what you want but it works. > >> > >> Alan. > >> > >> On Jan 23, 2013, at 9:22 PM, Prashant Kommireddi wrote: > >> > >>> Both. Think of it as an app server handling all of these requests. > >>> > >>> Sent from my iPhone > >>> > >>> On Jan 23, 2013, at 9:09 PM, Jonathan Coveney <[email protected]> > wrote: > >>> > >>>> Thousands of requests, or thousands of Pig jobs? Or both? > >>>> > >>>> > >>>> 2013/1/23 Prashant Kommireddi <[email protected]> > >>>> > >>>>> Did not want to have several threads launched for this. We might have > >>>>> thousands of requests coming in, and the app is doing a lot more > than only > >>>>> Pig. > >>>>> > >>>>> On Wed, Jan 23, 2013 at 5:44 PM, Jonathan Coveney < > [email protected] > >>>>>> wrote: > >>>>> > >>>>>> start a separate Process which runs Pig? > >>>>>> > >>>>>> > >>>>>> 2013/1/23 Prashant Kommireddi <[email protected]> > >>>>>> > >>>>>>> Hey guys, > >>>>>>> > >>>>>>> I am trying to do the following: > >>>>>>> > >>>>>>> 1. Launch a pig job asynchronously via Java program > >>>>>>> 2. Get a notification once the job is complete (something similar > to > >>>>>>> Hadoop callback with a servlet) > >>>>>>> > >>>>>>> I looked at PigServer.executeBatch() and it seems to be waiting > until > >>>>> job > >>>>>>> completes.This is not what I would like my app to do. > >>>>>>> > >>>>>>> Any ideas? > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>> > >>>>> > >> > >
