Mona, Thanks for the response.
That doesn't quite look like my problem though; my Hive Oozie actions are working fine. As are my Pig Oozie actions, but things start breaking when trying to use HCat from within the Pig action. Are there any additional arguments required -- or configuration options -- for a Pig job using HCat? Or any working examples anywhere? Thanks Garry -----Original Message----- From: Mona Chitnis [mailto:[email protected]] Sent: 09 June 2014 19:20 To: [email protected] Subject: Re: Using HCat within a Pig action Looks like some discussion on this problem already https://groups.google.com/a/cloudera.org/forum/#!topic/hue-user/m8NnJvzxGAQ On 6/9/14, 6:00 AM, "Garry Turkington" <[email protected]> wrote: >Hi, > >I've got some Pig scripts that access data via HCat. They run fine on >the command line but if I try to get some executed as part of an Oozie >action it is failing. Unfortunately with very little detailed error messages. > >So before I go into the specifics can I clarify what is needed to get >Pig/HCat integration working with Oozie? > >I'm running this on CDH5 and the output of "oozie admin -listsharelib" >includes Pig and Hcatalog. Within my Pig scripts I am referring to HCat >tables by name alone, i.e. no hcat:// URI. The hive-site.xml that >works for Hive actions is available. I have other non-HCat workflows >running fine, including Pig and Hive actions. > >When I run Pig scripts that use HCatalog from the CLI I need specify >-useHcatalog and have HCAT.BIN defined; should I be passing values for >these to the Pig script within <argument> elements in the action >definition? (I've tried both with and without). > >Anything else that is required for this to work? Or pointers to any >documentation with examples/specs for what's needed? I found different >parts of the picture spread around but no definitive spec or full >examples. > >I cut my script down to the following; note the commented out second >statement, we don't even get as far as trying to read the (existing and >containing data) table: > >REGISTER >/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/pig/piggybank.jar > >mydata = LOAD 'testtable_hcat' USING >org.apache.hcatalog.pig.HCatLoader(); >-- store mydata into '/tmp/zz.out' using PigStorage(); > >I commented out the store because the only error I get includes the >seeming code JA018 and it was suggested on the Google this may be >permission related. Anything I need consider here? The cluster isn't >using any external security provider and only basic authentication: > >job_1402172905909_0054 FAILED/KILLEDJA018 > >Here's the workflow.xml: ><workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf"> > <start to="pig-node"/> > <action name="pig-node"> ><pig> > <job-tracker>${jobTracker}</job-tracker> > <name-node>${nameNode}</name-node> > <job-xml>${workflowRoot}/hive-site.xml</job-xml> > <configuration> > <property> > <name>mapred.job.queue.name</name> > <value>${queueName}</value> > </property> > </configuration> > <script>${workflowRoot}/pig/simple.pig</script> > </pig> > <ok to="end"/> > <error to="fail"/> ></action> > > <kill name="fail"> > <message>Pig action failed, error >message[${wf:errorMessage(wf:lastErrorNode())}]</message> > </kill> > <end name="end"/> ></workflow-app> > >Thanks >Garry > > ----- No virus found in this message. Checked by AVG - www.avg.com Version: 2014.0.4570 / Virus Database: 3955/7637 - Release Date: 06/07/14
