Sorry, missed this mail earlier. The fact that these files are missing is OK. When you ask for status info via REST it tries to return a list of fields (each field like 'exitValue' is a file when using HDFStorage, which is the default) but it doesn't know which ones have never been written. For example, if you didn't register a callback URL when submitting the job, the 'notified' will never be created. That's why these are DEBUG level messages.
On Sat, Dec 21, 2013 at 1:07 PM, Jonathan Hodges <hodg...@gmail.com> wrote: > Sorry forgot to mention the job tracker UI shows a TempletonControllerJob > completing successfully. > > > On Sat, Dec 21, 2013 at 9:37 AM, Jonathan Hodges <hodg...@gmail.com>wrote: > >> Hi Eugene, >> >> The few lines I included above are from webhcat.log >> >> DEBUG | 29 Nov 2013 15:15:36,133 | org.apache.hcatalog.templeton.Server | >> queued job job_201311281741_0020 in 13403 ms >> DEBUG | 29 Nov 2013 15:16:09,583 | >> org.apache.hcatalog.templeton.tool.HDFSStorage | Couldn't find >> /templeton-hadoop/jobs/job_201311281741_0020/notified: File does not exist: >> /templeton-hadoop/jobs/job_201311281741_0020/notified >> DEBUG | 29 Nov 2013 15:16:09,584 | >> org.apache.hcatalog.templeton.tool.HDFSStorage | Couldn't find >> /templeton-hadoop/jobs/job_201311281741_0020/callback: File does not exist: >> /templeton-hadoop/jobs/job_201311281741_0020/callback >> >> Those files indeed don't exist. The job_201311281741_0020 directory was >> created after starting the Hive job, but only has 'completed', 'exitValue', >> and 'user' files. Could these missing files be the issue? If so, how do I >> ensure they are created? >> >> Thanks in advance for the assistance. >> -Jonathan >> >> >> On Wed, Dec 18, 2013 at 5:15 PM, Eugene Koifman <ekoif...@hortonworks.com >> > wrote: >> >>> It may be worth looking in webhcat.log and using job tracker UI >>> >>> >>> On Mon, Dec 2, 2013 at 6:21 AM, Jonathan Hodges <hodg...@gmail.com>wrote: >>> >>>> Hi, >>>> >>>> I have setup WebHCat that is bundled with Hive 0.11.0. I am able to >>>> kick of map reduce jobs with the REST API successfully. However I am >>>> having some issues with Hive commands over REST. The following is my >>>> $TEMPLETON_HOME/webhcat-site.xml. >>>> >>>> >>>> <?xml version="1.0" encoding="UTF-8"?> >>>> <!-- The default settings for Templeton. --> >>>> <!-- Edit templeton-site.xml to change settings for your local --> >>>> <!-- install. --> >>>> >>>> <configuration> >>>> >>>> <property> >>>> <name>templeton.pig.archive</name> >>>> >>>> <value>s3n://pearson-alto-hadoop/apps/webhcat/archives/pig-0.11.1.tar.gz</value> >>>> <description>The path to the Pig archive.</description> >>>> </property> >>>> >>>> <property> >>>> <name>templeton.pig.path</name> >>>> <value>pig-0.11.1.tar.gz/pig-0.11.1/bin/pig</value> >>>> <description>The path to the Pig executable.</description> >>>> </property> >>>> >>>> <property> >>>> <name>templeton.hive.archive</name> >>>> >>>> <value>s3n://pearson-alto-hadoop/apps/webhcat/archives/hive-0.11.0.tar.gz</value> >>>> <description>The path to the Hive archive.</description> >>>> </property> >>>> >>>> <property> >>>> <name>templeton.hive.path</name> >>>> <value>hive-0.11.0.tar.gz/hive-0.11.0-bin/bin/hive</value> >>>> <description>The path to the Hive executable.</description> >>>> </property> >>>> >>>> </configuration> >>>> >>>> >>>> curl -s -d user.name=hadoop \ >>>> > -d execute="show+tables;" \ >>>> > -d statusdir="s3n://pearson-alto-hadoop/webhcat/hive" \ >>>> > 'http://10.201.5.28:50111/templeton/v1/hive' >>>> {"id":"job_201311281741_0020"} >>>> >>>> >>>> When I check the statusdir and jobs folders I see the job had an exit >>>> status of 1 so it wasn't successful. >>>> >>>> hadoop fs -ls s3n://pearson-alto-hadoop/webhcat/hive >>>> Found 1 items >>>> -rwxrwxrwx 1 2 2013-11-29 15:15 /webhcat/hive/exit >>>> >>>> >>>> hadoop fs -ls /templeton-hadoop/jobs/job_201311281741_0020 >>>> Found 3 items >>>> -rw-r--r-- 1 hadoop supergroup 4 2013-11-29 15:15 >>>> /templeton-hadoop/jobs/job_201311281741_0020/completed >>>> -rw-r--r-- 1 hadoop supergroup 1 2013-11-29 15:15 >>>> /templeton-hadoop/jobs/job_201311281741_0020/exitValue >>>> -rw-r--r-- 1 hadoop supergroup 6 2013-11-29 15:15 >>>> /templeton-hadoop/jobs/job_201311281741_0020/user >>>> >>>> Here is what I see in the logs. >>>> >>>> DEBUG | 29 Nov 2013 15:15:36,133 | org.apache.hcatalog.templeton.Server >>>> | queued job job_201311281741_0020 in 13403 ms >>>> DEBUG | 29 Nov 2013 15:16:09,583 | >>>> org.apache.hcatalog.templeton.tool.HDFSStorage | Couldn't find >>>> /templeton-hadoop/jobs/job_201311281741_0020/notified: File does not exist: >>>> /templeton-hadoop/jobs/job_201311281741_0020/notified >>>> DEBUG | 29 Nov 2013 15:16:09,584 | >>>> org.apache.hcatalog.templeton.tool.HDFSStorage | Couldn't find >>>> /templeton-hadoop/jobs/job_201311281741_0020/callback: File does not exist: >>>> /templeton-hadoop/jobs/job_201311281741_0020/callback >>>> >>>> >>>> How do I figure out the reason for failure? >>>> >>>> Thanks, >>>> Jonathan >>>> >>> >>> >>> CONFIDENTIALITY NOTICE >>> NOTICE: This message is intended for the use of the individual or entity >>> to which it is addressed and may contain information that is confidential, >>> privileged and exempt from disclosure under applicable law. If the reader >>> of this message is not the intended recipient, you are hereby notified that >>> any printing, copying, dissemination, distribution, disclosure or >>> forwarding of this communication is strictly prohibited. If you have >>> received this communication in error, please contact the sender immediately >>> and delete it from your system. Thank You. >>> >> >> > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
