Re: Hive External Table issue

2013-06-20 Thread Stephen Sprague
i agree. conclusion: unless you're some kind of hive guru use a directory location and get that to work before trying to get clever with file locations - especially when you see an error message about "not a directory and unable to create it" :) Walk before you run good people. On Thu, Jun 20,

Re: Hive External Table issue

2013-06-20 Thread Nitin Pawar
Ramki, I was going through that thread before as Sanjeev said it worked so I was doing some experiment as well. As you I too had the impression that Hive tables are associated with directories and as pointed out I was wrong. Basically the idea of pointing a table to a file as mentioned on that th

Re: Hive External Table issue

2013-06-20 Thread Ramki Palle
Nitin, Can you go through the thread with subject "S3/EMR Hive: Load contents of a single file" on Tue, 26 Mar, 17:11> at http://mail-archives.apache.org/mod_mbox/hive-user/201303.mbox/thread?1 This gives the whole discussion about the topic of table location pointing to a filename vs. direct

Re: Hive External Table issue

2013-06-20 Thread Nitin Pawar
Also see this JIRA https://issues.apache.org/jira/browse/HIVE-951 I think issue you are facing is due to the JIRA On Thu, Jun 20, 2013 at 11:41 PM, Nitin Pawar wrote: > Mark has answered this before > > http://stackoverflow.com/questions/11269203/when-creating-an-external-table-in-hive-can-i-po

Re: Hive External Table issue

2013-06-20 Thread Ramki Palle
1. I was under the impression that you cannot refer the table location to a file. But, it looks like it works. Please see the discussion in the thread http://mail-archives.apache.org/mod_mbox/hive-user/201303.mbox/% 3c556325346ca26341b6f0530e07f90d96017084360...@gbgh-exch-cms.sig.ads%3e 2. It the

Re: Hive External Table issue

2013-06-20 Thread Nitin Pawar
Mark has answered this before http://stackoverflow.com/questions/11269203/when-creating-an-external-table-in-hive-can-i-point-the-location-to-specific-fil If this link does not answer your question, do let us know On Thu, Jun 20, 2013 at 11:33 PM, sanjeev sagar wrote: > Two issues: > > 1. I've

Re: Hive External Table issue

2013-06-20 Thread sanjeev sagar
Two issues: 1. I've created external tables in hive based on file location before and it work without any issue. It don't have to be a directory. 2. If there are more than one file in the directory, and you create external table based on directory then how the table knows that which file it need

Re: Hive External Table issue

2013-06-20 Thread Nitin Pawar
in hive when you create table and use the location to refer hdfs path, that path is supposed to be a directory. If the directory is not existing it will try to create it and if its a file it will throw an error as its not a directory thats the error you are getting that location you referred is a

Re: Hive External Table issue

2013-06-20 Thread sanjeev sagar
I did mention in my mail the hdfs file exists in that location. See below In HDFS: file exists hadoop fs -ls /user/flume/events/request_logs/ ar1.vgs.mypoints.com/13-06-13/FlumeData.1371144648033 Found 1 items -rw-r--r-- 3 hdfs supergroup 2242037226 2013-06-13 11:14 /user/flume/events/req

Re: Hive External Table issue

2013-06-20 Thread Nitin Pawar
MetaException(message:hdfs:// h1.vgs.mypoints.com:8020/user/flume/events/request_logs/ar1.vgs.mypoints.com/13-06-13/FlumeData.1371144648033 is not a directory or unable to create one) it clearly says its not a directory. Point to the dictory and it will work On Thu, Jun 20, 2013 at 10:52 PM, s