Re: Automatic creation of Hive output table based on SELECT statement

I had the same problem sometime back and I found a dirty workaround that actually works. 1) create view foo as # this automatically creates foo view with select output schema, this can contain joins, sel conditions, group/count operators etc. 2) create table bar like foo # creates bar with schema

Re: Hive .012 works with CDH 4.2.0

bcc: user@hive.apache.org + cdh-u...@cloudera.org No, they aren't tested together and we suggested not replace the packages as there is chance that you might corrupt your hive metastore data and this makes existing hive unusable. You might want to upgrade to CDH 5 beta that includes many of the b

Re: Is it possible to host hive server on port 80?

Ports below 1024 can only be used by superusers. Try doing a sudo! On Sun, Sep 22, 2013 at 11:55 AM, Gary Zhao wrote: > > > > On Sat, Sep 21, 2013 at 11:20 PM, Gary Zhao wrote: > >> Hello >> >> It failed to start hiveserver if I specify 80 port and there is no other >> process using it. I'm wo

Re: Enhancing Query Join to speed up Query

Hi, Can you please paste the output of "explain" for both the queries so that we can know if the predicate is being pushed down in both the cases. Thanks, On Fri, Jun 14, 2013 at 10:05 AM, Naga Vijay wrote: > Hi, > > Here are the test results for the two query options shown in my initial > em

Re: hive.metastore.warehouse.dir - Should it point to a physical directory

Yes ! On Tue, May 21, 2013 at 11:41 PM, Raj Hadoop wrote: > So that means I need to create a HDFS ( Not an OS physical directory ) > directory under Hadoop that need to be used in the Hive config file for > this property. Right? > > *From:* Dean Wampler > *To:* Raj Hadoop > *Cc:* Sanjay Subr

Re: hive.metastore.warehouse.dir - Should it point to a physical directory

Hi, If by "local physical directory" you mean a directory in the underlying OS file system, then No. You just need to create a directory in HDFS and ad it to that xml config file. Thanks, On Tue, May 21, 2013 at 11:19 PM, Raj Hadoop wrote: > Ok.I got it. My questions - > > 1) Should a local p

Re: Location of external table in hdfs

When you create an external table, original data ('/tmp/states' in this case) is NOT copied to the warehouse folder (or infact any other folder for that matter). So you can find it in '/tmp/states' ifself. On Tue, Mar 5, 2013 at 10:26 PM, Sai Sai wrote: > I have created an external table like bel

Re: hive odbc driver

Hi, This looks like a CDH specific issue. You'd get a better response in their mailing lists. Thanks, On Wed, Jan 30, 2013 at 11:59 AM, Chhaya Vishwakarma < chhaya.vishwaka...@lntinfotech.com> wrote: > Hi, > > ** ** > > I am using cloudera hive odbc driver made DSN but when I am trying t

Re: Partitions on hive hbase table

I'm not sure about partitioning but the scans are currently limited based on start and stop keys ( if predicates on rowkeys are provided in the query) See Hive-1643 ,2815 jiras ! On Mon, Oct 15, 2012 at 10:09 PM, kulkarni.swar...@gmail.com < kulkarni.swar...@gmail.com> wrote: > All, > > So, I ha

Re: Hive configuration property

I'm no expert in hive, but here are my 2 cents. By default hive schedules a reducer per every 1 GB of data ( change that value by modifying *hive.exec.reducers.bytes.per.reducer ) *. If your input data is huge, there will be large number of reducers, which might be unnecessary.( Sometimes large nu

Re: Does hive support nested queries?

Hive supports subqueries only in the FROM clause as of now. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+SubQueries On Mon, Sep 17, 2012 at 1:03 PM, Amila Maha Arachchi wrote: > Hi folks, > > I want to provide the output of a select query to a where clause of > another query a

Re: Performance: hive+hbase integration query against the row_key

Hey, Hive does all kinds of parsing , metadata lookups, query tree building and stuff before executing the query. Not sure if this all was included in those 36 seconds ! Also what hive does is, it builds a scan object with ranges based on predicates (and mappers too ) on key column and not a dire

Re: Long running Join Query - Reduce task fails due to failing to report status

tal committed heap usage (bytes)990,052,352 >>>> >>>> The tasktracker log gives a thread dump at that time but no exception. >>>> >>>> *2012-08-23 20:05:49,319 INFO org.apache.hadoop.mapred.TaskTracker: >>>> Process Thread Dump: lost task* >>>> *69 active threads* >>>> >>>> --- >>>> Thanks & Regards >>>> Himanish >>>> >>> >>> >>> >>> -- >>> "The whole world is you. Yet you keep thinking there is something else." >>> - Xuefeng Yicun 822-902 A.D. >>> >>> Tim R. Havens >>> Google Phone: 573.454.1232 >>> ICQ: 495992798 >>> ICBM: 37°51'34.79"N 90°35'24.35"W >>> ham radio callsign: NW0W >>> >> >> > > > -- > Bertrand Dechoux > -- Bharath Vissapragada, 4th Year undergraduate, IIIT Hyderabad. w: http://researchweb.iiit.ac.in/~bharath.v<http://research.iiit.ac.in/%7Ebharath.v>

Re: How to turn off logging in hive

See https://cwiki.apache.org/Hive/languagemanual-cli.html#LanguageManualCli-Logging . modify conf/hive-*-log4j.properties and set the logging level there! On Tue, Aug 7, 2012 at 12:44 PM, Ravindra wrote: > Hi , > > I used following property in the hive-site.xml and I still see that > logging i

Re: Benchmarking problems

ing tasks. > mapred.map.tasks.speculative.execution and > mapred.reduce.tasks.speculative.execution are the job conf options. > > > -Original Message- > From: bharath vissapragada [mailto:bharathvissapragada1...@gmail.com] > Sent: Tuesday, September 27, 2011 1:22 AM >

Re: Setting up stats database[SOLVED]

AM, wd wrote: > Maybe you should use > 'hive.stats.jdbcdriver="org.apache.mysql.jdbc.EmbeddedDriver"' > settings? > > via > http://mail-archives.apache.org/mod_mbox/hive-user/201103.mbox/%3c42360b00-72ec-437a-9d95-93f3ad9f1...@fb.com%3E > > On Fri, Aug

Re: Setting up stats database

Hi, Iam also getting the same error. However I am using mysql for stats. The thing is I configured mysql for metastore and it works fine and all the metadata gets populated normally. When the metastore classes can find the mysql jar in the class path , why cant the stats publisher find it. I look

Re: Flume Hive sink

Hey, Hive tables are nothing but some meta-data overlay on top of folders in HDFS containing table data. So I guess hdfs-sink of flume suffices. Please correct me if I am wrong. Thanks On Sun, Jun 5, 2011 at 1:52 AM, Prashanth R wrote: > Hi, > > Just throwing this out to get some good ideas.

Re: multiple hive access

Hey, Embedded mode is used just for testing purposes and the only once process can connect to it at a time. Configure it as a local/remote metastore. Refer http://wiki.apache.org/hadoop/Hive/AdminManual/MetastoreAdmin for details . Hope this helps. Thanks and Regards. On Thu, May 19, 2011 at 3:

Re: can I change the default field delimiter of the hive?

Hi , Please see http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL http://wiki.apache.org/hadoop/Hive/LanguageManual/DML These links are pretty good for starters ! Thanks On Sat, A

Re: skew join optimization

Hi Igor, See http://wiki.apache.org/hadoop/Hive/JoinOptimization and see the jira 1642 which automatically converts a normal join into map-join (Otherwise you can specify the mapjoin hints in the query itself.). Because your 'S' table is very small , it can be replicated across all the mappers and

Re: Not able to run Hive

I am also getting this error .. any suggestions? hive : 0.6 had :0.20.2 = On Mon, Jun 7, 2010 at 1:03 AM, Shuja Rehman wrote: > Hi all > Thanks for reply. > I have changed the heap size to 1024, then 512 then even 100 in the > specified file. But i am still getting this error. > I think

Re: Query Optimization in Hive

Hi , I updated the JIRA . Kindly give your suggestions so that I can go ahead and complete the task. Thanks On Tue, Feb 1, 2011 at 12:25 PM, bharath vissapragada wrote: > Thanks for replying namit.. > > It is motivating to receive a mail from the authors of Hive :). > > I

Re: Query Optimization in Hive

rom there. > > > This feature would be very useful to the community - We, at facebook, > Would definitely like to use it. > > > Thanks, > -namit > > > On 1/31/11 9:50 PM, "bharath vissapragada" > wrote: > >>Hi Ning,Anja, >> >>I am

Re: Query Optimization in Hive

Hi Ning,Anja, I am doing my Masters thesis on this topic . I have implemented all SQL features like joins , selects etc on top of Hadoop (before knowing about Hive) and we have derived some basic cost-models for join re-ordering which seem to be working fine on some basic scales of TPCH datasets .

Re: mergeJoinTree ??

Thanks for the help namit ! :) On Fri, Dec 24, 2010 at 12:00 PM, Namit Jain wrote: > Yes > > Sent from my iPhone > > On Dec 23, 2010, at 8:02 PM, "bharath vissapragada" >  > wrote: > >> Hi, >> >> Thanks for the reply . >> >> Thi

Re: mergeJoinTree ??

1,T2), T3) > into a new join operator with 3 children (T1,T2,T3) > > > On 12/23/10 4:24 AM, "bharath vissapragada" > wrote: > >>Hi all, >> >> >>Once the JoinTrees are created by the the semantic analyzer for a >>query block , a fu

mergeJoinTree ??

Hi all, Once the JoinTrees are created by the the semantic analyzer for a query block , a function mergeJoinTree(qb) is called . Can anyone tell me what it does ? Thanks in anticipation, Bharath.V

Functionality of RowResolver

Hi all, I am going through the src code of the hive trunk and Iam not clear about the usage of "org.apache.hadoop.hive.ql.parse.RowResolver" . Can anyone help me with this ? Thanks Bharath.V 4th Year UnderGraduate, IIIT Hyderabad