Re: metastore join TBLS to COLUMNS_V2

2014-08-25 Thread Alex Nastetsky
TBLS t on d.db_id=t.db_id join SDS s on t.sd_id=s.sd_id join COLUMNS_V2 c on s.cd_id=c.cd_id where d.name="some_database" and c.column_name="some_column_name"; On Mon, Aug 25, 2014 at 12:15 PM, Alex Nastetsky wrote: > Hi, > > I am trying to run some queries

metastore join TBLS to COLUMNS_V2

2014-08-25 Thread Alex Nastetsky
Hi, I am trying to run some queries against the metastore, but I am not sure how to query the list of Hive columns in the COLUMNS_V2 table for a specific Hive table in the TBLS table. I want to do something like select t.tbl_name, c.column_name from TBLS t join COLUMNS c on t.tbl_id = ... I tri

oozie and HADOOP_CLIENT_OPTS

2014-07-21 Thread Alex Nastetsky
Does anyone know where can I specify the equivalent of HADOOP_CLIENT_OPTS in Oozie? I have a Hive action in my workflow that does a Map Join, for which I need to increase the amount of client memory. When I just do it through Hive directly, I can do it by doing this: export HADOOP_CLIENT_OPTS=-Xm

oozie and HADOOP_CLIENT_OPTS

2014-07-21 Thread Alex Nastetsky
Does anyone know where can I specify the equivalent of HADOOP_CLIENT_OPTS in Oozie? I have a Hive action in my workflow that does a Map Join, for which I need to increase the amount of client memory. When I just do it through Hive directly, I can do it by doing this: export HADOOP_CLIENT_OPTS=-Xm

Re: custom table/column statistics

2014-06-09 Thread Alex Nastetsky
wrote: > Column group statistics is not supported in hive yet. > > Thanks > Prasanth > > Sent from my iPhone > > > On Jun 8, 2014, at 6:33 PM, Alex Nastetsky > wrote: > > > > Table statistics collection was added in HIVE-33 (numRows, rawDataSize, > et

custom table/column statistics

2014-06-08 Thread Alex Nastetsky
Table statistics collection was added in HIVE-33 (numRows, rawDataSize, etc). Is there anything that lets you create your own statistics gathering? For example, given table A with columns x, y, z, I want to gather count(distinct x, y) as a statistic that would be stored in the metastore. I know t

external tables and database authorization

2014-03-12 Thread Alex Nastetsky
My use case requires different groups of users to have access to different databases and to be able to create only external tables for those databases. The default database must not be usable at all. I am using storage based authorization on Hive 0.12.0.2.0.6.0. I have set the default database di

Re: disable internal tables

2014-01-31 Thread Alex Nastetsky
quires "execute" permission in order to copy the data into /apps/hive/warehouse. On Thu, Jan 30, 2014 at 12:00 PM, Alex Nastetsky wrote: > Thanks. But if I assign a group of the users to /apps/hive/warehouse then > they can still create internal tables, which is what I am trying

Re: disable internal tables

2014-01-30 Thread Alex Nastetsky
ite something at > '/apps/hive/warehouse', > it's best to assign a common group to your hive and hdfs users and assign > that group to both of these directories. I heard this issue is fixed in .12 > or .13, others can confirm. > > > On Thu, Jan 30, 2014 at 8:27 AM, Al

disable internal tables

2014-01-30 Thread Alex Nastetsky
Hi, I am trying to enforce all Hive tables to be created with EXTERNAL. The way I am doing this is by making the location of the warehouse (/apps/hive/warehouse in my case) to have permissions 000 (completely inaccessible). But then when I try to create an external table, I see that it still trie