HDFS, the Hive metastore and the hive client are all running as "hduser".
On Thu, Aug 25, 2011 at 8:22 PM, yongqiang he <heyongqiang...@gmail.com> wrote:
> what is your unix name on that machine? can u do a whoami?
>
> On Thu, Aug 25, 2011 at 5:15 PM, Alex Holmes <grep.a...@gmail.com> wrote:
>> Here's the hive-site.xml file (I use the same file for both the client
>> and remote metastore). We're using mysql as the metastore DB.
>>
>>
>> <?xml version="1.0"?>
>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>> <configuration>
>> <property>
>> <name>hive.security.authorization.enabled</name>
>> <value>true</value>
>> </property>
>> <property>
>> <name>hive.metastore.local</name>
>> <value>false</value>
>> </property>
>> <property>
>> <name>hive.metastore.uris</name>
>> <value>thrift://localhost:9083</value>
>> </property>
>> <property>
>> <name>javax.jdo.option.ConnectionURL</name>
>> <value>jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true</value>
>> </property>
>> <property>
>> <name>javax.jdo.option.ConnectionDriverName</name>
>> <value>com.mysql.jdbc.Driver</value>
>> </property>
>> <property>
>> <name>javax.jdo.option.ConnectionUserName</name>
>> <value>hive</value>
>> </property>
>> <property>
>> <name>javax.jdo.option.ConnectionPassword</name>
>> <value>secret</value>
>> </property>
>> </configuration>
>>
>>
>>
>> On Wed, Aug 24, 2011 at 6:06 PM, yongqiang he <heyongqiang...@gmail.com>
>> wrote:
>>> this is what i have tried with a remote metastore:
>>>
>>> > set hive.security.authorization.enabled=false;
>>> hive>
>>> >
>>> >
>>> > drop table src2;
>>> OK
>>> Time taken: 1.002 seconds
>>> hive> create table src2 (key int, value string);
>>> OK
>>> Time taken: 0.03 seconds
>>> hive>
>>> >
>>> >
>>> > set hive.security.authorization.enabled=true;
>>> hive> grant select on table src2 to user heyongqiang;
>>> OK
>>> Time taken: 0.113 seconds
>>> hive> select * from src2;
>>> OK
>>> Time taken: 0.188 seconds
>>> hive> show grant user heyongqiang on table src2;
>>> OK
>>>
>>> database default
>>> table src2
>>> principalName heyongqiang
>>> principalType USER
>>> privilege Select
>>> grantTime Wed Aug 24 15:03:51 PDT 2011
>>> grantor heyongqiang
>>>
>>> can u do a show grant?
>>>
>>> (But with remote metastore, i think hive should not return empty list
>>> instead of null for list_privileges etc.)
>>>
>>>
>>>
>>> On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes <grep.a...@gmail.com> wrote:
>>>> Authorization works for me with the local metastore. The remote
>>>> metastore works with authorization turned off, but as soon as I turn
>>>> it on and issue any commands I get these exceptions on the hive
>>>> client.
>>>>
>>>> Could you also try the remote metastore please? I'm pretty sure that
>>>> authorization does not work with it at all.
>>>>
>>>> Thanks,
>>>> Alex
>>>>
>>>> On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he <heyongqiang...@gmail.com>
>>>> wrote:
>>>>> I am using local metastore, and can not reproduce the problem.
>>>>>
>>>>> what message did you get when running local metastore?
>>>>>
>>>>> On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes <grep.a...@gmail.com> wrote:
>>>>>> Thanks for opening a ticket.
>>>>>>
>>>>>> Table-level grants aren't working for me either (HIVE-2405 suggests
>>>>>> that the bug is only related to global grants).
>>>>>>
>>>>>> hive> set hive.security.authorization.enabled=false;
>>>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>>>> OK
>>>>>> Time taken: 1.245 seconds
>>>>>> hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
>>>>>> FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
>>>>>> No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
>>>>>> hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE
>>>>>> pokes;
>>>>>> Copying data from file:/app/hadoop/hive1.in
>>>>>> Copying file: file:/app/hadoop/hive1.in
>>>>>> Loading data to table default.pokes
>>>>>> Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
>>>>>> OK
>>>>>> Time taken: 0.33 seconds
>>>>>> hive> select * from pokes;
>>>>>> OK
>>>>>> 1 a
>>>>>> 2 b
>>>>>> 3 c
>>>>>> Time taken: 0.095 seconds
>>>>>> hive> grant select on table pokes to user hduser;
>>>>>> OK
>>>>>> Time taken: 0.251 seconds
>>>>>> hive> set hive.security.authorization.enabled=true;
>>>>>> hive> select * from pokes;
>>>>>> FAILED: Hive Internal Error:
>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>> get_privilege_set failed: unknown result)
>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>> unknown result
>>>>>> at
>>>>>> org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>> at
>>>>>> org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>> at
>>>>>> org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>> ...
>>>>>>
>>>>>> mysql> select * from TBL_PRIVS;
>>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>>> | TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
>>>>>> PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
>>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>>> | 1 | 1314219701 | 0 | hduser | USER |
>>>>>> hduser | USER | Select | 1 |
>>>>>> +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
>>>>>>
>>>>>> Also, I noticed in HIVE-2405 that you get a meaningful error message:
>>>>>>
>>>>>> Authorization failed:No privilege 'Create' found for outputs {
>>>>>> database:default}. Use show grant to get more details.
>>>>>>
>>>>>> Whereas I just get an exception (as you can see above). Were you also
>>>>>> running with the remote metastore? I get these meaningful messages
>>>>>> with the local metastore (and authorization on), but with the remote
>>>>>> metastore with authorization turned on, I always get exceptions.
>>>>>>
>>>>>> Many thanks,
>>>>>> Alex
>>>>>>
>>>>>> On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he <heyongqiang...@gmail.com>
>>>>>> wrote:
>>>>>>> This is a bug. Will open a jira to fix this. and will backport it to
>>>>>>> 0.7.1.
>>>>>>> https://issues.apache.org/jira/browse/HIVE-2405
>>>>>>>
>>>>>>> thanks for reporting this one!
>>>>>>>
>>>>>>> On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes <grep.a...@gmail.com>
>>>>>>> wrote:
>>>>>>>> I created the mysql database (with the simple create database command)
>>>>>>>> and the remote metastore seemed to creat the mysql tables. Here's
>>>>>>>> some grant information and what I see in the database:
>>>>>>>>
>>>>>>>> [hduser@aholmes-desktop conf]$ hive
>>>>>>>> hive> grant all to user hduser;
>>>>>>>> OK
>>>>>>>> Time taken: 0.334 seconds
>>>>>>>> hive> show grant user hduser;
>>>>>>>> OK
>>>>>>>>
>>>>>>>> principalName hduser
>>>>>>>> principalType USER
>>>>>>>> privilege All
>>>>>>>> grantTime 1314191500
>>>>>>>> grantor hduser
>>>>>>>> Time taken: 0.046 seconds
>>>>>>>> hive> CREATE TABLE pokes (foo INT, bar STRING);
>>>>>>>> FAILED: Hive Internal Error:
>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>>>> get_privilege_set failed: unknown result)
>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>>>> unknown result
>>>>>>>> at
>>>>>>>> org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>>>> ...
>>>>>>>>
>>>>>>>> mysql> use hive;
>>>>>>>> Database changed
>>>>>>>> mysql> select * from GLOBAL_PRIVS;
>>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>>> | USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
>>>>>>>> | PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
>>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>>> | 1 | 1314191500 | 0 | hduser | USER
>>>>>>>> | hduser | USER | All |
>>>>>>>> +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
>>>>>>>> 1 row in set (0.00 sec)
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks for your help,
>>>>>>>> Alex
>>>>>>>>
>>>>>>>> On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he
>>>>>>>> <heyongqiang...@gmail.com> wrote:
>>>>>>>>> Have you created the metastore mysql tables for authorization? Can u
>>>>>>>>> do a show grant?
>>>>>>>>>
>>>>>>>>> thanks
>>>>>>>>> yongqiang
>>>>>>>>> On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes <grep.a...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I've been struggling with getting Hive authorization to work for a
>>>>>>>>>> few
>>>>>>>>>> hours, and I really hope someone can help me. I installed Hive 0.7.1
>>>>>>>>>> on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
>>>>>>>>>> configured Hive to enable authorization:
>>>>>>>>>>
>>>>>>>>>> <property>
>>>>>>>>>> <name>hive.security.authorization.enabled</name>
>>>>>>>>>> <value>true</value>
>>>>>>>>>> <description>enable or disable the hive client
>>>>>>>>>> authorization</description>
>>>>>>>>>> </property>
>>>>>>>>>>
>>>>>>>>>> I kept all the other Hive security configs with their default
>>>>>>>>>> settings.
>>>>>>>>>>
>>>>>>>>>> I'm running in pseudo-distributed mode on a single node. HDFS, the
>>>>>>>>>> Hive
>>>>>>>>>> metastore and the Hive CLI are all running as the same user (the HDFS
>>>>>>>>>> superuser). Here are the sequence of steps that are causing me
>>>>>>>>>> issues.
>>>>>>>>>> Without authorization everything works perfectly (creating, loading,
>>>>>>>>>> selecting).
>>>>>>>>>> I've also tried creating and loading the table without
>>>>>>>>>> authorization, granting
>>>>>>>>>> the select privilege at various levels (global, table, database),
>>>>>>>>>> turning on
>>>>>>>>>> auth and performing the select, resulting in the same exception.
>>>>>>>>>>
>>>>>>>>>> Any help with this would be greatly appreciated!
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Alex
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> [hduser@aholmes-desktop ~]$ hive
>>>>>>>>>> Hive history
>>>>>>>>>> file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
>>>>>>>>>> hive> set hive.security.authorization.enabled=false;
>>>>>>>>>> hive> grant all to user hduser;
>>>>>>>>>> OK
>>>>>>>>>> Time taken: 0.233 seconds
>>>>>>>>>> hive> set hive.security.authorization.enabled=true;
>>>>>>>>>> hive> CREATE TABLE pokes3 (foo INT, bar STRING);
>>>>>>>>>> FAILED: Hive Internal Error:
>>>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
>>>>>>>>>> get_privilege_set failed: unknown result)
>>>>>>>>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>>>>>>>>> org.apache.thrift.TApplicationException: get_privilege_set failed:
>>>>>>>>>> unknown result
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
>>>>>>>>>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
>>>>>>>>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>>>> at
>>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>>>>>> at
>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>>> Caused by: org.apache.thrift.TApplicationException: get_privilege_set
>>>>>>>>>> failed: unknown result
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
>>>>>>>>>> at
>>>>>>>>>> org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
>>>>>>>>>> ... 14 more
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
