Re: Spark SQL is not returning records for hive bucketed tables on HDP

Hi Varadharajan, That is the point, Spark SQL is able to recognize delta files. See below directory structure, ONE BASE (43 records) and one DELTA (created after last insert). And I am able see last insert through Spark SQL. *See below complete scenario :* *Steps:* - Inserted 43 records in

Re: Spark SQL is not returning records for hive bucketed tables on HDP

Hi Varadharajan, Can you elaborate on (you quoted on previous mail) : "I observed that hive transaction storage structure do not work with spark yet" If it is related to delta files created after each transaction and spark would not be able recognize them. then I have a table *mytable *(ORC , BU

Re: Add multiple users in admin role while running in hive configuration-reg

HI Thanks for the ideas..I checked out and its very useful link. Thanks, Matheswaran On Mon, Feb 22, 2016 at 11:57 PM, Takahiko Saito wrote: > Hi Matheswaran, > > I believe existing admin can create a new role(admin) for other users via > beeline: > > https://cwiki.apache.org/confluence/displ

Re: Hive 2 error,: ORC does not support type conversion from CHAR to VARCHAR

I created https://issues.apache.org/jira/browse/HIVE-13119 Title: "java.io.IOException: java.io.IOException: ORC does not support type conversion from CHAR to VARCHAR (due to Schema Evolution HIVE-11981)?" Thanks for reporting the issue. From: Mich Talebzadeh

Re: /tmp/hive/hive is exceeded: limit=1048576 items=1048576

> Is there a setting somewhere to automatically remove old temp files from >/tmp/hive/hive? Yes, in Falcon's data retention (FALCON-870 ) That's been backported as a hot-fix for those who need to enforce Calendar based retention policies from a

RE: /tmp/hive/hive is exceeded: limit=1048576 items=1048576

Actual, the dir got nothing but 1MM empty sub-dirs. I am sure the name node won’t like it. Ø hadoop fs -du -s /tmp/hive/hive 0 /tmp/hive/hive From: Frank Luo Sent: Monday, February 22, 2016 12:26 PM To: user@hive.apache.org Subject: /tmp/hive/hive is exceeded: limit=1048576 items=1048576 Is

Re: Add multiple users in admin role while running in hive configuration-reg

Hi Matheswaran, I believe existing admin can create a new role(admin) for other users via beeline: https://cwiki.apache.org/confluence/display/Hive/SQL+Standard+Based+Hive+Authorization#SQLStandardBasedHiveAuthorization-RoleManagementCommands On Mon, Feb 22, 2016 at 3:12 AM, mathes waran wrote:

/tmp/hive/hive is exceeded: limit=1048576 items=1048576

Is there a setting somewhere to automatically remove old temp files from /tmp/hive/hive? Otherwise, every hive user will be facing the problem and everyone has to develop something to fix it, righ? This email and any attachments transmitted with it are intended for use by the intended recipien

Hive 2 error,: ORC does not support type conversion from CHAR to VARCHAR

New version Hive 2 a simple Job fails with 2016-02-22T18:23:29,008 ERROR [main]: mr.MapredLocalTask (MapredLocalTask.java:executeInProcess(381)) - Hive Runtime Error: Map local work failed JAVA.IO.IOEXCEPTION: JAVA.IO.IOEXCEPTION: ORC DOES NOT SUPPORT TYPE CONVERSION FROM CHAR TO VARCHAR Th

Re: Hive release 2.0 beeline authentication issue

Ok sorted it out :( In hive-site.xml need to set the parameter hive.server2.enable.doAs to false hive.server2.enable.doAs FALSE Setting this property to true will have HiveServer2 execute Hive operations as the user making the calls to it. And then restart hive server2 and co

Hive release 2.0 beeline authentication issue

Hi, Just upgraded from Hive 1.2.1 to Hive 2. Hive connection works OK and Hive server started OK as well. When I use beeline command below it used to work in Hive 1.2.1 However, now I get this error message: BEELINE -U JDBC:HIVE2://RHES564:10010/DEFAULT ORG.APACHE.HIVE.JDBC.HIVEDRIVER -N H

divide column by its sum

select c, sum(c) from foo group by c; obviously does not work. there are 2 ways that i could find. 1) find out the sum in a separate query and hard code the sum in a new query. this obviously performs 2 passes over the table. 2) select a, a/sum(a) over() from foo group by a; how many passes over

Add multiple users in admin role while running in hive configuration-reg

Hi, I had set the property of set.users.admin.role as hive.users.in.admin.role user1,user2 Comma separated list of users who are in admin role for bootstrapping. More users can be added in ADMIN role later. in hive-site.xml. If i want to add another users,stop the pro