Re: Tables missing on the file system

2015-09-14 Thread Chetna C
Hi Ravi, Please make sure the location mentioned while create table exists at the time of *'MSCK REPAIR'*. This error occurs, if location does not exists on fs. Thanks, Chetna Chaudhari On 15 September 2015 at 12:03, ravi teja wrote: > Hi, > I am getting this exception when I repair a table

Tables missing on the file system

2015-09-14 Thread ravi teja
Hi, I am getting this exception when I repair a table. Not sure what this means, didnt get any info while searching also. Can someone guide , what this means? CREATE EXTERNAL TABLE IF NOT EXISTS test_table OK Time taken: 0.124 seconds MSCK REPAIR TABLE test_table OK Tables missing o

Re: Re: binary column data consistency in hive table copy

2015-09-14 Thread xihuyu2000
if use CTAS then a MR job occures. Maybe the problem is in the MR job. 2015-09-15 xihuyu2000 发件人:Jason Dere 发送时间:2015-09-15 06:00 主题:Re: binary column data consistency in hive table copy 收件人:"user@hive.apache.org" 抄送: Looks like your table is using text storage format. Binary data needs to

Re: Decomposing nested Hive statements with views

2015-09-14 Thread Gopal Vijayaraghavan
> We have many HQL scripts that select from nested sub-selects. In many >cases the nesting can be a few levels deep: ... > Such queries are difficult to test as they are complex monoliths. While >we can decompose and modularise them at the column level with UDFs and >macros, it is not as evident t

Re: binary column data consistency in hive table copy

2015-09-14 Thread Jason Dere
Looks like your table is using text storage format. Binary data needs to be stored as base64 in TextInputformat, so those values are probably being interpreted as base64 strings. From: Ujjwal Wadhawan Sent: Monday, September 14, 2015 2:32 PM To: user@hive.apach

binary column data consistency in hive table copy

2015-09-14 Thread Ujjwal Wadhawan
Hi all, I recently observed a behavior in hive that I’ll like to share and get inputs. *Scenario:* Say you have a hive table with a binary column. create table binsource (bincol binary); and some input data $ cat /nis3/home/ujjwal2/test2/binin 1101 121 10 1011 Asfs L

Decomposing nested Hive statements with views

2015-09-14 Thread Elliot West
Hello, We have many HQL scripts that select from nested sub-selects. In many cases the nesting can be a few levels deep: SELECT ... FROM ( SELECT ... FROM ( SELECT ... FROM ( SELECT ... FROM a WHERE ... ) A LEFT JOIN ( SELECT ... FROM b ) B ON (

Re: Organising Hive Scripts

2015-09-14 Thread Elliot West
Hi Charles, You can also split out column level logic using Hive macros. These also allow re-use of said logic: hive> create temporary macro MYSIGMOID(x DOUBLE) > 2.0 / (1.0 + exp(-x)); OK hive> select MYSIGMOID(1.0) from dual; OK 1.4621171572600098 Cheers - Elliot. On 11 September 2015

Re: Organising Hive Scripts

2015-09-14 Thread Daniel Lopes
Thanks Erwan, I needed this too, and works fine for me. *Daniel Lopes, B.Eng* Data Scientist - BankFacil CREA/SP 5069410560 Mob +55 (18) 99764-2733 Ph +55 (11) 3522-8009 http://about.me/dannyeuu Av. Nova Independênci