TEZ log missing

Hello,all:       【My environment versions are ：Hadoop 2.6.0 、hive 1.2.1、tez 0.7.0】 Now I executed a HQL “select count(*) from h_im;”(h_im is an external table, hbase table) in hive CLI, it throw exceptions: (I am sorry， I can not copy the error information here, because we u

Re: Grouping sets with table alias causes parse exception

I found that it is already well described and fixed at Hive 1.2. Thanks! Parsing Error in GROUPING SETS https://issues.apache.org/jira/browse/HIVE-6950 2016-01-18 19:28 GMT+09:00 goun na : > Hi, Users > > While converting legacy Oracle SQL to HiveQL using Grouping set, I found > an interesting

RE: ORC table stats from orcfiledump

Hi Prasanth, In your statement below “ORC will keep buffering the rows in memory. Orc has its own memory manager which will monitor the memory consumption every 5000 rows. If the memory consumption exceeds the stripe size (in this case 16MB) then it will flush the stripe. “ I gather

RE: File search by hashes in Hadoop

Thanks Ritesh I see there are two options here 1.Use UNIX like commands on hdfs to find the relevant files hdfs dfs -ls -R |grep sales drwxr-xr-x - hduser supergroup 0 2015-12-27 06:02 sales -rw-r--r-- 2 hduser supergroup 0 2015-12-27 06:02 sales/_SUCCESS

Re: January Hive User Group Meeting

For those who cannot attend in person, here is the webex info: https://cloudera.webex.com/meet/xzhang 1-650-479-3208 Call-in toll number (US/Canada) 623 810 662 (access code) Thanks, Xuefu On Wed, Jan 20, 2016 at 9:45 AM, Xuefu Zhang wrote: > Hi all, > > As a reminder, the meeting will be hel

Re: HIVE CLI does not escape \t ?

>I use the workaround cat * >> output.tsv but that's not ideal. > >Any way to constrain the number of files to 1 automatically? I generally use an "ORDER BY 0" to insert a single reducer, which produces exactly 1 file. This is generally not a problem if you have say, <= 1 million rows. HDFS all

Re: File search by hashes in Hadoop

Yes, it's possible to do both 1. Index based searching : http://www.drdobbs.com/parallel/indexing-and-searching-on-a-hadoop-distr/226300241?pgno=3 2. Wildcard based / Expression based searching : https://stackoverflow.com/questions/6297533/search-find-a-file-and-file-content-in-hadoop Thanks, *Ri

hide few columns of a table

Hi Is it possible to retstrict access on few columns of a table using view on top of table and expose allowed columns in view . But making table invisible in select queries and show tables queries while view is visible? Thanks

Re: Synchronizing Hive metastores across clusters

Following up on this: I've spent some time trying to evaluate the Hive replication features but in truth it's more been an exercise in trying to get them working! I thought I'd share my findings: - Conceptually this feature can sync (nearly) all Hive metadata and data changes between two clu

Stored By

Hi, So I am using the AccumuloStorageHandler to allow me to access Accumulo tables from Hive. This works fine. So typically I would use something like this: CREATE EXTERNAL TABLE test_text (rowid STRING, testint INT, testbig BIGINT, testfloat FLOAT, testdouble DOUBLE, teststring STRING, testboo

File search by hashes in Hadoop

Hi all, Apologies for the nature of this question. Someone asked me whether it is possible to perform file search by hashes in Hadoop. I am thinking that he means wildcard searches in HDFS? Anyone has ideas what file search by hash means in Hadoop? regards, Mich

Hive UDF and Avro table

Hello, I have a simple Hive UDF that works fine when executed against an external table that uses test files, but no result when used with an external table that uses Avro Snappy files. Does there any docuementation about that ? or what i have to change ? Cordially.

hive on tez serialization and deserialization( custom Serde initialize() just called one time in hive client, when split in AM)

Hello,all:       【My environment versions are ：Hadoop 2.6.0 、hive 1.2.1、tez 0.7.0】 Our term develop a plug-in in hive, its function is similiar to hive-hbase-handler. Now I executed a HQL “select count(*) from h_im;”(h_im is an external table, hbase table) in hive CLI, it thro

RE: HIVE CLI does not escape \t ?

Thanks Gopal I tried that but it generates as many files as mappers (no reducers in this query). Tried: - set hive.exec.max.created.files=1; does not work, execution stops after more than 1 file is created - set mapred.map.tasks=1; does not work either, still using 35 mappers I use the workar