Hi Peter,
There are other parameters work with hive.auto.convert.join. Please
check "hive.hashtable.max.memory.usage"
and "hive.smalltable.filesize". They're quite self-explanatory but if you
need more details you can check the auto mapjoin JIRA ticket
https://issues.apache.org/jira/browse/HIVE-16
Using Hive 0.8.1 on Amazon EMR Hadoop Job.
Some problems with using mapjoin:
1) Exceed memory, I got the following errors. Then I remove mapjoin in the
query and instead set hive.auto.convert.join=true, thinking that let hive
decides when mapjoin is suitable. It does run much farther in the job
Hi there,
I am trying the following:
select case when ta_end_datetime_berekenen = 'Y' then lead(ta_update_datetime)
over ( partition by dn_waarde_van, dn_waarde_tot order by ta_update_datetime )
else ea_end_datetime end ea_end_datetime, ta_insert_datetime,
ta_update_datetime from tmp_wtdh_bes
if you have a 250GB file, then how did it become 2.5TB ?
if you can not write a mapreduce job to process to write your data into
specific partitions, then the other way around would be load the entire
data into a temporary table and then load data into partitioned tabled and
then drop the temporar
Nitin
I am still confused, from the below data that i have given should the file
which sits in the folder Country=USA and state=IL have only the rows where
Country=USA and state=IL or will it have rows of other countries also.
The reason i ask is because if we have a 250GB file and would like to
when you specify the load data query with specific partition, it will put
the entire data into that partition.
On Mon, May 27, 2013 at 1:08 PM, Sai Sai wrote:
>
> After creating a partition for a country (USA) and state (IL) and when we
> go to the the hdfs site to look at the partition in the
After creating a partition for a country (USA) and state (IL) and when we go to
the the hdfs site to look at the partition in the browser we r seeing all the
records for all the countries and states rather than just for the partition
created for US and IL given below, is this correct behavior
Thanks Navis
From: Navis류승우
To: user@hive.apache.org; Sai Sai
Sent: Monday, 27 May 2013 12:15 PM
Subject: Re: Issue with Json tuple lateral view
Removing last ',' in second row would make result as you expected.
I can't tell it's bug or not.
2013/5/27 Sai
