Hi all, Found a very useful guide online. Link -> http://it-ebooks.info/book/941/
Richa On Tue, Jun 11, 2013 at 1:53 PM, Richa Sharma <mailtorichasha...@gmail.com>wrote: > Thanks for sharing! > > I looked at these links .. Is there any documentation with more examples > with both static and dynamic partitions covered together. > > Richa > > > On Tue, Jun 11, 2013 at 12:33 PM, Lefty Leverenz <le...@hortonworks.com>wrote: > >> Dynamic partitions are described in the Hive design docs here: >> https://cwiki.apache.org/confluence/display/Hive/DynamicPartitions. >> >> For the configuration parameters, though, you need to look in the >> language manual here: >> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties >> (search >> for "dynamic" to find various parameters related to dynamic partitions). >> >> – Lefty >> >> >> >> On Mon, Jun 10, 2013 at 7:06 AM, Owen O'Malley <omal...@apache.org>wrote: >> >>> You need to create the partitioned table and then copy the rows into it. >>> >>> create table foo_staging (int x, int y); >>> >>> create table foo(int x) partitioned by (int y) clustered by (x) into 16 >>> buckets; >>> >>> set hive.exec.dynamic.partition=true; >>> set hive.exec.dynamic.partition.mode=nonstrict; >>> set hive.enforce.bucketing = true; >>> >>> insert overwrite table partition (y) select * from foo_staging; >>> >>> >>> On Mon, Jun 10, 2013 at 6:38 AM, Nitin Pawar <nitinpawar...@gmail.com>wrote: >>> >>>> If a table is not partitioned and then you want to partition the table >>>> on the data already written but data is not in partition format, that is >>>> not doable. >>>> >>>> Best approach would be, create a new table definition with the >>>> partition columns you want. >>>> turn on the dynamic partitioning system before you load data into new >>>> table >>>> >>>> set hive.exec.dynamic.partition=true; >>>> set hive.exec.dynamic.partition.mode=nonstrict; >>>> >>>> insert overwrite table partitioned(columns) select * from oldtable >>>> >>>> >>>> remove old table >>>> >>>> PS: wait for others to add more suggestions. I may be very well wrong >>>> in suggesting this >>>> >>>> >>>> On Mon, Jun 10, 2013 at 7:01 PM, Peter Marron < >>>> peter.mar...@trilliumsoftware.com> wrote: >>>> >>>>> Hi,**** >>>>> >>>>> ** ** >>>>> >>>>> Using hive 0.10.0 over hadoop 1.0.4**** >>>>> >>>>> ** ** >>>>> >>>>> I have a (non-partitioned) table with loads of columns.**** >>>>> >>>>> I would like to create a partitioned table with the same set of >>>>> columns.**** >>>>> >>>>> So the approach that I have been taking is to use “CREATE TABLE copy >>>>> LIKE original;”**** >>>>> >>>>> then I can use ALTER TABLE to change the location and the INPUTFORMAT* >>>>> *** >>>>> >>>>> and the OUTPUTFORMAT and the SERDE and properties and pretty much**** >>>>> >>>>> everything else. However I don’t seem to be able to make it >>>>> partitioned.**** >>>>> >>>>> Sure I can add partitions if it’s already partitioned but I don’t seem >>>>> **** >>>>> >>>>> to be able to make it partitioned if it’s not already. I get errors >>>>> like this:**** >>>>> >>>>> ** ** >>>>> >>>>> hive> ALTER TABLE customerShortValues ADD PARTITION (aid='1') LOCATION >>>>> 'E7/phase2/values/aid=1';**** >>>>> >>>>> FAILED: Error in metadata: table is not partitioned but partition spec >>>>> exists: {aid=1}**** >>>>> >>>>> FAILED: Execution Error, return code 1 from >>>>> org.apache.hadoop.hive.ql.exec.DDLTask**** >>>>> >>>>> ** ** >>>>> >>>>> So, I guess that I could create the table I want by hand copying over >>>>> all the**** >>>>> >>>>> column definitions. But is there an easier way?**** >>>>> >>>>> ** ** >>>>> >>>>> Z**** >>>>> >>>> >>>> >>>> >>>> -- >>>> Nitin Pawar >>>> >>> >>> >> >
