Hive Alter

Hi Team, I am trying to write my query results to a file. hive -e "alter table {0}.{1} partition(year='{2}',month='{3}',day='{4}',hour='{5}') CONCATENATE > /tmp/srinivas/test.log;"'.format(row[0],row[1],row[2],row[3],row[4],row[5])

Re: Hive alter table concatenate loses data - can parquet help?

Thank you very much for thinking of this. I do not have such files. I will file a bug as per your suggestion. On Monday, March 14, 2016, Prasanth Jayachandran < pjayachand...@hortonworks.com> wrote: > Hi Marcin > > I came across this issue recently. Do you have old orc files (created with > hive

Re: Hive alter table concatenate loses data - can parquet help?

Hi Marcin I came across this issue recently. Do you have old orc files (created with hive 0.11) in the table/partition? If so this patch is required https://issues.apache.org/jira/browse/HIVE-13285 Thanks Prasanth On Mar 10, 2016, at 5:02 PM, Prasanth Jayachandran mailto:pjayachand...@hortonw

Re: Hive alter table concatenate loses data - can parquet help?

After hive 1.2.1 there is one patch that went in related to alter table concatenation. https://issues.apache.org/jira/browse/HIVE-12450 I am not sure if its related though. Could you please file a bug for this? It will be great if you can attach a small enough repro for this issue. I can verify

Re: Hive alter table concatenate loses data - can parquet help?

Hi Mich, ddl as below. Hi Prasanth, Hive version as reported by Hortonworks is 1.2.1.2.3. Thanks, Marcin CREATE TABLE ``( `col1` string, `col2` bigint, `col3` string, `col4` string, `col4` string, `col5` bigint, `col6` string, `col7` string, `col8` string, `col9`

Re: Hive alter table concatenate loses data - can parquet help?

Hi Marcin What hive version are you using? There has been some fixes to concatenate lately. I will let you know if your hive version contains all fixes. Thanks Prasanth On Mar 8, 2016, at 3:29 AM, Mich Talebzadeh mailto:mich.talebza...@gmail.com>> wrote: Hi can you please provide DDL for thi

Re: Hive alter table concatenate loses data - can parquet help?

Hi can you please provide DDL for this table "show create table " Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpre

Hive alter table concatenate loses data - can parquet help?

Hi All, Following on from from our parquet vs orc discussion, today I observed hive's alter table ... concatenate command remove rows from an ORC formatted table. 1. Has anyone else observed this (fuller description below)? And 2. How to do parquet users handle the file fragmentation issue? Desc

Re: Hive Alter Partition Location Issue

endorsed by Peridale Ltd, its >> subsidiaries or their employees, unless expressly so stated. It is the >> responsibility of the recipient to ensure that this email is virus free, >> therefore neither Peridale Ltd, its subsidiaries nor their employees accept &g

Re: Hive Alter Partition Location Issue

d >> recipient, you should destroy it immediately. Any information in this >> message shall not be understood as given or endorsed by Peridale Ltd, its >> subsidiaries or their employees, unless expressly so stated. It is the >> responsibility of the recipient to ensure

Re: Hive Alter Partition Location Issue

given or endorsed by Peridale Ltd, its > subsidiaries or their employees, unless expressly so stated. It is the > responsibility of the recipient to ensure that this email is virus free, > therefore neither Peridale Ltd, its subsidiaries nor their employees accept > any responsibil

RE: Hive Alter Partition Location Issue

its subsidiaries nor their employees accept any responsibility. From: Harsha N [mailto:harsha.hadoo...@gmail.com] Sent: 30 April 2015 07:24 To: user@hive.apache.org Subject: Hive Alter Partition Location Issue Hi All, Can experts share your view on Hive behaviour in below scenario. I am f

Hive Alter Partition Location Issue

Hi All, Can experts share your view on Hive behaviour in below scenario. I am facing below issue on using alter partition locations in hive. *select count(*) from table1 where dt = 201501;* *Total jobs = 1* *Launching Job 1 out of 1* *Number of reduce tasks determined at compile time: 1* *In or

Re: Hive - Alter column datatype

Changing the datatype of a column will *not* alter the column's data itself - just Hive's metadata for that table. To modify the type of existing data: 1. Create a new table with the desired structure 2. Copy the existing table into the new table - applying any necessary type casting 3. Drop

Re: Hive - Alter column datatype

let's put it this way. what makes you ask the question that altering the datatype wouldn't work? After all that's why it's there. :) On Thu, Jul 18, 2013 at 2:45 AM, Manickam P wrote: > Hi experts, > > I have created a table in hive and loaded the data into it. now i want to > change the dat

Hive - Alter column datatype

Hi experts, I have created a table in hive and loaded the data into it. now i want to change the datatype of one particular column. Do i need to drop and move the file again to hive?will it work fine if i just alter the data type alone in hive? Thanks,Manickam P

Re: hive alter partition error

Hi All, When creating a hive partition it is actually creating a dir under the table name which will be called the partition name. It needs to have the right permissions to create this dir. In this case, Creating the partition with the user web_front will work. Thanks Hortonworks, Inc. Te

Re: Please HELP: HIVE alter table add new partition to schema...

ll-files-problem/ > > http://blog.rapleaf.com/dev/2008/11/20/give-me-liberty-or-give-me-death-but-dont-give-me-small-files/ > > In general, you would like your file sizes to be atleast of HDFS block > size, most likely a small multiple of the block size. If you do find that > you

Re: Please HELP: HIVE alter table add new partition to schema...

are running into the small files problem, there are other ways to get around like bucketing. Good luck! Mark - Original Message - From: "Edward Capriolo" To: user@hive.apache.org Sent: Tuesday, June 19, 2012 11:12:48 AM Subject: Re: Please HELP: HIVE alter table add new parti

Re: Please HELP: HIVE alter table add new partition to schema...

You can not change the partition columns. I would use a dynamic partition insert to select all the data from the original table into the new table. On 6/19/12, Tim Havens wrote: > So...I have a table that has thousands of files, and Billions of rows > related it. > > Lets make this a simple table

Please HELP: HIVE alter table add new partition to schema...

So...I have a table that has thousands of files, and Billions of rows related it. Lets make this a simple table: CREATE TABLE test_table ( ts BIGINT, exec_time DOUBLE, domain_id BIGINT, domain_name STRING, ) PARTITIONED BY (logdate STRING, source STRING, datacenter STRING, hostnam

Hive alter table SerDe doesn't work in hive CDH3 for me

Hi all: hive alter serde seems doesn't work in CDH3u2 Hadoop and CDH3 Hive 1. create table with wrong regex: CREATE TABLE sample1(key string, value string) row format serde 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe' with serdeproperties (“input.regex”=”[\\w] is [\\w]”) 2. i