SemanticException Partition spec {p1=null, p2=null} contains non-partition columns

2015-08-21 Thread Buntu Dev
I'm running into this error while doing a dynamic partition insert. Heres how I created the table: CREATE TABLE `part_table`( `c1` bigint, `c2` bigint, `c3` bigint) PARTITIONED BY (p1 string, `p2` string) STORED AS PARQUET; Here is the insert table: SET hive.exec.dynamic.pa

RE: Partition Columns

2015-05-15 Thread Martin Benson
you bury the partition columns in a derived field like in Query 2 it is unable to spot that and so does a full table scan. I think (but don’t know for sure) that this will be fairly typical of all SQL engines. Your best bet is to use direct conditions like in Query 1. In this case it may have

Re: Partition Columns

2015-05-14 Thread Appan Thirumaligai
;> Thanks, > >> Viral > >> > >> > >> > >> On Thu, May 14, 2015 at 1:48 PM, Appan Thirumaligai > >> wrote: > >>> > >>> Hi, > >>> > >>> I have a question on Hive Optimizer. I have a t

Re: Partition Columns

2015-05-14 Thread Mungeol Heo
hat version of Hive are you using ? >> >> Thanks, >> Viral >> >> >> >> On Thu, May 14, 2015 at 1:48 PM, Appan Thirumaligai >> wrote: >>> >>> Hi, >>> >>> I have a question on Hive Optimizer. I have a table with pa

Re: Partition Columns

2015-05-14 Thread Appan Thirumaligai
27;s not a straight up filtering and involves using functions (aka > UDFs). > > What version of Hive are you using ? > > Thanks, > Viral > > > > On Thu, May 14, 2015 at 1:48 PM, Appan Thirumaligai > wrote: > >> Hi, >> >> I have a question on Hive O

Re: Partition Columns

2015-05-14 Thread Viral Bajaria
i, > > I have a question on Hive Optimizer. I have a table with partition columns > eg.,Sales partitioned by year, month, day. Assume that I have two years > worth of data on this table. I'm running two queries on this table. > > Query 1: Select * from Sales where year=2015 an

Partition Columns

2015-05-14 Thread Appan Thirumaligai
Hi, I have a question on Hive Optimizer. I have a table with partition columns eg.,Sales partitioned by year, month, day. Assume that I have two years worth of data on this table. I'm running two queries on this table. Query 1: Select * from Sales where year=2015 and month = 5 and day betw

Re: analyze hive tables with null values in partition columns

2014-04-28 Thread Dileep Kumar
> I have a table that has partition based on column ss_sold_date_sk which > has null value partition as well. > > When I run the analyze ..compute stat it fails with attached exception. > Is there a way to avoid this or bypass this exception, also what would be > the impact on query performance of