Re: Impact of partitioning on certain queries

2016-01-08 Thread Jörn Franke
y of the recipient to ensure that this email is virus free, > therefore neither Peridale Ltd, its subsidiaries nor their employees accept > any responsibility. > > From: Jörn Franke [mailto:jornfra...@gmail.com] > Sent: 08 January 2016 08:49 > To: user@hive.apache.org >

RE: Impact of partitioning on certain queries

2016-01-08 Thread Mich Talebzadeh
It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility. From: Jörn Franke [mailto:jornfra...@gmail.com] Sent: 08 January 2016 09:54 To: user@hive.apache.org Subject: Re:

Re: Impact of partitioning on certain queries

2016-01-08 Thread Jörn Franke
free, > therefore neither Peridale Ltd, its subsidiaries nor their employees accept > any responsibility. > > > -Original Message- > From: Gopal Vijayaraghavan [mailto:go...@hortonworks.com] On Behalf Of Gopal > Vijayaraghavan > Sent: 08 January 2016 09:34 >

RE: Impact of partitioning on certain queries

2016-01-08 Thread Mich Talebzadeh
Of Gopal Vijayaraghavan Sent: 08 January 2016 09:34 To: user@hive.apache.org Subject: Re: Impact of partitioning on certain queries > Ok we hope that partitioning improves performance where the predicate >is on partitioned columns Nope. Partitioning *only* improves performa

Re: Impact of partitioning on certain queries

2016-01-08 Thread Gopal Vijayaraghavan
> Ok we hope that partitioning improves performance where the predicate is >on partitioned columns Nope. Partitioning *only* improves performance if your queries run with set hive.mapred.mode=strict; That's the "use strict" easy way to make sure you're writing good queries. Even then, schem

RE: Impact of partitioning on certain queries

2016-01-08 Thread Mich Talebzadeh
rn Franke [mailto:jornfra...@gmail.com] Sent: 08 January 2016 08:49 To: user@hive.apache.org Subject: Re: Impact of partitioning on certain queries Well you use a text format for your data so you should not be surprised. For text based formats, such as csv, you can always use the hive bitmap in

Re: Impact of partitioning on certain queries

2016-01-08 Thread Jörn Franke
sponsibility of the recipient to ensure that this email is virus free, > therefore neither Peridale Ltd, its subsidiaries nor their employees accept > any responsibility. > > From: Jörn Franke [mailto:jornfra...@gmail.com] > Sent: 08 January 2016 06:20 > To: user@hive.apache.or

RE: Impact of partitioning on certain queries

2016-01-08 Thread Mich Talebzadeh
neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility. From: Jörn Franke [mailto:jornfra...@gmail.com] Sent: 08 January 2016 06:20 To: user@hive.apache.org Subject: Re: Impact of partitioning on certain queries This observation is correct and it is the

Re: Impact of partitioning on certain queries

2016-01-07 Thread Jörn Franke
This observation is correct and it is the same behavior as you see it in other databases supporting partitions. Usually you should avoid many small partitions. > On 07 Jan 2016, at 23:53, Mich Talebzadeh wrote: > > Ok we hope that partitioning improves performance where the predicate is on >

Impact of partitioning on certain queries

2016-01-07 Thread Mich Talebzadeh
Ok we hope that partitioning improves performance where the predicate is on partitioned columns I have two tables. One a basic table called smallsales defined as below CREATE TABLE `smallsales`( | | `prod_id` bigint,