Re: Metastore performance on HDFS-backed table with 15000+ partitions

2014-02-27 Thread Norbert Burger
file a bug report. > > bottom line hiveserver2 isn't production bullet proof just yet, IMHO. > Others may disagree. > > Regards, > Stephen. > > > > On Sat, Feb 22, 2014 at 9:50 AM, Norbert Burger > wrote: > >> Thanks all for the quick feedback. >&

Re: Metastore performance on HDFS-backed table with 15000+ partitions

2014-02-22 Thread Norbert Burger
On Saturday, February 22, 2014, Terje Marthinussen < > tmarthinus...@gmail.com> wrote: > > Query optimizer in hive is awful on memory consumption. 15k partitions > sounds a bit early for it to fail though.. > > > > What is your heap size? > > > &g

Metastore performance on HDFS-backed table with 15000+ partitions

2014-02-21 Thread Norbert Burger
Hi folks, We are running CDH 4.3.0 Hive (0.10.0+121) with a MySQL metastore. In Hive, we have an external table backed by HDFS which has a 3-level partitioning scheme that currently has 15000+ partitions. Within the last day or so, queries against this table have started failing. A simple query