Re: wide ORC partitioned table insert causes "GC Overhead Limit Exceeded" error

2017-05-24 Thread Prasanth Jayachandran
ay, May 24, 2017 10:53 PM To: user@hive.apache.org Subject: wide ORC partitioned table insert causes "GC Overhead Limit Exceeded" error I get the "GC Overhead Limit Exceeded" and “java heap space” errors when trying to insert from Stage table to wide (~2500 columns)

RE: wide ORC partitioned table insert causes "GC Overhead Limit Exceeded" error

2017-05-24 Thread Revin Chalil
Forgot to mention that we use hive 1.2. From: Revin Chalil [mailto:rcha...@expedia.com] Sent: Wednesday, May 24, 2017 10:53 PM To: user@hive.apache.org Subject: wide ORC partitioned table insert causes "GC Overhead Limit Exceeded" error I get the "GC Overhead Limit Exceeded&

wide ORC partitioned table insert causes "GC Overhead Limit Exceeded" error

2017-05-24 Thread Revin Chalil
I get the "GC Overhead Limit Exceeded" and "java heap space" errors when trying to insert from Stage table to wide (~2500 columns) ORC partitioned table with dynamic partitioning. It consistently fails with TEZ and but succeeds with lesser data in "MR" execution engine. We have been trying to ad