Hi
Found that in my config there was
hive.exec.dynamic.partition = true;
I turned it false and most of the times I can create table now but not
every time.
Margus (margusja) Roo
http://margus.roo.ee
skype: margusja
+372 51 48 780
On 29/02/16 09:27, Margus Roo wrote:
Hi
Can someone confirm that Hive checks files in destination directory
before creating external table?
At the moment in Hive 1.2.1 end user can just easily kill whole Hive
server creating external table and pointing to directory where are
loads of files.
Margus (margusja) Roo
http://margus.roo.ee
skype: margusja
+372 51 48 780
On 26/02/16 17:27, Margus Roo wrote:
Basically the question is:
Does Hive checks files in location before creating table?
Because if I move files away before creating tables it works and
after table is created I can move files back and all works :)
Margus (margusja) Roo
http://margus.roo.ee
skype: margusja
+372 51 48 780
On 26/02/16 16:40, Margus Roo wrote:
Hi
I try to create external table and in the location there are 8960
small files
And I am getting every time something like that:
GC pool 'PS MarkSweep' had collection(s): count=1 time=1672ms
GC pool 'PS Scavenge' had collection(s): count=1 time=45ms
2016-02-26 15:18:29,721 INFO
[org.apache.hadoop.util.JvmPauseMonitor$Monitor@3af36922]:
util.JvmPauseMonitor (JvmPauseMonitor.java:run(195)) - Detected
pause in JVM or host machine (eg GC): pause of approximately 1995ms
GC pool 'PS MarkSweep' had collection(s): count=1 time=1936ms
And all 16 cpu cores are in 100% and all 16G memory almost gone.
Helps only if I restart hive server.
I use Hive 1.2.1 from HDP-2.3 by packaged by Hortonworks
hive.tez.java.opts=-server -Djava.net.preferIPv4Stack=true
-XX:NewRatio=8 -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB
-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps
