RE: Mapper OOMs disappear after disabling JVM reuse

2011-04-08 Thread Steven Wong
] Sent: Friday, April 08, 2011 9:50 AM To: user@hive.apache.org Cc: Steven Wong Subject: Re: Mapper OOMs disappear after disabling JVM reuse I had a similar problem until I set this parameter to 1 (although 3 seems to work fine too). There is an explanation somewhere on the web. Basically, if you run

Re: Mapper OOMs disappear after disabling JVM reuse

2011-04-08 Thread Igor Tatarinov
I had a similar problem until I set this parameter to 1 (although 3 seems to work fine too). There is an explanation somewhere on the web. Basically, if you run 20 tasks and the garbage collector cannot catch up with accumulated garbage, the java process grows too big so when it finally decides to