In that situation I might turn the setting off entirely. You would guess that the common use case is not "really small datasets" better the query to take longer then to oom a machine.
On Tue, Feb 5, 2013 at 3:12 AM, Bennie Schut <bsc...@ebuddy.com> wrote: > Hi, > > > > Just in case anyone else ever runs into this. > > Lately our cluster kept on killing itself with an OOM message in the kernel > log. It took me a while to realize why this happened since no single process > was causing it. > > I traced it back to a few queries running concurrently on a really small > datasets. This caused all of these queries to run localmode. Then I realized > there isn’t a limit to how many queries can run in localmode and since they > use the normal hadoop memory settings it’s pretty easy to hit OOM on a > machine this way. > > I’m not sure about the long term solution (some kind of limit on the number > of localmode processes), but for now I’ll probably disable localmode on > these queries. > > > > Bennie.
