Thanks for clarifying Chris! On Tue, Mar 10, 2015 at 9:16 AM, Chris Riccomini <criccom...@apache.org> wrote:
> Hey Jordan, > > There are two settings you need to care about when changing memory. One is > yarn.container.memory.mb. The other is the task.opts setting, which will > allow you to specify a custom -Xmx parameter. > > If you set the yarn.container.memory.mb to 4000 (4GB), as you suggest, the > JAVA_OPTS check still takes affect, and your Xmx will remain 768MB. If you > wish to increase your heap, you'll need to set both > yarn.container.memory.mb *and* the task.opts setting to be something like > task.opts=-Xmx3g. This will tell YARN to kill any container that uses > 4GB > of physical memory on the machine. The Xmx tells the container not to use > more than 3GB of heap. > > It's a bit confusing, but there's a reason for this. The problem we have is > that we (Samza) don't know how much off-heap/non-JVM memory you're going to > use in your container. YARN only pays attention to the amount of physical > memory used by a process. With Java, you can set the heap, but there's also > permgen, JVM, JNI libraries, and off-heap memory usage. All of these > contribute to the physical memory usage that YARN cares about, but are > outside the JVM heap. This means that we can't just use one memory setting > for both YARN and Java. We have to have two. > > Cheers, > Chris > > On Tue, Mar 10, 2015 at 1:13 AM, Jordan Shaw <jor...@pubnub.com> wrote: > > > Hey Everyone, > > This I have a question somewhat related to SAMZA-109 and this line in > > run-class.sh: > > # Check if a max-heap size is specified. If not - set a 768M heap [[ > > $JAVA_OPTS != *-Xmx* ]] && JAVA_OPTS="$JAVA_OPTS -Xmx768M" > > > > If I were to set the container.memory.mb for yarn to 4GB ( > > yarn.container.memory.mb = 4096) the above JAVA_OPTS check would cause > this > > to be ignored right? I can understand why preventing the Heap from going > > crazy is important in these long running jobs but to me this might cause > > some confusion especially when trying to debug a thrashing GC and Java > > isn't committing the memory amount set in yarn.container.memory.mb. > > > > I was trying to follow why this was decided on in SAMZA-109 but nothing > > stuck out to me. Would it make more sense to just not specifying a max > heap > > and letting Yarn kill the job if it goes over it's specified allotment > > and/or let the user explicitly set these opts? Thanks! > > > > - Jordan > > > -- Jordan Shaw Full Stack Software Engineer PubNub Inc 1045 17th St San Francisco, CA 94107
