Re: Troubleshooting JVM OOM during Spark Unit Tests

Here’s that log file from a different run of the unit tests that also failed. I’m not sure what to look for. If it matters any, I also changed JAVA_OPTS as follows for this run: export JAVA_OPTS="-Xms512m -Xmx1024m -XX:PermSize=64m -XX:MaxPe

Re: Troubleshooting JVM OOM during Spark Unit Tests

What does /tmp/jvm-21940/hs_error.log tell you? It might give hints to what threads are allocating the extra off-heap memory. On Fri, Nov 21, 2014 at 1:50 PM, Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > Howdy folks, > > I’m trying to understand why I’m getting “insufficient memory” e

Troubleshooting JVM OOM during Spark Unit Tests

Howdy folks, I’m trying to understand why I’m getting “insufficient memory” errors when trying to run Spark Units tests within a CentOS Docker container. I’m building Spark and running the tests as follows: # build sbt/sbt -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -Pkinesis-asl -Phive -Phive-th