looks like you're OOMing trying to compact a very large row. solution: smaller rows, or larger heap.
On Fri, Sep 24, 2010 at 3:03 PM, Alaa Zubaidi <alaa.zuba...@pdf.com> wrote: > Nothing is working, after disabling swap entirely, the heap is not > exhausted but Cassandra crashed with out of memory error. > I even slowed down the reading and writing, tried access mode=standard > nothing is working. > am I using Cassandra wrong or is it Windows? > > Any help is appreciated? > Alaa > > On 9/21/2010 6:56 PM, Jonathan Ellis wrote: >> >> Disabling swap entirely is usually the easiest fix, yes. >> >> On Mon, Sep 20, 2010 at 8:10 PM, Alaa Zubaidi<alaa.zuba...@pdf.com> >> wrote: >>> >>> Thanks Peter, >>> I decreased the heap size, it did not help, however, it delayed the >>> problem. >>> I noticed that its swapping, so, do you think that I should set windows >>> to >>> Not to swap? >>> >>> Do you think its related to this issue? >>> https://issues.apache.org/jira/browse/CASSANDRA-1014 >>> >>> Thanks, >>> Alaa >>> >>> On 9/18/2010 7:04 AM, Peter Schuller wrote: >>>>> >>>>> I see a spike in heap memory usage on Node 2 where it goes from around >>>>> 1G >>>>> to >>>>> 6GB (max) in less than an hour, and then goes our of memory. >>>>> There are some errors in the log file that are reported by other >>>>> people, >>>>> but >>>>> I don't think that these errors are the reason, because it use to >>>>> happen >>>>> even before I have seen them. >>>>> >>>>> Can someone help me understand whats going on?? >>>> >>>> Only partially: >>>> >>>> 54 seconds ParNew GC:s are wild and crazy: >>>> >>>> INFO [GC inspection] 2010-09-17 14:53:59,403 GCInspector.java (line >>>> 129) GC for ParNew: 54095 ms, 53297952 reclaimed leaving 4712568360 >>>> used; max is 6563430400 >>>> >>>> Is the machine swapping? >>>> >>>> I noticed there is hinted hand-off activity going on. Maybe that is a >>>> result of nodes dropping in and out due to swapping. In any case, you >>>> definitely don't want to have the machine swapping to death. I'm not >>>> sure what the best way is to avoid this on Windows, other than >>>> decreasing heap size. >>>> >>>> The repeated exceptions in your log aren't normal as far as I know. I >>>> don't remember, but IIRC the UTF-8 encoding issues can be caused by >>>> changing partitioner after inserting data (but I'm not sure). >>>> >>>> With respect to memory use, you don't seem to be inserting so much >>>> data for bloom filters and sstable index samples to be a problem. >>>> Memtable flushing could cause problems if they're happening too slowly >>>> (maybe plausible with swapping) - except that the stage statistics >>>> don't indicate there are lots of memtables in memory waiting to be >>>> flushed, so that shouldn't be it. >>>> >>>> Hinted handoff maybe, but I don't remember whether hinted handoff has >>>> the potential to accumulate data in RAM (no time to check now). >>>> >>>> Regardless, I'd recommend fixing any swapping issues you have before >>>> trying to draw conclusions about performance. And you don't want those >>>> exceptions in your logs. >>>> >>> -- >>> Alaa Zubaidi >>> PDF Solutions, Inc. >>> 333 West San Carlos Street, Suite 700 >>> San Jose, CA 95110 USA >>> Tel: 408-283-5639 (or 408-280-7900 x5639) >>> fax: 408-938-6479 >>> email: alaa.zuba...@pdf.com >>> >>> >>> >> >> > > -- > Alaa Zubaidi > PDF Solutions, Inc. > 333 West San Carlos Street, Suite 700 > San Jose, CA 95110 USA > Tel: 408-283-5639 (or 408-280-7900 x5639) > fax: 408-938-6479 > email: alaa.zuba...@pdf.com > > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com
