Thanks for your answers, Jonathan, yes it was load avg and iowait was lower than 2% all that time - the only load was the user one.
Robert, we had -Xmx4012m which was automatically calculated by the default cassandra-env.sh (1/4 of total memory - 16G) - we didn't change that. 2013/12/5 Robert Coli <rc...@eventbrite.com> > On Thu, Dec 5, 2013 at 4:33 AM, Alexander Shutyaev <shuty...@gmail.com>wrote: > >> Cassandra version is 2.0.3. ... We've plugged it into our production >> environment as a cache in front of postgres. >> > > https://engineering.eventbrite.com/what-version-of-cassandra-should-i-run/ > > >> What can be the reason? Can it be high number of tables? Do we need to >> adjust some settings for this setup? Is it ok to have so many tables? >> Theoretically we can stuck them all in 3-4 tables. >> > > 500 column families ("tables") is "a lot" but not necessarily "too many." > > During that time cluster read latency goes from 2ms to 200ms >> > > In perfectly normal operation, things operating in the JVM may exhibit a > few hundreds of milliseconds of latency, when the JVM pauses for garbage > collection. If you absolutely require consistent latency below this time > for all requests, something running in the JVM may not be for you. > > If compaction triggers "runaway GC" then you probably have too much steady > state heap consumption. You don't mention anything about the hardware > environment, how much heap is available, etc.. ? > > =Rob >
