2012/6/8 aaron morton <aa...@thelastpickle.com>: > Ruslan, > Why did you suggest changing the disk_access_mode ?
Because this bring problems on empty seat, in any case for me mmap bring similar problem and i doesn't have find any solution to resolve it, only change disk_access_mode:-((. For me also will be interesting hear results of author of this theme > > Gurpreet, > I would leave the disk_access_mode with the default until you have a reason > to change it. > >> > 8 core, 16 gb ram, 6 data disks raid0, no swap configured > > is swap disabled ? > >> Gradually, >> > the system cpu becomes high almost 70%, and the client starts getting >> > continuous timeouts > > 70% of one core or 70% of all cores ? > Check the server logs, is there GC activity ? > check nodetool cfstats to see the read latency for the cf. > > Take a look at vmstat to see if you are swapping, and look at iostats to see > if io is the problem > http://spyced.blogspot.co.nz/2010/01/linux-performance-basics.html > > Cheers > > ----------------- > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 8/06/2012, at 9:00 PM, Gurpreet Singh wrote: > > Thanks Ruslan. > I will try the mmap_index_only. > Is there any guideline as to when to leave it to auto and when to use > mmap_index_only? > > /G > > On Fri, Jun 8, 2012 at 1:21 AM, ruslan usifov <ruslan.usi...@gmail.com> > wrote: >> >> disk_access_mode: mmap?? >> >> set to disk_access_mode: mmap_index_only in cassandra yaml >> >> 2012/6/8 Gurpreet Singh <gurpreet.si...@gmail.com>: >> > Hi, >> > I am testing cassandra 1.1 on a 1 node cluster. >> > 8 core, 16 gb ram, 6 data disks raid0, no swap configured >> > >> > cassandra 1.1.1 >> > heap size: 8 gigs >> > key cache size in mb: 800 (used only 200mb till now) >> > memtable_total_space_in_mb : 2048 >> > >> > I am running a read workload.. about 30 reads/second. no writes at all. >> > The system runs fine for roughly 12 hours. >> > >> > jconsole shows that my heap size has hardly touched 4 gigs. >> > top shows - >> > SHR increasing slowly from 100 mb to 6.6 gigs in these 12 hrs >> > RES increases slowly from 6 gigs all the way to 15 gigs >> > buffers are at a healthy 25 mb at some point and that goes down to 2 >> > mb in >> > these 12 hrs >> > VIRT stays at 85 gigs >> > >> > I understand that SHR goes up because of mmap, RES goes up because it is >> > showing SHR value as well. >> > >> > After around 10-12 hrs, the cpu utilization of the system starts >> > increasing, >> > and i notice that kswapd0 process starts becoming more active. >> > Gradually, >> > the system cpu becomes high almost 70%, and the client starts getting >> > continuous timeouts. The fact that the buffers went down from 20 mb to 2 >> > mb >> > suggests that kswapd0 is probably swapping out the pagecache. >> > >> > Is there a way out of this to avoid the kswapd0 starting to do things >> > even >> > when there is no swap configured? >> > This is very easily reproducible for me, and would like a way out of >> > this >> > situation. Do i need to adjust vm memory management stuff like >> > pagecache, >> > vfs_cache_pressure.. things like that? >> > >> > just some extra information, jna is installed, mlockall is successful. >> > there >> > is no compaction running. >> > would appreciate any help on this. >> > Thanks >> > Gurpreet >> > >> > > > >
