Re: [ceph-users] OSD killed by OOM when many cache available

2017-11-17 Thread Eric Nelson
As I understand it the memory would show up in /proc/meminfo or vmstat as cached, however since it's being used for the page cache the kernel cannot allocate it at the time of oom-killer invocation since it's paged out (likely for cached reads/writes to your storage device). You could call # sync ;

[ceph-users] Use case for multiple "zonegroup"s in a realm

2017-11-17 Thread Richard Chan
What are good use cases for multiple zonegroups: is this mainly a namespacing thing? What do you use it for? Please help my understanding: Within a realm, all zonegroups share the same namespace(object IDs?) for tenants, users, and buckets(is this correct)? All these entities are created in the m

Re: [ceph-users] OSD killed by OOM when many cache available

2017-11-17 Thread Sam Huracan
@Eric: How can I check status of fscache? Why can it be root cause? Thanks 2017-11-18 7:30 GMT+07:00 Eric Nelson : > One thing that doesn't show up is fs cache, which is likely the cause > here. We went through this on our SSDs and had to add the following to stop > the crashes. I believe vm.vfs

Re: [ceph-users] OSD killed by OOM when many cache available

2017-11-17 Thread Eric Nelson
One thing that doesn't show up is fs cache, which is likely the cause here. We went through this on our SSDs and had to add the following to stop the crashes. I believe vm.vfs_cache_pressure and min_free_kbytes were the really helpful things in getting the crashes to stop. HTH! sysctl_param 'vm.vf

Re: [ceph-users] OSD killed by OOM when many cache available

2017-11-17 Thread Sam Huracan
I see some more logs about memory in syslog: Nov 17 10:47:17 ceph1 kernel: [2810698.553749] Node 0 DMA free:14828kB min:32kB low:40kB high:48kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15996kB managed:15896kB

[ceph-users] OSD killed by OOM when many cache available

2017-11-17 Thread Sam Huracan
Today, one of our Ceph OSDs was down, I've check syslog and see this OSD process was killed by OMM Nov 17 10:01:06 ceph1 kernel: [2807926.762304] Out of memory: Kill process 3330 (ceph-osd) score 7 or sacrifice child Nov 17 10:01:06 ceph1 kernel: [2807926.763745] Killed process 3330 (ceph-osd) to

[ceph-users] I/O stalls when doing fstrim on large RBD

2017-11-17 Thread Brendan Moloney
Hi, I guess this isn't strictly about Ceph, but I feel like other folks here must have run into the same issues. I am trying to keep my thinly provisioned RBD volumes thin. I use virtio-scsi to attach the RBD volumes to my VMs with the "discard=unmap" option. The RBD is formatted as XFS and s

Re: [ceph-users] Bluestore performance 50% of filestore

2017-11-17 Thread Milanov, Radoslav Nikiforov
Here's some more results, I'm reading 12.2.2 will have performance improvements for bluestore and should be released soon? Iodepth=not specified Filestore write: io=3511.9MB, bw=19978KB/s, iops=4994, runt=180001msec write: io=3525.6MB, bw=20057KB/s, iops=5014, runt=180001msec write: io=355

Re: [ceph-users] Moving bluestore WAL and DB after bluestore creation

2017-11-17 Thread Ronny Aasen
On 16.11.2017 09:45, Loris Cuoghi wrote: Le Wed, 15 Nov 2017 19:46:48 +, Shawn Edwards a écrit : On Wed, Nov 15, 2017, 11:07 David Turner wrote: I'm not going to lie. This makes me dislike Bluestore quite a bit. Using multiple OSDs to an SSD journal allowed for you to monitor the writ

Re: [ceph-users] Rename iscsi target_iqn

2017-11-17 Thread Jason Dillaman
On Fri, Nov 17, 2017 at 8:01 AM, Frank Brendel wrote: > Hi, > > how can I rename an iscsi target_iqn? That operation is not supported via gwcli. > And where is the configuration that I made with gwcli stored? It's stored in a JSON object within the 'rbd' pool named "gateway.conf". > > Thank yo

[ceph-users] Rename iscsi target_iqn

2017-11-17 Thread Frank Brendel
Hi, how can I rename an iscsi target_iqn? And where is the configuration that I made with gwcli stored? Thank you Frank ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Is the 12.2.1 really stable? Anybody have production cluster with Luminous Bluestore?

2017-11-17 Thread Cassiano Pilipavicius
I have a small cluster on 12.2.1, used only for storing VHDS on RBD and it is pretty stable so far. I've upgraded from jewel to luminous and the only thing that caused me instability right after the upgrade is that I was using JEMalloc for the OSDs and after converting the OSDs to bluestore the