Re: [ceph-users] How to do quiesced rbd snapshot in libvirt?

Hello again! Unfortunately I have to raise the problem again. I have constantly hanging snapshots on several images. My Ceph version is now 0.94.5. RBD CLI always giving me this: root@slpeah001:[~]:# rbd snap create volumes/volume-26c89a0a-be4d-45d4-85a6-e0dc134941fd --snap test 2016-01-13 12:04:3

Re: [ceph-users] How to check the block device space usage

Thank Josh for sharing the idea. I have try the command for calculating the block size by ' sudo rbd diff PoolName/ImageName |awk '{ SUM += $2 } END { print SUM/1024/1024 " MB" }''. It seems take much time even to 1-2 minutes. I think it's not suitable for production environment. WD -Origin

Re: [ceph-users] Ceph cluster + Ceph client upgrade path for production environment

Hello Guys Need help with this , thanks - vickey - On Tue, Jan 12, 2016 at 12:10 PM, Vickey Singh wrote: > Hello Community , wishing you a great new year :) > > This is the recommended upgrade path > http://docs.ceph.com/docs/master/install/upgrading-ceph/ > > Ceph Deploy > Ceph Monitors > Cep

Re: [ceph-users] Ceph cache tier and rbd volumes/SSD primary, HDD replica crush rule!

What are the recommanded specs of a SSD for journaling. It's a little bit tricky now to move the journals for spinners on them, because i have data on them. I now have all HDD journals on separate SSD. The problem is when i first made the cluster i assigned one journal SSD to 8x4TB HDD. Now i see

Re: [ceph-users] pg is stuck stale (osd.21 still removed)

Hi ceph-users, any idea to fix my cluster? OSD.21 removed, but still some (staled) PG's pointing to OSD.21... I don't know how to proceed... Help is very welcome! Best regards Daniel > -Original Message- > From: Daniel Schwager > Sent: Friday, January 08, 2016 3:10 PM > To: 'ceph-us..

[ceph-users] lost OSD due to failing disk

Hi there, we recently had a problem with two OSDs failing because of I/O errors of the underlying disks. We run a small ceph cluster with 3 nodes and 18 OSDs in total. All 3 nodes are dell poweredge r515 servers with PERC H700 (MegaRAID SAS 2108) RAID controllers. All disks are configured as s

Re: [ceph-users] lost OSD due to failing disk

So let me get this straight! You have 3 hosts with 6 drives each in raid 0. So you have set 3 OSDs in crushmap, right? You said replication level is 2, so you have 2 copies of the original data! So the pool size is 3, right? You said 2 out of 3 OSD are down. So you are left with only one copy of th

Re: [ceph-users] lost OSD due to failing disk

On 13 January 2016 at 12:26, Magnus Hagdorn wrote: > Hi there, > we recently had a problem with two OSDs failing because of I/O errors of the > underlying disks. We run a small ceph cluster with 3 nodes and 18 OSDs in > total. All 3 nodes are dell poweredge r515 servers with PERC H700 (MegaRAID >

Re: [ceph-users] CentOS 7.2, Infernalis, preparing osd's and partprobe issues.

Hey All, Not trying to hijack this thread but since you are running Cent7.2 and Infernalis - If you get wrong results from "ceph osd tree" or download/decompiled crushmap could you please let me know ? I have asked in other threads for help on this but nothing back yet that was helpful. wade.hol.

Re: [ceph-users] Ceph cache tier and rbd volumes/SSD primary, HDD replica crush rule!

Check this blog post http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/ Intel wise, you really want dc3700’s or 3500’s if you won’t be write heavy. IO’s tend to be generally more important, but flat out write bandwidth can be impor

Re: [ceph-users] Ceph cluster + Ceph client upgrade path for production environment

Hi Vickey, under "Upgrade procedures", you will see that it is recommended to upgrade clients after having upgraded your cluster [1] [1] http://docs.ceph.com/docs/master/install/upgrading-ceph/#upgrading-a-client Regards On 13 January 2016 at 12:44, Vickey Singh wrote: > Hello Guys > > Need help

Re: [ceph-users] pg is stuck stale (osd.21 still removed) - SOLVED.

Well, ok - I found the solution: ceph health detail HEALTH_WARN 50 pgs stale; 50 pgs stuck stale pg 34.225 is stuck inactive since forever, current state creating, last acting [] pg 34.225 is stuck unclean since forever, current state creating,

[ceph-users] Ceph node stats back to calamari

I have calamari setup and running, but I'm only getting node stats from the node running calamari and ceph together (docker.test.com) The other nodes show the below error *ceph2.test.com :* *'ceph.get_heartbeats' is not available.* *ceph3.test.com

Re: [ceph-users] RBD export format for start and end snapshots

On Tue, Jan 12, 2016 at 12:09 PM, Josh Durgin wrote: > On 01/12/2016 06:10 AM, Alex Gorbachev wrote: > >> Good day! I am working on a robust backup script for RBD and ran into a >> need to reliably determine start and end snapshots for differential >> exports (done with rbd export-diff). >> >> I

[ceph-users] cephfs - inconsistent nfs and samba directory listings

Hello. Since we upgraded to Infernalis last, we have noticed a severe problem with cephfs when we have it shared over Samba and NFS Directory listings are showing an inconsistent view of the files: $ ls /lts-mon/BD/xmlExport/ | wc -l 100 $ sudo umount /lts-mon $ sudo mount /lts-mon $ ls /l

Re: [ceph-users] cephfs - inconsistent nfs and samba directory listings

On Wed, Jan 13, 2016 at 11:24 AM, Mike Carlson wrote: > Hello. > > Since we upgraded to Infernalis last, we have noticed a severe problem with > cephfs when we have it shared over Samba and NFS > > Directory listings are showing an inconsistent view of the files: > > > $ ls /lts-mon/BD/xmlExport/

Re: [ceph-users] cephfs - inconsistent nfs and samba directory listings

Hey Greg, The inconsistent view is only over nfs/smb on top of our /ceph mount. When I look directly on the /ceph mount (which is using the cephfs kernel module), everything looks fine It is possible that this issue just went unnoticed, and it only being a infernalis problem is just a red herrin

[ceph-users] Securing/Mitigating brute force attacks, Rados Gateway + Keystone

Hello Ceph Users, We've recently deployed a RGW service (0.94.3), We've also integrated this RGW instance to an external OpenStack Keystone identity service, RGW + Keystone integration/service are working well, On a high-level, our RGW service looks like:

Re: [ceph-users] How to do quiesced rbd snapshot in libvirt?

Definitely would like to see the "debug rbd = 20" logs from 192.168.254.17 when this occurs. If you are co-locating your OSDs, MONs, and qemu-kvm processes, make sure your ceph.conf has "log file = " defined in the [global] or [client] section. -- Jason Dillaman - Original Message ---

Re: [ceph-users] How to do quiesced rbd snapshot in libvirt?

Thanks, Jason, I forgot about this trick! These are the qemu rbd logs (last 200 lines). These lines are endlessly repeating when snapshot taking hangs: 2016-01-14 04:56:34.469568 7ff80e93e700 20 librbd::AioCompletion: AioCompletion::finalize() copied resulting 4096 bytes to 0x55bcc86c6000 2016-01-

Re: [ceph-users] How to do quiesced rbd snapshot in libvirt?

And here is my ceph.conf: [global] fsid = 78eef61a-3e9c-447c-a3ec-ce84c617d728 mon initial members = slpeah001,slpeah002,slpeah007 mon host = 192.168.254.11:6780,192.168.254.12:6780,192.168.254.17 public network = 192.168.254.0/23 cluster network = 192.168.252.0/23 auth cluster required = cephx au

Re: [ceph-users] cephfs - inconsistent nfs and samba directory listings

On Thu, Jan 14, 2016 at 3:37 AM, Mike Carlson wrote: > Hey Greg, > > The inconsistent view is only over nfs/smb on top of our /ceph mount. > > When I look directly on the /ceph mount (which is using the cephfs kernel > module), everything looks fine > > It is possible that this issue just went unn

Re: [ceph-users] pg is stuck stale (osd.21 still removed)

Hi Daniel, On Friday, January 8, 2016, Daniel Schwager wrote: > One more - I tried to recreate the pg but now this pg this "stuck > inactive": > > root@ceph-admin:~# ceph pg force_create_pg 34.225 > pg 34.225 now creating, ok > > root@ceph-admin:~# ceph health detail >