Re: [ceph-users] can not find files .asok of osds in the folder /var/run/ceph

Hi John, It seems this is issue that i had. If i start osds with initctl (start/stop) then after i can't stop them with "service ceph". I stopped all osds with initctl and remove files upstart in the folder /var/lib/ceph/osd/ceph-/, then start ceph with "service ceph -a start", everything is ok, i

[ceph-users] Ceph Multi-site Implementation

Hi all, I have some questions about the Ceph multi-site implementation. I am thinking to have Ceph as the storage solution for across three internal site. I think, with a good internet connection, using the Multi-site object storage with RADOS (or RGW) might be a good use here. Thus, each site wi

[ceph-users] Multi-site Implementation

Hi all, I have some questions about the Ceph multi-site implementation. I am thinking to have Ceph as the storage solution for across three internal site. I think, with a good internet connection, using the Multi-site object storage with RADOS (or RGW) might be a good use here. Thus, each site

[ceph-users] Multi-site Implementation

Hi all, I have some questions about the Ceph multi-site implementation. I am thinking to have Ceph as the storage solution for across three internal site. I think, with a good internet connection, using the Multi-site object storage with RADOS (or RGW) might be a good use here. Thus, each site wi

[ceph-users] Could anyone tell me How to remove MDS in cluster? Thanks

ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s). If you are not an intended recipie

Re: [ceph-users] RBD does not load at boot

Use /etc/modules, not /etc/rc.modules, if you're using Ubuntu. You're using RedHat config files for a Debian system. On Tue, Apr 1, 2014 at 1:50 PM, Dan Koren wrote: > Hi Ivan, > I am using the repos. > I don't however see how this could have anything to do with the repos, > since rbd is install

Re: [ceph-users] RBD does not load at boot

Hi Ivan, I am using the repos. I don't however see how this could have anything to do with the repos, since rbd is installed and can be modrobed. It just doesn't load at boot even though it is included in /etc/rc.modules. The OS is Ubuntu 12.04. Thx, dk On Tue, Apr 1, 2014 at 12:55 AM, Igor Laskov

Re: [ceph-users] Largest Production Ceph Cluster

Our (DreamHost's) largest cluster is roughly the same size as yours, ~3PB on just shy of 1100 OSDs currently. The architecture's quite similar too, except we have "separate" 10G front-end and back-end networks with a partial spine-leaf architecture using 40G interconnects. I say "separate" becaus

Re: [ceph-users] can not find files .asok of osds in the folder /var/run/ceph

There was an issue[1] with service restarts killing .asok files that was fixed, I wonder if this is related. What distribution and version? John 1. http://tracker.ceph.com/issues/7188 On Tue, Apr 1, 2014 at 8:13 AM, Thanh Tran wrote: > Hi, > > When I upgrade ceph from version emperor (0.72.2

Re: [ceph-users] ceph 0.78 mon and mds crashing (bus error)

On Tue, Apr 1, 2014 at 7:12 AM, Yan, Zheng wrote: > On Tue, Apr 1, 2014 at 10:02 PM, Kenneth Waegeman > wrote: >> After some more searching, I've found that the source of the problem is with >> the mds and not the mon.. The mds crashes, generates a core dump that eats >> the local space, and in t

Re: [ceph-users] ceph 0.78 mon and mds crashing (bus error)

- Message from "Yan, Zheng" - Date: Tue, 1 Apr 2014 22:12:15 +0800 From: "Yan, Zheng" Subject: Re: [ceph-users] ceph 0.78 mon and mds crashing (bus error) To: Kenneth Waegeman Cc: Joao Eduardo Luis , ceph-users On Tue, Apr 1, 2014 at 10:02 PM, Kenneth Waegema

Re: [ceph-users] rbd map error - numerical result out of range

On Tue, Apr 1, 2014 at 6:55 PM, Tom wrote: > Thanks for the reply. > > Ceph is version 0.73-1precise, and the kernel release is > 3.11.9-031109-generic. > > also rbd showmapped shows 16 lines of output. Are there snapshots involved? Thanks, Ilya _

Re: [ceph-users] rbd map error - numerical result out of range

Thanks for the reply. Ceph is version 0.73-1precise, and the kernel release is 3.11.9-031109-generic. also rbd showmapped shows 16 lines of output. Thanks again. Tom. On 01/04/14 15:49, Ilya Dryomov wrote: On Tue, Apr 1, 2014 at 6:31 PM, Tom wrote: Hi, I'm seeing this error trying to m

Re: [ceph-users] rbd map error - numerical result out of range

On Tue, Apr 1, 2014 at 6:31 PM, Tom wrote: > Hi, > > I'm seeing this error trying to map an rbd. > > root@fh-ceph01-export01:~# rbd map > hosting_linux_sharedweb/linux_ded_backup_01 --id=ff-ceph01-export > rbd: add failed: (34) Numerical result out of range > > Is perhaps the string I'm writing to

[ceph-users] rbd map error - numerical result out of range

Hi, I'm seeing this error trying to map an rbd. root@fh-ceph01-export01:~# rbd map hosting_linux_sharedweb/linux_ded_backup_01 --id=ff-ceph01-export rbd: add failed: (34) Numerical result out of range Is perhaps the string I'm writing too large for /sys/bus/rbd/add? Output from strace: ope

Re: [ceph-users] Largest Production Ceph Cluster

Hi, On 1 Apr 2014 at 15:59:07, Andrey Korolyov (and...@xdel.ru) wrote: On 04/01/2014 03:51 PM, Robert Sander wrote: > On 01.04.2014 13:38, Karol Kozubal wrote: > >> I am curious to know what is the largest known ceph production deployment? > > I would assume it is the CERN

Re: [ceph-users] ceph 0.78 mon and mds crashing (bus error)

On Tue, Apr 1, 2014 at 10:02 PM, Kenneth Waegeman wrote: > After some more searching, I've found that the source of the problem is with > the mds and not the mon.. The mds crashes, generates a core dump that eats > the local space, and in turn the monitor (because of leveldb) crashes. > > The erro

Re: [ceph-users] radosgw multipart-uploaded downloads fail

Hi Yehuda, > 2014-04-01 15:49 GMT+02:00 Yehuda Sadeh : > It could be the gateway's fault, might be related to the new manifest > that went in before 0.78. I'll need more logs though, can you > reproduce with 'debug ms = 1', and 'debug rgw = 20', and provide a log > for all the upload and for the

Re: [ceph-users] ceph 0.78 mon and mds crashing (bus error)

After some more searching, I've found that the source of the problem is with the mds and not the mon.. The mds crashes, generates a core dump that eats the local space, and in turn the monitor (because of leveldb) crashes. The error in the mds log of one host: 2014-04-01 15:46:34.414615 7f

Re: [ceph-users] Largest Production Ceph Cluster

On 04/01/2014 03:51 PM, Robert Sander wrote: > On 01.04.2014 13:38, Karol Kozubal wrote: > >> I am curious to know what is the largest known ceph production deployment? > > I would assume it is the CERN installation. > > Have a look at the slides from Frankfurt Ceph Day: > > http://www.slidesha

Re: [ceph-users] radosgw multipart-uploaded downloads fail

On Tue, Apr 1, 2014 at 1:19 AM, Benedikt Fraunhofer wrote: > Hello everyone, > > I can't download anything that's been uploaded as a multipart-upload. > I'm on 0.78 (f6c746c314d7b87b8419b6e584c94bfe4511dbd4) > on a non-ec-pool. > > The upload is acknowledged as beeing ok > > 2014-03-31 14:56:56.7

Re: [ceph-users] RBD as backend for iSCSI SAN Targets

Hi everyone, I want to know the scheme if someone has a Ceph Cluster and running on it Cassandra databases, and if possible to send some performance numbers. Also if i wanna know if someone has a linux server (Ubuntu or CentOS) running on an HP Blade server and use a Ceph cluster as Back

Re: [ceph-users] ceph 0.78 mon and mds crashing (bus error)

- Message from Joao Eduardo Luis - Date: Tue, 01 Apr 2014 13:33:05 +0100 From: Joao Eduardo Luis Subject: Re: [ceph-users] ceph 0.78 mon and mds crashing (bus error) To: Kenneth Waegeman , ceph-users On 04/01/2014 01:17 PM, Kenneth Waegeman wrote: Hi all, We have

Re: [ceph-users] ceph 0.78 mon and mds crashing (bus error)

On 04/01/2014 01:17 PM, Kenneth Waegeman wrote: Hi all, We have installed Ceph 0.78 on 3 hosts running SL6, each serving OSD and MON daemons, 2 of them running MDS in active/backup. Since today the MON and MDS daemons are crashing every time (after a short while). Rebooting the nodes didn't help

[ceph-users] ceph 0.78 mon and mds crashing (bus error)

Hi all, We have installed Ceph 0.78 on 3 hosts running SL6, each serving OSD and MON daemons, 2 of them running MDS in active/backup. Since today the MON and MDS daemons are crashing every time (after a short while). Rebooting the nodes didn't help. The cluster was running for 1-2 weeks w

Re: [ceph-users] Security Hole?

0.72 is emperor. On Mar 31, 2014, at 11:00 AM, Dan Van Der Ster wrote: > Hi, > I can't reproduce that with a dumpling cluster: > > # cat ceph.client.dpm.keyring > [client.dpm] >key = xxx >caps mon = "allow r" >caps osd = "allow x, allow rwx pool=dpm" > > # ceph healt

Re: [ceph-users] Largest Production Ceph Cluster

On 01.04.2014 13:38, Karol Kozubal wrote: > I am curious to know what is the largest known ceph production deployment? I would assume it is the CERN installation. Have a look at the slides from Frankfurt Ceph Day: http://www.slideshare.net/Inktank_Ceph/scaling-ceph-at-cern Regards -- Robert S

[ceph-users] Largest Production Ceph Cluster

Hi All, I am curious to know what is the largest known ceph production deployment? I am looking for information in regards to: * number of nodes * number of OSDs * total capacity And if available details in regards to IOPS, types of disks, types of network interfaces, switches and

Re: [ceph-users] RBD as backend for iSCSI SAN Targets

On Fri 28 Mar 2014 08:55:30 AM CST, Karol Kozubal wrote: > Hi Jianing, > Sorry for the late reply, I missed your contribution to the thread. > Thank you for your response. I am still waiting for some of my hardware > and will begin testing the new setup with firefly once it is available as

Re: [ceph-users] OSD mystery

Could clock skew on mon.rts23 cause osd from not listing ? On 04/01/2014 02:28 AM, Dan Koren wrote: > clock skew detected on mon.rts23 signature.asc Description: OpenPGP digital signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://l

[ceph-users] radosgw multipart-uploaded downloads fail

Hello everyone, I can't download anything that's been uploaded as a multipart-upload. I'm on 0.78 (f6c746c314d7b87b8419b6e584c94bfe4511dbd4) on a non-ec-pool. The upload is acknowledged as beeing ok 2014-03-31 14:56:56.722727 7f4080ff9700 2 req 8:0.023285:s3:POST /file1:complete_multipart:http

Re: [ceph-users] RBD does not load at boot

Hi Dan, Have you tried use repos ? http://ceph.com/docs/master/install/get-packages/ On Tue, Apr 1, 2014 at 6:01 AM, Dan Koren wrote: > Even though it is included in /etc/rc.modules > and initramfs has been updated. > Suggestions much appreciated. > MTIA, > dk > > > ___

Re: [ceph-users] RDO - CEPH

Check if RDO is properly configured to use ceph , also look for MON and OSD logs at cep bend , you should get some messages from when qemu fails to crate volume Karan Singh Systems Specialist , Storage Platforms CSC - IT Center f

[ceph-users] can not find files .asok of osds in the folder /var/run/ceph

Hi, When I upgrade ceph from version emperor (0.72.2) to version firefly (0.78-367-gd9a2dea), i can't find the files .asok of osds on hosts in the folder /var/run/ceph as they ever exist as before. What's wrong? Best regards, Thanh Tran ___ ceph-users m