Re: [ceph-users] Advantages of using Ceph with LXC

Hi! > What are a few advantages of using Ceph with LXC ? I'm using ceph daemons, packed in docker containers (http://docker.io). The main advantages is security and reliability, the software don't interact between each other, all daemons has different IP addresses, different filesystems, etc. A

Re: [ceph-users] Multiple MDS servers...

On Sun, Nov 23, 2014 at 10:36 PM, JIten Shah wrote: > Hi Greg, > > I haven’t setup anything in ceph.conf as mds.cephmon002 nor in any ceph > folders. I have always tried to set it up as mds.lab-cephmon002, so I am > wondering where is it getting that value from? No idea, sorry. Probably some odd

[ceph-users] can I use librgw APIS ?

can I use librgw APIS like librados? if I can, how to do it? baijia...@126.com___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ceph inconsistency after deep-scrub

On 11/21/2014 10:46 PM, Paweł Sadowski wrote: > W dniu 21.11.2014 o 20:12, Gregory Farnum pisze: >> On Fri, Nov 21, 2014 at 2:35 AM, Paweł Sadowski wrote: >>> Hi, >>> >>> During deep-scrub Ceph discovered some inconsistency between OSDs on my >>> cluster (size 3, min size 2). I have fund broken ob

[ceph-users] Virtual machines using RBD remount read-only on OSD slow requests

Hi, I have a Ceph cluster with 4 disk servers, 14 OSDs and replica size of 3. A number of KVM virtual machines are using RBD as their only storage device. Whenever some OSDs (always on a single server) have slow requests, caused, I believe, by flaky hardware or, in one occasion, by a S.M.A.R.T com

[ceph-users] [rgw] chunk size

Hi all Context : firefly 0.80.7 OS : ubuntu 14.04.1 LTS I'd like to change the chunk size for object stored with rgw to 1MB (4MB is the default) I changed ceph.conf setting rgw object stripe size = 1048576 and restarted the rgw. The chunk size remains to 4MB. I saw in different exchanges tha

[ceph-users] How to mount cephfs from fstab

Hi, I would like to mount a cephfs share from fstab, but it doesn't completely work. First of all, I followed the documentation [1], which resulted in the following line in fstab: ceph-01:6789:/ /mnt/cephfs/ ceph name=testhost,secretfile=/root/testhost.key,noacl 0 2 Yes, this works when I manua

Re: [ceph-users] How to mount cephfs from fstab

I mount mine with an init-script. Michael Kuriger mk7...@yp.com 818-649-7235 MikeKuriger (IM) On 11/24/14, 9:08 AM, "Erik Logtenberg" wrote: >Hi, > >I would like to mount a cephfs share from fstab, but it doesn't >completely work. > >First of all, I followed the documentation [1], which re

Re: [ceph-users] How to mount cephfs from fstab

Hi there, On Mon, 24 Nov 2014, 18:08:50 +0100, Erik Logtenberg wrote: > Hi, > > I would like to mount a cephfs share from fstab, but it doesn't > completely work. > > First of all, I followed the documentation [1], which resulted in the > following line in fstab: > > ceph-01:6789:/ /mnt/cephfs/

[ceph-users] ceph-deploy osd activate Hang - (doc followed step by step)

Everytime i try to create a second OSD i get this hang: $ ceph-deploy osd activate ceph-node2:/var/local/osd1 [cut ...] [ceph_deploy.cli][INFO ] Invoked (1.5.20): /usr/bin/ceph-deploy osd activate ceph-node2:/var/local/osd1 [ceph_deploy.osd][DEBUG ] Activating cluster ceph disks

[ceph-users] [ann] fio plugin for libcephfs

I've posted a preliminary patch set to support a libcephfs io engine in fio: http://github.com/noahdesu/fio cephfs You can use this right now to generate load through libcephfs, but the plugin needs a bit more work before it goes upstream (patches welcome), but feel free to play around with it

[ceph-users] Questions about deploying multiple cluster on same servers

Dear Ceph Team, Our company was trying to using ceph as our backend support. And I’ve encountered an error while deploying multiple clusters on the same servers. I wonder whether this is a bug or I did somthing wrong. According to the documents on ceph.com , I successfully

Re: [ceph-users] isolate_freepages_block and excessive CPU usage by OSD process

On 11/15/2014 12:48 PM, Andrey Korolyov wrote: > Hello, > > I had found recently that the OSD daemons under certain conditions > (moderate vm pressure, moderate I/O, slightly altered vm settings) can > go into loop involving isolate_freepages and effectively hit Ceph > cluster performance. I found

Re: [ceph-users] isolate_freepages_block and excessive CPU usage by OSD process

On 11/15/2014 06:10 PM, Andrey Korolyov wrote: > On Sat, Nov 15, 2014 at 7:32 PM, Vlastimil Babka wrote: >> On 11/15/2014 12:48 PM, Andrey Korolyov wrote: >>> Hello, >>> >>> I had found recently that the OSD daemons under certain conditions >>> (moderate vm pressure, moderate I/O, slightly altered

[ceph-users] Way to improve "rados get" bandwidth?

Hi, I did some bandwidth tests with "rados put/get" (1), (2). I'm wondering, why "rados get" (read) bandwidth-results are not that good. If I run - 1 instance of rados get (2), the bandwidth is about 170MB/sec, with - 4 instances (2) in parallel, I get 4 times ~170MB/sec == ~ 680MB/sec Is the

[ceph-users] Ceph fs has error: no valid command found; 10 closest matches: fsid

Hi All, When i create a new cephfs, this error shows Is it a bug? [root@ho-srv-ceph-02 ceph]# ceph fs new cephfs cephfs_metadata cephfs_data no valid command found; 10 closest matches: fsid Error EINVAL: invalid command Regards, Ndhuynh This e-mail message including any attachment

Re: [ceph-users] Stanza to add cgroup support to ceph upstart jobs

Is anyone else doing this? I think it would be a very good use case when we have systems with 40+ cores to separate the stacks. cory ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] how to run rados common by non-root user in ceph node

Hi All, After setting up the ceph, i can run ceph, rados command by root user, how can i run them by non-root account? i created a key already: [root@ho-srv-ceph-02 ~]# ceph auth list client.oneadmin key: AQBQY21UaLLhCBAAKjsM8qRxFpJA4ppbA7Rn9A== caps: [mon] allow r caps: [osd] allow

Re: [ceph-users] how to run rados common by non-root user in ceph node

The non root user needs to be able to read the key file. Michael Kuriger mk7...@yp.com 818-649-7235 MikeKuriger (IM) From: Huynh Dac Nguyen mailto:ndhu...@spsvietnam.vn>> Date: Wednesday, November 19, 2014 at 8:44 PM To: "ceph-users@lists.ceph.com" mailto:ceph-us

Re: [ceph-users] Regarding Federated Gateways - Zone Sync Issues

I'm really not sure. I'm using the S3 interface rather than the Swift interface. Once my non-systm user replicated, I was able to access everything in the secondary cluster just fine. Hopefully somebody else with Swift experience will chime in. On Sat, Nov 22, 2014 at 12:47 AM, Vinod H I wro

Re: [ceph-users] Ceph fs has error: no valid command found; 10 closest matches: fsid

Hi, Use ceph mds newfs {metaid} {dataid} instead JC > On Nov 20, 2014, at 20:43, Huynh Dac Nguyen wrote: > > Hi All, > > When i create a new cephfs, this error shows > > Is it a bug? > > [root@ho-srv-ceph-02 ceph]# ceph fs new cephfs cephfs_metadata cephfs_data > no valid command found; 1

Re: [ceph-users] Virtual machines using RBD remount read-only on OSD slow requests

Hi, try to mount your filesystems with errors=continue option From the mount (8) man page errors={continue|remount-ro|panic} Define the behaviour when an error is encountered. (Either ignore errors and just mark the filesystem erroneous and continue, or remount the filesystem read-only, or

Re: [ceph-users] Ceph fs has error: no valid command found; 10 closest matches: fsid

You're likely not running a release which includes this command - it was only introduced in v0.84. If you're running giant, it should work. If not, you won't have it (and won't need to create a filesystem - it will exist by default). *Chris Armstrong*Head of Services & Community OpDemand / Deis.i

Re: [ceph-users] How to mount cephfs from fstab

On 24.11.2014 19:08, Erik Logtenberg wrote: ... So, how do my fellow cephfs-users do this? I do not use cephfs yet, but there seems a measure for your problem for a systemd based OS: http://www.cepheid.org/~jeff/?p=69 Kind Regards, Alek __

Re: [ceph-users] ceph-announce list

On Fri, Nov 21, 2014 at 12:34 AM, JuanFra Rodriguez Cardoso wrote: > Hi all: > > As it was asked weeks ago.. what is the way the ceph community uses to > stay tuned on new features and bug fixes? I asked Sage about this today and he said he'd set one up. Seems like a good idea; just not something

Re: [ceph-users] Client forward compatibility

On Thu, Nov 20, 2014 at 9:08 AM, Dan van der Ster wrote: > Hi all, > What is compatibility/incompatibility of dumpling clients to talk to firefly > and giant clusters? We sadly don't have a good matrix about this yet, but in general you should assume that anything which changed the way the data i

[ceph-users] fiemap bug on giant

Bug #10166 (http://tracker.ceph.com/issues/10166) can cause recovery to result in incorrect object sizes on giant if the setting 'filestore fiemap' is set to true. This setting is disabled by default. This should be fixed in a future point release, though filestore fiemap will probably continue t

[ceph-users] announcing ceph-announce

This list will get release announcements and other general news items. It will be moderated. It's intended for those that aren't interested in the volume of messages on ceph-users but would like to hear general announcements about the project. Sign up here: http://lists.ceph.com/listi

Re: [ceph-users] Radosgw agent only syncing metadata

On 22/11/14 10:54, Yehuda Sadeh wrote: On Thu, Nov 20, 2014 at 6:52 PM, Mark Kirkwood wrote: Fri Nov 21 02:13:31 2014 x-amz-copy-source:bucketbig/_multipart_big.dat.2/fjid6CneDQYKisHf0pRFOT5cEWF_EQr.meta /bucketbig/__multipart_big.dat.2%2Ffjid6CneDQYKisHf0pRFOT5cEWF_EQr.meta 2014-11-21 15:13:

Re: [ceph-users] Radosgw agent only syncing metadata

On Mon, Nov 24, 2014 at 2:43 PM, Mark Kirkwood wrote: > On 22/11/14 10:54, Yehuda Sadeh wrote: >> >> On Thu, Nov 20, 2014 at 6:52 PM, Mark Kirkwood >> wrote: > > >>> Fri Nov 21 02:13:31 2014 >>> >>> x-amz-copy-source:bucketbig/_multipart_big.dat.2/fjid6CneDQYKisHf0pRFOT5cEWF_EQr.meta >>> /bucketb

Re: [ceph-users] announcing ceph-announce

Is it safe to say that if we're subbed to ceph-users, we don't need to sub separately to ceph-announce? (i.e. we'll already be aware of updates here?) *Chris Armstrong*Head of Services & Community OpDemand / Deis.io GitHub: https://github.com/deis/deis -- Docs: http://docs.deis.io/ On Mon, Nov

Re: [ceph-users] announcing ceph-announce

On Mon, 24 Nov 2014, Christopher Armstrong wrote: > Is it safe to say that if we're subbed to ceph-users, we don't need to sub > separately to ceph-announce? (i.e. we'll already be aware of updates here?) Yes. Everything sent to -announce list will also copy -users and -devel at a minimum. sage

Re: [ceph-users] Radosgw agent only syncing metadata

On 25/11/14 11:58, Yehuda Sadeh wrote: On Mon, Nov 24, 2014 at 2:43 PM, Mark Kirkwood wrote: On 22/11/14 10:54, Yehuda Sadeh wrote: On Thu, Nov 20, 2014 at 6:52 PM, Mark Kirkwood wrote: Fri Nov 21 02:13:31 2014 x-amz-copy-source:bucketbig/_multipart_big.dat.2/fjid6CneDQYKisHf0pRFOT5cEWF

Re: [ceph-users] Optimal or recommended threads values

Tuning these values depends on a lot more than just the SSDs and HDDs. Which kernel and IO scheduler are you using? Does your HBA do write caching? It also depends on what your goals are. Tuning for a RadosGW cluster is different that for a RDB cluster. The short answer is that you are the only

Re: [ceph-users] Optimal or recommended threads values

Don't forget number of cores in the node. Basically you want enough threads to keep all of the cores busy while not having so many that you end up with a ton of context switching overhead. Also as you said there's a lot of other factors that may have an affect, like the number of AGs (assumin

Re: [ceph-users] Negative number of objects degraded for extended period of time

To disable RadosGW GC, you could bump rgw_gc_obj_min_wait to something really big. If you set it to a week, you should have a week with no GC. When you return it to normal, it should just need a couple passes, depending on how much stuff you delete while GC is stalled. injectargs doesn't appear

Re: [ceph-users] fiemap bug on giant

It's surprised that the test machine run on a very new kernel but occur this problem: plana 47 is 12.04.5 with kernel 3.18.0-rc6-ceph-00024-geb0e5fd plana 50 is 12.04.4 with kernel 3.17.0-rc6-ceph-2-ge8acad6 Which local filesystem is ran on? On Tue, Nov 25, 2014 at 5:03 AM, Samuel Just wro

Re: [ceph-users] Ceph fs has error: no valid command found; 10 closest matches: fsid

Hi JC, Thank you, JC. # ceph osd lspools 0 data,1 metadata,2 rbd,3 one,4 cephfs_data,5 cephfs_metadata, # ceph mds newfs 4 5 --yes-i-really-mean-it new fs with metadata pool 4 and data pool 5 - So is the document wrong now? - When "ceph mds newfs 4 5" runs without "--yes-i-really-mea

Re: [ceph-users] Ceph fs has error: no valid command found; 10 closest matches: fsid

Hi Chris, I see. I'm runing on version 0.80.7. How do we know which part of document for our version? As you see, we have only one ceph document here, It make us confused. Could you show me the document for ceph version 0.80.7? Regards, Ndhuynh >>> Christopher Armstrong 11/25/20

Re: [ceph-users] how to run rados common by non-root user in ceph node

Hi Michael, It works now, after "chmod +r /etc/ceph/*" It's a stupid mistake, I remember that i have checked it before. Thank you Michael. Regards, Ndhuynh >>> Michael Kuriger 11/25/2014 1:21 AM >>> The non root user needs to be able to read the key file. Michael Kuriger mk7...