Re: [ceph-users] MySQL and ceph volumes

2017-03-08 Thread Wido den Hollander
> Op 8 maart 2017 om 0:35 schreef Matteo Dacrema : > > > Thank you Adrian! > > I’ve forgot this option and I can reproduce the problem. > > Now, what could be the problem on ceph side with O_DSYNC writes? > As mentioned nothing, but what you can do with MySQL is provide it multiple RBD disk

Re: [ceph-users] MySQL and ceph volumes

2017-03-08 Thread Matteo Dacrema
Ok, thank you guys. I changed the innodb flush method to O_DIRECT and seems to performs quite better. Regards Matteo This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom t

[ceph-users] broken links to ceph papers

2017-03-08 Thread Martin Bukatovic
Dear Ceph community, I noticed that many links on publications page[1] are broken, including link to weil-thesis.pdf Could you fix broken links so that the old links are working again? [1] http://ceph.com/publications/ -- Martin Bukatovic USM QE team ___

[ceph-users] re enable scrubbing

2017-03-08 Thread Laszlo Budai
Hello, is there any risk related to cluster overload when the scrub is re enabled after a certain amount of time being disabled? I am thinking of the following scenario: 1. scrub/deep scrub are disabled. 2. after a while (few days) we re enable them. How will the cluster perform? Will it run a

Re: [ceph-users] Much more dentries than inodes, is that normal?

2017-03-08 Thread John Spray
On Tue, Mar 7, 2017 at 3:05 PM, Xiaoxi Chen wrote: > Thanks John. > > Very likely, note that mds_mem::ino + mds_cache::strays_created ~= > mds::inodes, plus the MDS was the active-standby one, and become > active days ago due to failover. > > mds": { > "inodes": 1291393, > } > "mds_cache":

Re: [ceph-users] re enable scrubbing

2017-03-08 Thread Peter Maloney
It will stick to the config. If you limit the amount of work scrub does at a time, then you can let it do whatever it wants without issues (except 10.2.x which had a bug fixed in 10.2.4, but skip to 10.2.5 to fix a regression). For example: > # less scrub work at a time, with delay > osd scrub chu

[ceph-users] Jewel v10.2.6 released

2017-03-08 Thread Abhishek L
This point release fixes several important bugs in RBD mirroring, RGW multi-site, CephFS, and RADOS. We recommend that all v10.2.x users upgrade. For more detailed information, see the complete changelog[1] and the release notes[2] Notable Changes --- * build/ops: add hostname sani

Re: [ceph-users] Replication vs Erasure Coding with only 2 elementsinthe failure-domain.

2017-03-08 Thread Maxime Guyot
Hi, If using Erasure Coding, I think that should be using “choose indep” rather than “firstn” (according to http://lists.ceph.com/pipermail/ceph-users-ceph.com/2014-January/007306.html) “- min_size 4 - max_size 4 - step take - step chooseleaf firstn 2 type host - step emit - step take - step

Re: [ceph-users] re enable scrubbing

2017-03-08 Thread Laszlo Budai
In my case we have 72 OSDs. We are experiencing some performance issues. We believe that the reason is the scrubbing, so we want to turn scrubbing off for a few days. Given the default parameters of 1 day for scrub and 7 days for deep scrub. We turn off scrub for let's say 6 days, then when we

Re: [ceph-users] Strange read results using FIO inside RBD QEMU VM ...

2017-03-08 Thread Xavier Trilla
After some investigation, we realized it looks like the bottleneck is in the OSDs IOPS. The time it takes to resolve every IOP seems to be too high. We'll apply the following upgrades: * Ceph.conf modifications to allow better utilization of SSD Drives * Some extra sysctl modifications

[ceph-users] Shrinking lab cluster to free hardware for a new deployment

2017-03-08 Thread Kevin Olbrich
Hi! Currently I have a cluster with 6 OSDs (5 hosts, 7TB RAID6 each). We want to shut down the cluster but it holds some semi-productive VMs we might or might not need in the future. To keep them, we would like to shrink our cluster from 6 to 2 OSDs (we use size 2 and min_size 1). Should I set th

Re: [ceph-users] re enable scrubbing

2017-03-08 Thread Peter Maloney
On 03/08/17 13:50, Laszlo Budai wrote: > > In my case we have 72 OSDs. We are experiencing some performance > issues. We believe that the reason is the scrubbing, so we want to > turn scrubbing off for a few days. > Given the default parameters of 1 day for scrub and 7 days for deep > scrub. We tur

Re: [ceph-users] Shrinking lab cluster to free hardware for a new deployment

2017-03-08 Thread Maxime Guyot
Hi Kevin, I don’t know about those flags, but if you want to shrink your cluster you can simply set the weight of the OSDs to be removed to 0 like so: “ceph osd reweight osd.X 0” You can either do it gradually if your are concerned about client I/O (probably not since you speak of a test / semi

Re: [ceph-users] Shrinking lab cluster to free hardware for a new deployment

2017-03-08 Thread Henrik Korkuc
On 17-03-08 15:39, Kevin Olbrich wrote: Hi! Currently I have a cluster with 6 OSDs (5 hosts, 7TB RAID6 each). We want to shut down the cluster but it holds some semi-productive VMs we might or might not need in the future. To keep them, we would like to shrink our cluster from 6 to 2 OSDs (we

Re: [ceph-users] [Jewel] upgrade 10.2.3 => 10.2.5 KO : first OSD server freeze every two days :)

2017-03-08 Thread pascal.pu...@pci-conseil.net
Hello, No new information. Every two night server OSD 1 freeze with a load > 500. It's every 2 days. Sometime during scrub, sometime during fstrim, sometime during nothing... But this night, this OSD server came not a life after some minutes as before... 8 hours without this server and all i

[ceph-users] clarification for rgw installation and conflagration ( jwel )

2017-03-08 Thread Yair Magnezi
Hello Guys . I'm new to RGW and need some clarification ( i'm running 10.2.5 ) As much as i understand 'jewl' uses Civetweb instead of Apache and FastCGI but in the configuration guide ( just the next step in the the install guide ) it says "Configuring a Ceph Object Gateway requires a running

Re: [ceph-users] Ceph PG repair

2017-03-08 Thread Reed Dier
This PG/object is still doing something rather odd. Attempted to repair the object, which it supposedly attempted, but now I appear to have less visibility. > $ ceph health detail > HEALTH_ERR 3 pgs inconsistent; 4 scrub errors; mds0: Many clients (20) > failing to respond to cache pressure; no

Re: [ceph-users] clarification for rgw installation and conflagration ( jwel )

2017-03-08 Thread Abhishek Lekshmanan
On 03/08/2017 04:55 PM, Yair Magnezi wrote: > Hello Guys . > > I'm new to RGW and need some clarification ( i'm running 10.2.5 ) > As much as i understand 'jewl' uses Civetweb instead of Apache and > FastCGI but in the configuration guide ( just the next step in the the > install guide ) it

Re: [ceph-users] broken links to ceph papers

2017-03-08 Thread Patrick McGarry
Hey Martin, All of the links should be updated with the exception of the SK Telecom paper that was linked to IEEE. I'm working on getting a hard copy of that paper to host on ceph.com. Thanks for letting us know. On Wed, Mar 8, 2017 at 4:22 AM, Martin Bukatovic wrote: > Dear Ceph community, > >

Re: [ceph-users] broken links to ceph papers

2017-03-08 Thread Daniel W Corley
On this subject, I have noticed there are no downloads available for the documentation at http://docs.ceph.com/docs/master/. Would there be any concern if this were pulled via wget scripts and made into a PDF for offline reading or printing ? Possibly even being made available to share. --- -

Re: [ceph-users] broken links to ceph papers

2017-03-08 Thread Gregory Farnum
You'd might have an easier time grabbing the source out of ceph.git/doc and converting the raw rst files to whatever you want. :) On Wed, Mar 8, 2017 at 10:33 AM Daniel W Corley wrote: > On this subject, I have noticed there are no downloads available for the > documentation at http://docs.ceph

[ceph-users] cephfs and erasure coding

2017-03-08 Thread Rhian Resnick
Two questions on Cephfs and erasure coding that Google couldn't answer. 1) How well does cephfs work with erasure coding? 2) How would you move an existing cephfs pool that uses replication to erasure coding? Rhian Resnick Assistant Director Middleware and HPC Office of Information Techno

Re: [ceph-users] cephfs and erasure coding

2017-03-08 Thread John Spray
On Wed, Mar 8, 2017 at 7:54 PM, Rhian Resnick wrote: > Two questions on Cephfs and erasure coding that Google couldn't answer. > > > > 1) How well does cephfs work with erasure coding? > In the current released versions, you cannot use erasure coded pools with CephFS, unless there is a replicate

[ceph-users] [DR] master is on a different period

2017-03-08 Thread Daniel Picolli Biazus
Hi Guys, After a disaster recovery process, making a Secondary zone the Master, and the old Master as a Secondary zone. We could see the metadata stop syncing between the clusters, and any new bucket or users is replicated to Secondary Zone. *Version Running: 10.2.6* Running "radosgw-admin s

Re: [ceph-users] cephfs and erasure coding

2017-03-08 Thread David Turner
I use CephFS on erasure coding at home using a cache tier. It works fine for my use case, but we know nothing about your use case to know if it will work well for you. The answer as to how to move an existing cephfs pool from replication to erasure coding (and vice versa) is to create the new

[ceph-users] Why is librados for Python so Neglected?

2017-03-08 Thread Kent Borg
Python is such a great way to learn things. Such a shame the librados Python library is missing so much. It makes RADOS look so much more limited than it is. -kb ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/

[ceph-users] Object Map Costs (Was: Snapshot Costs (Was: Re: Pool Sizes))

2017-03-08 Thread Kent Borg
I'm slowly working my way through Ceph's features... I recently happened upon object maps. (I had heard of LevelDB being in there but never saw how to use it: That's because I have been using Python! And the Python library is missing lots of features! Grrr.) How fast are those omap calls? Wh

Re: [ceph-users] Object Map Costs (Was: Snapshot Costs (Was: Re: Pool Sizes))

2017-03-08 Thread Gregory Farnum
On Wed, Mar 8, 2017 at 2:01 PM, Kent Borg wrote: > I'm slowly working my way through Ceph's features... > > I recently happened upon object maps. (I had heard of LevelDB being in there > but never saw how to use it: That's because I have been using Python! And > the Python library is missing lots

Re: [ceph-users] Why is librados for Python so Neglected?

2017-03-08 Thread John Spray
On Wed, Mar 8, 2017 at 9:28 PM, Kent Borg wrote: > Python is such a great way to learn things. Such a shame the librados Python > library is missing so much. It makes RADOS look so much more limited than it > is. Specifically? I'm not saying you're wrong, but I am curious which bits in particula

Re: [ceph-users] Why is librados for Python so Neglected?

2017-03-08 Thread Kent Borg
On 03/08/2017 05:08 PM, John Spray wrote: Specifically? I'm not saying you're wrong, but I am curious which bits in particular you missed. Object maps. Those transaction-y things. Object classes. Maybe more I don't know about because I have been learning via Python. -kb, the Kent who has be

Re: [ceph-users] Why is librados for Python so Neglected?

2017-03-08 Thread Josh Durgin
On 03/08/2017 02:15 PM, Kent Borg wrote: On 03/08/2017 05:08 PM, John Spray wrote: Specifically? I'm not saying you're wrong, but I am curious which bits in particular you missed. Object maps. Those transaction-y things. Object classes. Maybe more I don't know about because I have been learni

[ceph-users] Bogus "inactive" errors during OSD restarts with Jewel

2017-03-08 Thread Christian Balzer
Hello, during OSD restarts with Jewel (10.2.5 and .6 at least) I've seen "stuck inactive for more than 300 seconds" errors like this when observing things with "watch ceph -s" : --- health HEALTH_ERR 59 pgs are stuck inactive for more than 300 seconds 223 pgs degrad

Re: [ceph-users] Error with ceph to cloudstack integration.

2017-03-08 Thread frank
Hi, We have made sure that the key,ceph user ,ceph admin keys are correct. could you let us know if there is any other possibility that would mess up the integration. Regards, Frank On 03/06/2017 01:22 PM, Wido den Hollander wrote: Op 6 maart 2017 om 6:26 schreef frank : Hi, We have se

[ceph-users] Jewel problems with sysv-init and non ceph-deploy (udev trickery) OSDs

2017-03-08 Thread Christian Balzer
Hello, Yes, this is Debian Jessie with sysv-init, not systemd. I prefer my servers to be deterministic. Firstly an issue with /var/run/ceph. The init.d/ceph script has these lines: --- if [ ! -d $run_dir ]; then # assume /var/run exists install -d -m0

[ceph-users] How does ceph preserve read/write consistency?

2017-03-08 Thread 许雪寒
Hi, everyone. Recently, in our test, we found a strange phenomenon: a READ req from client A that arrived later than a WRITE req from client B is finished ealier than that WRITE req. The logs are as follows(we did a little modification to the level of some logs to 1 in order to get some insigh

[ceph-users] Posix AIO vs libaio read performance

2017-03-08 Thread Xavier Trilla
Hi, I'm trying to debut why there is a big difference using POSIX AIO and libaio when performing read tests from inside a VM using librbd. The results I'm getting using FIO are: POSIX AIO Read: Type: Random Read - IO Engine: POSIX AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Targ

Re: [ceph-users] cephfs and erasure coding

2017-03-08 Thread Maxime Guyot
Hi, >“The answer as to how to move an existing cephfs pool from replication to >erasure coding (and vice versa) is to create the new pool and rsync your data >between them.” Shouldn’t it be possible to just do the “ceph osd tier add ecpool cachepool && ceph osd tier cache-mode cachepool writeb