Re: [ceph-users] Flapping OSDs, Large meta directories in OSDs

On Tue, Dec 1, 2015 at 12:20 AM, Tom Christensen wrote: > What counts as ancient? Concurrent to our hammer upgrade we went from > 3.16->3.19 on ubuntu 14.04. We are looking to revert to the 3.16 kernel > we'd been running because we're also seeing an intermittent (its happened > twice in 2 weeks

Re: [ceph-users] High 0.94.5 OSD memory use at 8GB RAM/TB raw disk during recovery

On Mon, 2015-11-30 at 18:58 -0600, Mark Nelson wrote: > Hi Laurent, > > Wow, that's excessive! I'd see if anyone else has any tricks first, but > if nothing else helps, running an OSD through valgrind with massif will > probably help pinpoint what's going on. Have you tweaked the recovery > t

Re: [ceph-users] rbd_inst.create

Ok thank you Jason. [@@ THALES GROUP INTERNAL @@] -Message d'origine- De : Jason Dillaman [mailto:dilla...@redhat.com] Envoyé : lundi 30 novembre 2015 15:38 À : NEVEU Stephane Cc : Ceph Users; Gregory Farnum Objet : Re: [ceph-users] rbd_inst.create ... and once you create a pool-level s

Re: [ceph-users] State of nfs-ganesha CEPH fsal

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I've been testing below options out, but I still have the same problem that files are not visible on different clients. After a "touch" of a new file (or directory) all files are visible again. It definitly looks like a directory cache problem. C

Re: [ceph-users] Flapping OSDs, Large meta directories in OSDs

I believe that ‘filestore xattr use omap’ is no longer used in Ceph – can anybody confirm this? I could not find any usage in the Ceph source code except that the value is set in some of the test software… Paul From: ceph-users mailto:ceph-users-boun...@lists.ceph.com>> on behalf of Tom Chri

[ceph-users] F21 pkgs for Ceph Hammer release ?

Hi, Does anybody how/where I can get the F21 repo for ceph hammer release ? In download.ceph.com/rpm-hammer/ I only see F20 dir, not F21 F21 distro repo only carries firefly release, but I want to install Ceph Hammer, hence the Q thanx, deepak ___ cep

Re: [ceph-users] Would HEALTH_DISASTER be a good addition?

On 26-11-15 07:58, Wido den Hollander wrote: > On 11/25/2015 10:46 PM, Gregory Farnum wrote: >> On Wed, Nov 25, 2015 at 11:09 AM, Wido den Hollander wrote: >>> Hi, >>> >>> Currently we have OK, WARN and ERR as states for a Ceph cluster. >>> >>> Now, it could happen that while a Ceph cluster is i

Re: [ceph-users] does anyone know what xfsaild and kworker are?they make osd disk busy. produce 100-200iops per osd disk?

On 2014年11月11日 12:23, duan.xuf...@zte.com.cn wrote: ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of t

Re: [ceph-users] Cache Tiering Investigation and Potential Patch

Hi Sage/Mark, I have completed some initial testing of the tiering fix PR you submitted compared to my method I demonstrated at the perf meeting last week. >From a high level both have very similar performance when compared to the >current broken behaviour. So I think until Jewel, either way wo

Re: [ceph-users] Cache Tiering Investigation and Potential Patch

On 12/01/2015 10:30 AM, Nick Fisk wrote: Hi Sage/Mark, I have completed some initial testing of the tiering fix PR you submitted compared to my method I demonstrated at the perf meeting last week. From a high level both have very similar performance when compared to the current broken beha

Re: [ceph-users] Cache Tiering Investigation and Potential Patch

> -Original Message- > From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel- > ow...@vger.kernel.org] On Behalf Of Mark Nelson > Sent: 01 December 2015 16:58 > To: Nick Fisk ; 'Sage Weil' > Cc: 'ceph-users' ; ceph-de...@vger.kernel.org > Subject: Re: Cache Tiering Investigation an

Re: [ceph-users] does anyone know what xfsaild and kworker are?they make osd disk busy. produce 100-200iops per osd disk?

This is xfs metadata sync process...when it is waking up and there are lot of data to sync it will throttle all the process accessing the drive...There are some xfs settings to control the behavior, but you can't stop that Sent from my iPhone >> On Dec 1, 2015, at 8:26 AM, flisky wrote: >> >>

Re: [ceph-users] Flapping OSDs, Large meta directories in OSDs

Another thing that we don't quite grasp is that when we see slow requests now they almost always, probably 95% have the "known_if_redirected" state set. What does this state mean? Does it indicate we have OSD maps that are lagging and the cluster isn't really in sync? Could this be the cause of

[ceph-users] OSD on a partition

I am looking through google, and I am not seeing a good guide as to how to put an OSD on a partition (GPT) of a disk. I see lots of options for file system, or single physical drive but not partition. http://dachary.org/?p=2548 This is only thing I found but that

[ceph-users] Ceph job posting

Just dropping a note to say that Teradata (I work there!) is hiring to build out a small-at-first Ceph team in our Rancho Bernardo office (near San Diego, CA). We're looking for engineers interested in getting Ceph to spin like a top for our data warehouse applications. You should know C/C++, vir

Re: [ceph-users] Flapping OSDs, Large meta directories in OSDs

Another "new" thing we see with hammer is constant: mon.0 [INF] from='client.52217412 :/0' entity='client.admin' cmd='[{"prefix": "osd blacklist", "blacklistop": "add", "addr": ":0/3562049007"}]': finished entries in the log and while watching ceph -w The cluster appears to generate a new osdmap

Re: [ceph-users] does anyone know what xfsaild and kworker are?they make osd disk busy. produce 100-200iops per osd disk?

On 2015年12月02日 01:31, Somnath Roy wrote: > This is xfs metadata sync process...when it is waking up and there are lot of > data to sync it will throttle all the process accessing the drive...There are > some xfs settings to control the behavior, but you can't stop that May I ask how to tune the x

Re: [ceph-users] OSD on a partition

On 12/01/2015 07:29 PM, Marek Dohojda wrote: > I am looking through google, and I am not seeing a good guide as to how > to put an OSD on a partition (GPT) of a disk. I see lots of options for > file system, or single physical drive but not partition. > > http://dachary.org/?p=2548 > > This is

Re: [ceph-users] OSD on a partition

Well so here is my problem. I want to move journal to SSD, but I have no more Disk slots available, and the SSD I do have is used for my higher IO OSDs. Hence I don’t want to lose my OSD. So my thought was to partition the SSD into 10G and the rest with the “rest” being used for OSD, while

Re: [ceph-users] OSD on a partition

> -Original Message- > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of > Marek Dohojda > Sent: 01 December 2015 19:34 > To: Wido den Hollander > Cc: ceph-users@lists.ceph.com > Subject: Re: [ceph-users] OSD on a partition > > Well so here is my problem. I wan

Re: [ceph-users] High 0.94.5 OSD memory use at 8GB RAM/TB raw disk during recovery

> On Nov 30, 2015, at 6:52 PM, Laurent GUERBY wrote: > > Hi, > > We lost a disk today in our ceph cluster so we added a new machine with > 4 disks to replace the capacity and we activated straw1 tunable too > (we also tried straw2 but we quickly backed up this change). > > During recovery OSD

[ceph-users] Cinder-CEPH Job Openings with @WalmartLabs [Location: India, Bangalore]

Hello, I am looking for hiring best CEPH minds to work on building scalable Block and Object storage solutions (In progress), If interested do drop me an email/CV. Note: Job location is India, Bangalore. Thanks, Janardhan ___ ceph-users mailing list c

Re: [ceph-users] does anyone know what xfsaild and kworker are?they make osd disk busy. produce 100-200iops per osd disk?

Sure..The following settings helped me minimizing the effect a bit for the PR https://github.com/ceph/ceph/pull/6670 sysctl -w fs.xfs.xfssyncd_centisecs=72 sysctl -w fs.xfs.xfsbufd_centisecs=3000 sysctl -w fs.xfs.age_buffer_centisecs=72 But, for existing Ceph write path you may ne

Re: [ceph-users] OSD on a partition

Thus far parted isn’t very nice with ext4, I suppose I could try to do so and the worst that would happen is me loosing 1 OSD, however in my tests this wasn’t very reliable. Non GPT partition, utilizing fdisk I can do this without a problem, but OSD requires GPT (to the best of my knowledge a

Re: [ceph-users] OSD on a partition

-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 The documentation is a little sparse in this regard, here is what I use: sgdisk --new=1:0:+10240M --change-name=1:"ceph journal" --typecode=1:45b0969e-9b03-4f30-b4c6-b4b80ceff106 /dev/sdc sgdisk --new=2:0:0 --change-name=2:"ceph data" --typecode=2:4

Re: [ceph-users] OSD on a partition

Thank you! I w > On Dec 1, 2015, at 3:03 PM, Robert LeBlanc wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > The documentation is a little sparse in this regard, here is what I use: > > sgdisk --new=1:0:+10240M --change-name=1:"ceph journal" > --typecode=1:45b0969e-9b03-4f30

Re: [ceph-users] OSD on a partition

Didn’t mean to send previous email, I appologize for the spam. Anyway Thank you, I will give that a shot. Whenever I tried to use ceph-disk prepare on a partition I get OSD compalining that this isn’t a block device. Will the sgdisk typecode fix that? > On Dec 1, 2015, at 3:03 PM, Robert

Re: [ceph-users] OSD on a partition

-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 The typecode allows udev to automatically boot the OSD when it is detected. From what I remember, ceph-disk will prepare the partition regardless if the typecode is there, but it has been so long since I've tried. I've used this on Hammer as recently

Re: [ceph-users] High 0.94.5 OSD memory use at 8GB RAM/TB raw disk during recovery

On Tue, 2015-12-01 at 13:51 -0600, Ryan Tokarek wrote: > > On Nov 30, 2015, at 6:52 PM, Laurent GUERBY wrote: > > > > Hi, > > > > We lost a disk today in our ceph cluster so we added a new machine with > > 4 disks to replace the capacity and we activated straw1 tunable too > > (we also tried str

Re: [ceph-users] Number of OSD map versions

Thanks Dan, I'll use these ones from Infernalis: [global] osd map message max = 100 [osd] osd map cache size = 200 osd map max advance = 150 osd map share max epochs = 100 osd pg epoch persisted max stale = 150 George On Mon, Nov 30, 2015 at 4:20 PM, Dan van der Ster wrote: > I wouldn't ru

[ceph-users] Infernalis for Debian 8 armhf

Hi, Any plans to release Infernalis Debian 8 binary packages for armhf. As I only see it for amd64. — Swapnil Jain signature.asc Description: Message signed with OpenPGP using GPGMail ___ ceph-users mailing list ceph-users@lists.ceph.com http://l

Re: [ceph-users] High 0.94.5 OSD memory use at 8GB RAM/TB raw disk during recovery

Hi, May be a bit late now, but if you are adding additional capacity I would recommend adding one OSD at a time. If you want even more control add the OSD with a weight of 0 and gradually increase. This limits the amount of data that needs to be moved and resources required. Regards, Ross