[ceph-users] CephFS questions

Hello, everyone! I have a few questions related to the CephFS part of Ceph: - is it production ready? - can multiple CephFS be created on the same cluster? The CephFS creation page describes how to create a CephFS using (at least)

Re: [ceph-users] Uneven CPU usage on OSD nodes

Hi Craig, An uneven primaries distribution was indeed my first thought. I should have been more explicit on the percentages of the histograms I gave, lets see them in detail in a more comprehensive way. On a 27938 bench objects seen by osdmap, the hosts are distributed like that : 20904 host

[ceph-users] Ceph in Production: best practice to monitor OSD up/down status

Hello, I started to work with CEPH few weeks ago, I might ask a very newbie question, but I could not find an answer in the docs or in the ml archive for this. Quick description of my setup: I have a ceph cluster with two servers. Each server has 3 SSD drives I use for journal only. To map to dif

Re: [ceph-users] CephFS questions

Hi, Le 22/03/2015 09:29, Bogdan SOLGA a écrit : > I have a few questions related to the CephFS part of Ceph: > >- is it production ready? Currently, this is not considered "production ready". However, some people use Cephfs in production and I think they are satisfied. Personally, we are g

[ceph-users] Can't Start OSD

I have an OSD that’s failing to start. I can’t make heads or tails of the error (pasted below). Thanks! ~Noah 2015-03-22 16:32:39.265116 7f4da7fa0780 0 ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7), process ceph-osd, pid 13483 2015-03-22 16:32:39.269499 7f4da7fa0780 1 files

Re: [ceph-users] Can't Start OSD

Noah, Is this fresh installation or after upgrade ? It seems related to omap (leveldb) stuff. Thanks & Regards Somnath -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Noah Mehl Sent: Sunday, March 22, 2015 9:34 AM To: ceph-users@lists.ceph.com

Re: [ceph-users] Can't Start OSD

In production for over a year, and no upgrades. Thanks! ~Noah > On Mar 22, 2015, at 1:01 PM, Somnath Roy wrote: > > Noah, > Is this fresh installation or after upgrade ? > > It seems related to omap (leveldb) stuff. > > Thanks & Regards > Somnath > -Original Message- > From: ceph-use

Re: [ceph-users] Uneven CPU usage on OSD nodes

Hi Frederick, Need some information here. 1. Just to clarify, you are saying it is happening g in 0.87.1 and not in Firefly ? 2. Is it happening after some hours of run or just right away ? 3. Please provide ‘perf top’ output of all the OSD nodes. 4. Provide the ceph.conf file from your OSD no

Re: [ceph-users] Question Blackout

Thank you Craig thats the Kind of answer I was expecting for :) actually we have had an outage and 1 server never came up but was because of human error and 1 osd from order node were down so we Couldnt even access to the filesystem but At last I saw that the 90% of the nodes didnt crush or some

Re: [ceph-users] OSD Forece Removal

Yeah Illdo that, the version is .87.1-0.el7 Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 On Mar 20, 2015, at 5:02 PM, Robert LeBlanc mailto:rob...@leblancnet.us>> wrote: Yes, at this point,

Re: [ceph-users] Can't Start OSD

Are you seeing any error related to the disk (where OSD is mounted) in dmesg ? Could be a leveldb corruption or ceph bug. Now, unfortunately not enough log in that portion of the code base to reveal exactly why we are not getting infoos object from leveldb :-( Thanks & Regards Somnath -Origi

Re: [ceph-users] Can't Start OSD

Somnath, You are correct, there are dmesg errors about the drive. How can I replace the drive? Can I copy all of the readable contents from this drive to a new one? Because I have the following output from “ceph health detail” HEALTH_WARN 43 pgs stale; 43 pgs stuck stale pg 7.5b7 is stuck st

Re: [ceph-users] Can't Start OSD

You should be having replicated copies on other OSDs (disks), so, no need to worry about the data loss. You add a new drive and follow the steps in the following link (either 1 or 2) 1. For manual deployment, http://ceph.com/docs/master/rados/operations/add-or-rm-osds/ 2. With ceph-deploy, ht

[ceph-users] Finding out how much data is in the journal

Hi all! Trying to figure out how much my journals are used, using SSDs as journals and SATA-drives as storage, I dive into perf dump. But I can’t figure out why journal_queue_bytes is at constant 0. The only thing that differs is dirtied in WBThrottle. Maybe I’ve disable that when setting the i

[ceph-users] Giant 0.87 update on CentOs 7

All, Got a test cluster on a 4 node CentOs 7, which fails to pull updates, any hints: [root@n1 ~]# rpm -qa | grep -i ceph python-ceph-0.87-0.el7.centos.x86_64 ceph-release-1-0.el7.noarch ceph-deploy-1.5.21-0.noarch ceph-common-0.87-0.el7.centos.x86_64 libcephfs1-0.87-0.el7.centos.x86_64 ceph-0.8

Re: [ceph-users] arm cluster install

Hello Yann, Thanks for your reply. Unfortunately, I found it by chance during a search, since you didn't include me in the reply, I never got it on my email. I am interested in what you mentioned so far. I'm not looking into making any production grade cluster, just a couple of nodes for testin

Re: [ceph-users] Giant 0.87 update on CentOs 7

> On 22/03/2015, at 22.28, Steffen W Sørensen wrote: > > All, > > Got a test cluster on a 4 node CentOs 7, which fails to pull updates, any > hints: > > > [root@n1 ~]# rpm -qa | grep -i ceph > python-ceph-0.87-0.el7.centos.x86_64 > ceph-release-1-0.el7.noarch > ceph-deploy-1.5.21-0.noarch >

Re: [ceph-users] Giant 0.87 update on CentOs 7

> :) Now disabling epel which seems the confusing Repo above just renders me > with TOs from http://ceph.com … are Ceph.com > down currently? http://eu.ceph.com answers currently… properly the trans-atlantic line or my provider :/ > >

[ceph-users] About ceph-dokan

Dear All, Is there anybody use the ceph-dokan for windows client? Is it stable enough? Best Regards, Daobang Wang. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] SSD Hardware recommendation

Hi, Sorry Christian for my late answer. I was a little busy. Christian Balzer a wrote: > You're asking the wrong person, as I'm neither a Ceph or kernel > developer. ^o^ No, no, the rest of the message proves to me that I talk to the right person. ;) > Back then Mark Nelson from the Ceph team

Re: [ceph-users] SSD Hardware recommendation

On Mon, 23 Mar 2015 02:33:20 +0100 Francois Lafont wrote: > Hi, > > Sorry Christian for my late answer. I was a little busy. > > Christian Balzer a wrote: > > > You're asking the wrong person, as I'm neither a Ceph or kernel > > developer. ^o^ > > No, no, the rest of the message proves to me t

Re: [ceph-users] ceph-users Digest, Vol 26, Issue 20

hi Linke, Thanks a lot. I modified the weight os each device and the status returns HEALTH_OK. But I don't know why all the weight is 0.000. I installed the ceph through ceph-deploy. Maybe the disk size of each OSD is very low, just 10G. Regards, Guanghua > Message: 29 > Date: F

Re: [ceph-users] Finding out how much data is in the journal

On Mon, Mar 23, 2015 at 2:53 AM, Josef Johansson wrote: > Hi all! > > Trying to figure out how much my journals are used, using SSDs as journals > and SATA-drives as storage, I dive into perf dump. > But I can’t figure out why journal_queue_bytes is at constant 0. The only > thing that differs i

Re: [ceph-users] SSD Hardware recommendation

Hi, Isn't it in the nature of ssd to have write amplication ? Generaly, they have a erase block size of 128k, so the worst case could be 128/4 = 32x write amplification. (of course ssd algorithms and optimisations reduce this write amplification). Now, it could be great to see if it's coming f

Re: [ceph-users] SSD Hardware recommendation

>>(not tested, but I think with journal and O_DSYNC writes, it can give use ssd >>write amplification) also, I think that enterprise ssd with supercapacitor, should be able to cache theses o_dsync writes in the ssd buffer, and do bigger writes to reduce amplification. Don't known how ssd intern