Re: [ceph-users] Is this a deadlock?

On Wed, 4 Jan 2017 07:49:03 + 许雪寒 wrote: > Hi, everyone. > > Recently in one of our online ceph cluster, one OSD suicided itself after > experiencing some network connectivity problem, and the OSD log is as follows: > Version of Ceph and all relevant things would help. Also "some network co

[ceph-users] Ceph monitor first deployment error

Hi All, I’m new to Ceph, I’m trying to install Ceph on VMs on my laptop. I’m running CentOS Linux release 7.3.1611 (Core) with kernel 4.4.39 and Ceph 10.2.5 I’ve the following config file: [root@ceph-mon ~]# cat /etc/ceph/ceph.conf fsid = 6f34b66d-1893-4d4b-8e20-08206525a0a5 mon initial member

[ceph-users] 答复: Is this a deadlock?

Hi, thanks for the quick reply. We manually deployed this OSD, and it has been running for more than half a year. The output last night should be the latter one that you metioned Last night, one of our switch got some problem and made the OSD unconnected to other peer, which in turn made the mon

[ceph-users] Fwd: Is this a deadlock?

We've already restarted the OSD successfully. Now, we are trying to figure out why the OSD suicide itself Re: [ceph-users] Is this a deadlock? Hi, thanks for the quick reply. We manually deployed this OSD, and it has been running for more than half a year. The output last night should be the la

Re: [ceph-users] Fwd: Is this a deadlock?

On Wed, Jan 4, 2017 at 6:05 PM, 许雪寒 wrote: > We've already restarted the OSD successfully. > Now, we are trying to figure out why the OSD suicide itself Network issue which causes pretty unstable communication with other OSDs in same acting set causes suicide usually. > > Re: [ceph-users] Is thi

[ceph-users] High OSD apply latency right after new year (the leap second?)

Hi List, Three of our Ceph OSDs got unreasonably high latency right after the first second of the new year (2017/01/01 00:00:00 UTC, I have attached the metrics and I am in UTC+8 timezone). There is exactly a pg (size=3) just contains these 3 OSDs. The OSD apply latency is usually up to 25 min

Re: [ceph-users] Automatic OSD start on Jewel

On Wed, Jan 04, 2017 at 12:03:39PM +0100, Florent B wrote: > Hi everyone, > > I have a problem with automatic start of OSDs on Debian Jessie with Ceph > Jewel. > > My osd.0 is using /dev/sda5 for data and /dev/sda2 for journal, it is > listed in ceph-disk list : > > /dev/sda : > /dev/sda1 other

Re: [ceph-users] High OSD apply latency right after new year (the leap second?)

yes, same here on 3 productions clusters. no impact, but a nice happy new year alert ;) Seem that google provide ntp servers to avoid brutal 1 second leap https://developers.google.com/time/smear - Mail original - De: "Craig Chi" À: "ceph-users" Envoyé: Mercredi 4 Janvier 2017 11:2

Re: [ceph-users] Automatic OSD start on Jewel

On Wed, Jan 04, 2017 at 12:55:56PM +0100, Florent B wrote: > On 01/04/2017 12:18 PM, Fabian Grünbichler wrote: > > On Wed, Jan 04, 2017 at 12:03:39PM +0100, Florent B wrote: > >> Hi everyone, > >> > >> I have a problem with automatic start of OSDs on Debian Jessie with Ceph > >> Jewel. > >> > >> My

Re: [ceph-users] performance with/without dmcrypt OSD

Thank you Nick. To summarize - dmcrypt doesn't add notable performance impact on the ceph cluster. Thanks Swami On Tue, Jan 3, 2017 at 7:00 PM, Nick Fisk wrote: > > > > > *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf > Of *Kent Borg > *Sent:* 03 January 2017 12:47 > *

Re: [ceph-users] performance with/without dmcrypt OSD

Thanks for the link. On Tue, Jan 3, 2017 at 7:18 PM, Adrien Gillard wrote: > There has been talks on the subject in the mailing list before [1] which > concur with Nick's experience as long as you use AES-XTS. > > > [1] http://lists.ceph.com/pipermail/ceph-users-ceph.com/ > 2016-March/008444.htm

Re: [ceph-users] performance with/without dmcrypt OSD

On Tue, Jan 3, 2017 at 10:31 PM, Graham Allan wrote: > We did some CBT tests here a few months ago which included some dmcrypt > comparisons - the performance hit was non-zero, but close enough, around > ~2-3%. > > (CentOS 7.2 with E5-2630 v4 cpus, jewel release, default dmcrypt > parameters whic

Re: [ceph-users] Ceph pg active+clean+inconsistent

# ceph pg debug unfound_objects_exist FALSE Andras On 01/03/2017 11:38 PM, Shinobu Kinjo wrote: Would you run: # ceph pg debug unfound_objects_exist On Wed, Jan 4, 2017 at 5:31 AM, Andras Pataki wrote: Here is the output of ceph pg query for one of hte active+clean+inconsistent PGs: {

[ceph-users] client.admin accidently removed caps/permissions

Hello: I was trying to fix an problem with mds caps, and caused my admin user to have no mon caps. I ran: ceph auth caps client.admin mds 'allow *' I didn’t realize I had to pass the mon and osd caps as well. Now, when I try to run any command, I get 2017-01-04 08:58:44.009250 7f5441f62700 0

Re: [ceph-users] Migrate cephfs metadata to SSD in running cluster

Wido, all, can you point me to the "recent benchmarks" so I can have a look? How do you define "performance"? I would not expect cephFS throughput to change, but it is surprising to me that metadata on SSD will have no measurable effect on latency. - mike On 1/3/17 10:49 AM, Wido den Holland

Re: [ceph-users] client.admin accidently removed caps/permissions

Disregard, you can fix this by doing using the monitor id and keyring file: cd /var/lib/ceph/mon/monname ceph -n mon. --keyring keyring auth caps client.admin mds 'allow *' osd 'allow *' mon 'allow *' Sent from Mail for Windows 10 From

[ceph-users] Tonight's CDM Cancelled

Hey cephers, Given the number of devs still out on holiday I am cancelling the Ceph Developer Monthly call that was slated for 9p EST tonight. Sorry for the short notice. We'll see you in a month at the 12p time slot. Thanks. -- Best Regards, Patrick McGarry Director Ceph Community || Red Hat

Re: [ceph-users] Estimate Max IOPS of Cluster

Max iops depends on the hardware type/configuration for disks/cpu/network. For disks, the theoretical iops limit is read = physical disk iops x number of disks write (with journal on same disk) = physical disk iops x number of disks / num of replicas / 3 in practice real benchmarks will vary

Re: [ceph-users] Cephalocon Sponsorships Open

Hey Wes, We'd love to have you guys. I'll send out another note once we open the CFP though, this is just for those who wish to sponsor to help make it happen. Thanks for your interest, and keep an eye out for the CFP! :) On Thu, Dec 22, 2016 at 2:16 PM, Wes Dillingham wrote: > I / my group / o

Re: [ceph-users] Storage system

Moving this to ceph-user list where it'll get some attention. On Thu, Dec 22, 2016 at 2:08 PM, SIBALA, SATISH wrote: > Hi, > > > > Could you please give me an recommendation on kind of Ceph storage to be > used with NGINX proxy server (Object / Block / FileSystem)? > > > > Best Regards > > Satis

Re: [ceph-users] Estimate Max IOPS of Cluster

if you are asking about what tools to use: http://tracker.ceph.com/projects/ceph/wiki/Benchmark_Ceph_Cluster_Performance You should run many concurrent processes on different clients From: Maged Mokhtar Sent: Wednesday, January 04, 2017 6:45 PM To: John Petrini ; ceph-users Subject: Re: [ce

Re: [ceph-users] radosgw setup issue

> On Dec 26, 2016, at 2:48 AM, Orit Wasserman wrote: > > On Fri, Dec 23, 2016 at 3:42 AM, Kamble, Nitin A > wrote: >> I am trying to setup radosgw on a ceph cluster, and I am seeing some issues >> where google is not helping. I hope some of the developers would be able to >> help here. >> >>

Re: [ceph-users] radosgw setup issue

Regardless of whether it worked before, have you verified your RadosGWs have write access to monitors? They will need it if you want the RadosGW to create its own pools. ceph auth get On Wed, Jan 4, 2017 at 8:59 AM, Kamble, Nitin A wrote: > > > On Dec 26, 2016, at 2:48 AM, Orit Wasserman wrot

Re: [ceph-users] Estimate Max IOPS of Cluster

Thank you both for the tools an suggestions. I expected the response "there are many variables" but this gives me a place to start in determining what our configuration is capable of. ___ John Petrini NOC Systems Administrator // *CoreDial, LLC* // coredial.com // [image: Twitter]

Re: [ceph-users] Storage system

Based on this limited info, Object storage if behind proxy. We use Ceph behind HAProxy and hardware load-balancers at Bloomberg. Our Chef recipes are at https://github.com/ceph/ceph-chef and https://github.com/bloomberg/chef-bcs. The chef-bcs cookbooks show the HAProxy info. Thanks, Chris On Wed,

Re: [ceph-users] radosgw setup issue

On Wed, Jan 4, 2017 at 7:08 PM, Brian Andrus wrote: > Regardless of whether it worked before, have you verified your RadosGWs have > write access to monitors? They will need it if you want the RadosGW to > create its own pools. > > ceph auth get > I agree, it could be permissions issue > On Wed

Re: [ceph-users] Ceph - Health and Monitoring

Crai, Hi, I did not understand. I installed the plugins on the Ceph server, but what should I install on the Nagios server? The plugins installed on the Ceph server are working, but on the Nagios server they do not work, because the plugins need the installed Celp. What should I do? My plugi

Re: [ceph-users] Ceph - Health and Monitoring

I can definitely recommend Prometheus but I prefer the exporter for Ceph that I wrote :) https://github.com/jcollie/ceph_exporter On Mon, Jan 2, 2017 at 7:55 PM, Craig Chi wrote: > Hello, > > I suggest Prometheus with ceph_exporter > and Grafana

Re: [ceph-users] Pool Sizes

Think "many objects, few pools". The number of pools do not scale well because of PG limitations. Keep a small number of pools with the proper number of PGs. See this tool for pool sizing: https://ceph.com/pgcalc By default, the developers have chosen a 4MB object size for the built-in clients. T

Re: [ceph-users] Cluster pause - possible consequences

On Mon, Jan 2, 2017 at 6:46 AM, Wido den Hollander wrote: > > > Op 2 januari 2017 om 15:43 schreef Matteo Dacrema : > > > > > > Increasing pg_num will lead to several slow requests and cluster freeze, > but due to creating pgs operation , for what I’ve seen until now. > > During the creation per

Re: [ceph-users] Ceph - Health and Monitoring

actually, what you need is an ceph-common package (ubuntu) which contains /usr/bin/ceph, You have to be sure the command's going to be executed on which host. make sure the keys and ceph.conf are correctly configured on that host. you could just run the commands to make sure the configure's ok. eg