Re: [ceph-users] scalability new node to the existing cluster

[nokia ceph]

Hi Serkan Coban,

We adapted the script and the solution you proposed is working fine . Thank
you for your support.

Thanks,
Muthu

On Wed, Apr 18, 2018 at 8:53 PM, Serkan Çoban  wrote:

> >68 OSDs per node sounds an order of magnitude above what you should be
> doing, unless you have vast experience with Ceph and its memory
> requirements under stress.
> I don't think so. We are also evaluating 90 OSDs per node. In order to
> know it works you need to test all the scenarios. Redhat supports max
> 72 OSD per host. So they are still in support limits.
>
> When QoS support arrives I hope we can put bandwidth limits to
> recovery, otherwise we need to do what is acceptable and works for
> now...
>
> On Wed, Apr 18, 2018 at 5:50 PM, Hans van den Bogert
>  wrote:
> > I keep seeing these threads where adding nodes has such an impact on the
> cluster as a whole, that I wonder what the rest of the cluster looks like.
> Normally I’d just advise someone to put a limit on the concurrent backfills
> that can be done, and `osd max backfills` by default already is 1. Could it
> be that the real culprit here is that the hardware is heavily overbooked?
> 68 OSDs per node sounds an order of magnitude above what you should be
> doing, unless you have vast experience with Ceph and its memory
> requirements under stress.
> > I wonder if this cluster would even come online after an outage, or
> would also crumble due to peering and possible backfilling.
> >
> > To be honest I don’t even get why using the weight option would solve
> this. The same amount of data needs to be transferred any way at one point;
> it seems like a poor-man’s throttling mechanism. And if memory shortage is
> the case here, due to, again, the many OSDs than the reweight strategy will
> only give you slightly better odds.
> >
> > So
> > 1) I would keep track of memory usage on the nodes to see if that
> increases under peering/backfilling,
> >   - If this is the case, and you’re using bluestore: try lowering
> bluestore_cache_size* params, to give you some leeway.
> > 2) If using bluestore, try throttling by changing the following params,
> depending on your environment:
> >   - osd recovery sleep
> >   - osd recovery sleep hdd
> >   - osd recovery sleep ssd
> >
> > There are other throttling params you can change, though most defaults
> are just fine in my environment, and I don’t have experience with them.
> >
> > Good luck,
> >
> > Hans
> >
> >
> >> On Apr 18, 2018, at 1:32 PM, Serkan Çoban 
> wrote:
> >>
> >> You can add new OSDs with 0 weight and edit below script to increase
> >> the osd weights instead of decreasing.
> >>
> >> https://github.com/cernceph/ceph-scripts/blob/master/
> tools/ceph-gentle-reweight
> >>
> >>
> >> On Wed, Apr 18, 2018 at 2:16 PM, nokia ceph 
> wrote:
> >>> Hi All,
> >>>
> >>> We are having 5 node cluster with EC 4+1 . Each node has 68 HDD . Now
> we are
> >>> trying to add new node with 68 disks to the cluster .
> >>>
> >>> We tried to add new node and created all OSDs in one go , the cluster
> >>> stopped all client traffic and does only backfilling .
> >>>
> >>> Any procedure to add the new node without affecting the client traffic
> ?
> >>>
> >>> If we create  OSDs one by one , then there is no issue in client
> traffic
> >>> however  time taken to add new node with 68 disks will be several
> months.
> >>>
> >>> Please provide your suggestions..
> >>>
> >>> Thanks,
> >>> Muthu
> >>>
> >>>
> >>> ___
> >>> ceph-users mailing list
> >>> ceph-users@lists.ceph.com
> >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>>
> >> ___
> >> ceph-users mailing list
> >> ceph-users@lists.ceph.com
> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ceph 12.2.4 MGR spams syslog with "mon failed to return metadata for mds"

[Marc Roos]

 
Remote syslog server, and buffering writes to the log?


Actually this is another argument to fix logging to syslog a bit, 
because the default syslog is also be set to throttle and group the 
messages like:

Mar 9 17:59:35 db1 influxd: last message repeated 132 times


https://www.mail-archive.com/ceph-users@lists.ceph.com/msg45025.html




-Original Message-
From: Charles Alva [mailto:charlesa...@gmail.com] 
Sent: vrijdag 20 april 2018 8:08
To: ceph-users@lists.ceph.com
Subject: [ceph-users] Ceph 12.2.4 MGR spams syslog with "mon failed to 
return metadata for mds"

Hi All,

Just noticed on 2 Ceph Luminous 12.2.4 clusters, Ceph mgr spams the 
syslog with lots of "mon failed to return metadata for mds" every 
second.

```
2018-04-20 06:06:03.951412 7fca238ff700  1 mgr send_beacon active 
2018-04-20 06:06:04.934477 7fca14809700  0 ms_deliver_dispatch: 
unhandled message 0x55bf897f0a00 mgrreport(mds.mds1 +24-0 packed 214) v5 
from mds.0 10.100.100.114:6800/4132681434 2018-04-20 06:06:04.934937 
7fca25102700  1 mgr finish mon failed to return metadata for mds.mds1: 
(2) No such file or directory ```

How to fix this issue? or disable it completely to reduce disk IO and 
increase SSD life span?



Kind regards,

Charles Alva
Sent from Gmail Mobile



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Hello

[朱顺意]

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] London Ceph day yesterday

[Sean Purdy]

Just a quick note to say thanks for organising the London Ceph/OpenStack day.  
I got a lot out of it, and it was nice to see the community out in force.

Sean Purdy
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ceph 12.2.4 MGR spams syslog with "mon failed to return metadata for mds"

[Charles Alva]

Hi Marc,

I'm using CephFS and mgr could not get the metadata of the mds. I enabled
the dashboard module and everytime I visit the ceph filesystem page, it got
internal error 500.

Kind regards,

Charles Alva
Sent from Gmail Mobile

On Fri, Apr 20, 2018 at 9:24 AM, Marc Roos  wrote:

>
> Remote syslog server, and buffering writes to the log?
>
>
> Actually this is another argument to fix logging to syslog a bit,
> because the default syslog is also be set to throttle and group the
> messages like:
>
> Mar 9 17:59:35 db1 influxd: last message repeated 132 times
>
>
> https://www.mail-archive.com/ceph-users@lists.ceph.com/msg45025.html
>
>
>
>
> -Original Message-
> From: Charles Alva [mailto:charlesa...@gmail.com]
> Sent: vrijdag 20 april 2018 8:08
> To: ceph-users@lists.ceph.com
> Subject: [ceph-users] Ceph 12.2.4 MGR spams syslog with "mon failed to
> return metadata for mds"
>
> Hi All,
>
> Just noticed on 2 Ceph Luminous 12.2.4 clusters, Ceph mgr spams the
> syslog with lots of "mon failed to return metadata for mds" every
> second.
>
> ```
> 2018-04-20 06:06:03.951412 7fca238ff700  1 mgr send_beacon active
> 2018-04-20 06:06:04.934477 7fca14809700  0 ms_deliver_dispatch:
> unhandled message 0x55bf897f0a00 mgrreport(mds.mds1 +24-0 packed 214) v5
> from mds.0 10.100.100.114:6800/4132681434 2018-04-20 06:06:04.934937
> 7fca25102700  1 mgr finish mon failed to return metadata for mds.mds1:
> (2) No such file or directory ```
>
> How to fix this issue? or disable it completely to reduce disk IO and
> increase SSD life span?
>
>
>
> Kind regards,
>
> Charles Alva
> Sent from Gmail Mobile
>
>
>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Tens of millions of objects in a sharded bucket

[Rafał Wądołowski]

Hi,

We have some buckets with ~25M files inside.

We've also using bucket index sharding. The performance is good, we are
focused on read.



BR,
Rafal Wadolowski


On 20.04.2018 00:57, Robert Stanford wrote:
>
>  The rule of thumb is not to have tens of millions of objects in a
> radosgw bucket, because reads will be slow.  If using bucket index
> sharding (with 128 or 256 shards), does this eliminate this concern? 
> Has anyone tried tens of millions (20-40M) of objects with sharded
> indexes?
>
>  Thank you
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ceph 12.2.4 MGR spams syslog with "mon failed to return metadata for mds"

[Marc Roos]

 
Hi Charles, 

I am more or less responding to your syslog issue. I don’t have the 
experience on cephfs to give you a reliable advice. So lets wait for the 
experts to reply. But I guess you have to give a little more background 
info, like

This happened to running cluster, you didn’t apply any changes to?
Looks like your dashboard issue is not related to "1 mgr finish mon 
failed to return metadata for mds.mds1"


-Original Message-
From: Charles Alva [mailto:charlesa...@gmail.com] 
Sent: vrijdag 20 april 2018 10:33
To: Marc Roos
Cc: ceph-users
Subject: Re: [ceph-users] Ceph 12.2.4 MGR spams syslog with "mon failed 
to return metadata for mds"

Hi Marc,

I'm using CephFS and mgr could not get the metadata of the mds. I 
enabled the dashboard module and everytime I visit the ceph filesystem 
page, it got internal error 500.

Kind regards,

Charles Alva
Sent from Gmail Mobile


On Fri, Apr 20, 2018 at 9:24 AM, Marc Roos  
wrote:


 
Remote syslog server, and buffering writes to the log?


Actually this is another argument to fix logging to syslog a bit, 
because the default syslog is also be set to throttle and group the 

messages like:

Mar 9 17:59:35 db1 influxd: last message repeated 132 times


https://www.mail-archive.com/ceph-users@lists.ceph.com/msg45025.htm
l 
 





-Original Message-
From: Charles Alva [mailto:charlesa...@gmail.com] 
Sent: vrijdag 20 april 2018 8:08
To: ceph-users@lists.ceph.com
Subject: [ceph-users] Ceph 12.2.4 MGR spams syslog with "mon failed 
to 
return metadata for mds"

Hi All,

Just noticed on 2 Ceph Luminous 12.2.4 clusters, Ceph mgr spams the 

syslog with lots of "mon failed to return metadata for mds" every 
second.

```
2018-04-20 06:06:03.951412 7fca238ff700  1 mgr send_beacon active 
2018-04-20 06:06:04.934477 7fca14809700  0 ms_deliver_dispatch: 
unhandled message 0x55bf897f0a00 mgrreport(mds.mds1 +24-0 packed 
214) v5 
from mds.0 10.100.100.114:6800/4132681434 2018-04-20 
06:06:04.934937 
7fca25102700  1 mgr finish mon failed to return metadata for 
mds.mds1: 
(2) No such file or directory ```

How to fix this issue? or disable it completely to reduce disk IO 
and 
increase SSD life span?



Kind regards,

Charles Alva
Sent from Gmail Mobile







___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] What are the current rados gw pools

[Marc Roos]

I manually created the radosgw pools from serveral different sources. 
And I have it sort of running. But as you can see some pools stay empty. 


Can I delete all used 0 pools?

POOL_NAME  USED   
.intent-log 0 
.rgw.buckets0 
.rgw.buckets.extra  0 
.rgw.buckets.index  0 
.rgw.gc 0 
.rgw.root1113 
.usage  0 
.users  0 
.users.email0 
.users.swift0 
.users.uid  0 
default.rgw 0 
default.rgw.buckets.data6046M 
default.rgw.buckets.index   0 
default.rgw.buckets.non-ec  0 
default.rgw.control 0 
default.rgw.log  2431 
default.rgw.meta 4702 

Would be nice if this could be put on the manual here
http://docs.ceph.com/docs/master/install/install-ceph-gateway/#
  
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Creating first Ceph cluster

[Shantur Rathore]

Thanks Alfredo.  I will use ceph-volume.

On Thu, Apr 19, 2018 at 4:24 PM, Alfredo Deza  wrote:

> On Thu, Apr 19, 2018 at 11:10 AM, Shantur Rathore
>  wrote:
> > Hi,
> >
> > I am building my first Ceph cluster from hardware leftover from a
> previous
> > project. I have been reading a lot of Ceph documentation but need some
> help
> > to make sure I going the right way.
> > To set the stage below is what I have
> >
> > Rack-1
> >
> > 1 x HP DL360 G9 with
> >- 256 GB Memory
> >- 5 x 300GB HDD
> >- 2 x HBA SAS
> >- 4 x 10GBe Networking Card
> >
> > 1 x SuperMicro chassis with 17 x HP Enterprise 400GB SSD and 17 x HP
> > Enterprise 1.7TB HDD
> > Chassis and HP server are connected with 2 x SAS HBA for redundancy.
> >
> >
> > Rack-2 (Same as Rack-1)
> >
> > 1 x HP DL360 G9 with
> >- 256 GB Memory
> >- 5 x 300GB HDD
> >- 2 x HBA SAS
> >- 4 x 10GBe Networking Card
> >
> > 1 x SuperMicro chassis with 17 x HP Enterprise 400GB SSD and 17 x HP
> > Enterprise 1.7TB HDD
> > Chassis and HP server are connected with 2 x SAS HBA for redundancy.
> >
> >
> > Rack-3
> >
> > 5 x HP DL360 G8 with
> >- 128 GB Memory
> >- 2 x 400GB HP Enterprise SSD
> >- 3 x 1.7TB Enterprise HDD
> >
> > Requirements
> > - To serve storage to around 200 VMware VMs via iSCSI. VMs use disks
> > moderately.
> > - To serve storage to some docker containers using ceph volume driver
> > - To serve storage to some legacy apps using NFS
> >
> > Plan
> >
> > - Create a ceph cluster with all machines
> > - Use Bluestore as osd backing ( 3 x SSD for DB and WAL in SuperMicro
> > Chassis and 1 x SSD for DB and WAL in Rack 3 G8s)
> > - Use remaining SSDs ( 14 x in SuperMicro and 1 x Rack 3 G8s ) for Rados
> > Cache Tier
> > - Update CRUSH map to make Rack as minimum failure domain. So almost all
> > data is replicated across racks and in case one of the host dies the
> storage
> > still works.
> > - Single bonded network (4x10GBe) connected to ToR switches.
> > - Same public and cluster network
> >
> > Questions
> >
> > - First of all, is this kind of setup workable.
> > - I have seen that Ceph uses /dev/sdx names in guides, is it a good
> approach
> > considering the disks die and can come up with different /dev/sdx
> identifier
> > on reboot.
>
> In the case of ceph-volume, these will not matter since it uses LVM
> behind the scenes and LVM takes care of figuring out if /dev/sda1 is
> now really /dev/sdb1 after
> a reboot.
>
> If using ceph-disk however, the detection is done a bit differently,
> by reading partition labels and depending on UDEV triggers that
> sometimes can be troublesome, specially
> on reboot. In the case of a successful detection via UDEV the
> non-persistent names wouldn't matter much still.
>
> > - What should be the approx size of WAL and DB partitions for my kind of
> > setup?
> > - Can i install ceph in a VM and use other VMs on these hosts. Is Ceph
> too
> > CPU demanding?
> >
> > Thanks,
> > Shantur
> >
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Radosgw switch from replicated to erasure

[Marc Roos]

I want to start using the radowsgw a bit. For now I am fine with the 3 
replicated setup, in the near future when I add a host. I would like to 
switch to ec, is there something I should do now to make this switch 
more smoothly?






___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Using ceph deploy with mon.a instead of mon.hostname?

[Oliver Schulz]

Dear Ceph Experts,

I'm try to switch an old Ceph cluster from manual administration to
ceph-deploy, but I'm running into the following error:

# ceph-deploy gatherkeys HOSTNAME

[HOSTNAME][INFO  ] Running command: /usr/bin/ceph --connect-timeout=25 
--cluster=ceph --admin-daemon=/var/run/ceph/ceph-mon.HOSTNAME.asok mon_status
[HOSTNAME][ERROR ] "ceph mon_status HOSTNAME" returned 22
[HOSTNAME][DEBUG ] admin_socket: exception getting command descriptions: [Errno 
2] No such file or directory

The problem seems to stem from the fact that the cluster still uses
the naming scheme "mon.a", "mon.b", etc., while ceph-deploy assumes
"mon.HOSTNAME".

Is the a workaround for this, resp. a way to fix the cluster? Or
will it be necessary to recreate all mons?


Cheers,

Oliver
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Radosgw switch from replicated to erasure

[Janne Johansson]

2018-04-20 6:06 GMT+02:00 Marc Roos :

>
> I want to start using the radowsgw a bit. For now I am fine with the 3
> replicated setup, in the near future when I add a host. I would like to
> switch to ec, is there something I should do now to make this switch
> more smoothly?
>

That will not be super-easy if you can't have a bit of downtime:

http://cephnotes.ksperis.com/blog/2015/04/15/ceph-pool-migration/

The comments also seem to hint at some methods being deprecated or at least
have more risk than you would imagine.

-- 
May the most significant bit of your life be positive.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ceph 12.2.4 MGR spams syslog with "mon failed to return metadata for mds"

[Charles Alva]

Marc,

Thanks.

The mgr log spam occurs even without dashboard module enabled. I never
checked the ceph mgr log before because the ceph cluster is always healthy.
Based on the ceph mgr logs in syslog, the spam occurred long before and
after I enabled the dashboard module.

# ceph -s
>   cluster:
> id: xxx
> health: HEALTH_OK
>
>   services:
> mon: 3 daemons, quorum mon1,mon2,mon3
> mgr: mon1(active), standbys: mon2, mon3
> mds: cephfs-1/1/1 up  {0=mds1=up:active}, 2 up:standby
> osd: 14 osds: 14 up, 14 in
> rgw: 3 daemons active
>
>   data:
> pools:   10 pools, 248 pgs
> objects: 546k objects, 2119 GB
> usage:   6377 GB used, 6661 GB / 13039 GB avail
> pgs: 248 active+clean
>
>   io:
> client:   25233 B/s rd, 1409 kB/s wr, 6 op/s rd, 59 op/s wr



My ceph mgr log is spam with following log every second. This happens on 2
separate Ceph 12.2.4 clusters.

# less +F /var/log/ceph/ceph-mgr.mon1.log

 ...

2018-04-20 06:21:18.782861 7fca238ff700  1 mgr send_beacon active
> 2018-04-20 06:21:19.050671 7fca14809700  0 ms_deliver_dispatch: unhandled
> message 0x55bf897d1c00 mgrreport(mds.mds1 +24-0 packed 214) v5 from mds.0
> 10.100.100.114:6800/4132681434
> 2018-04-20 06:21:19.051047 7fca25102700  1 mgr finish mon failed to return
> metadata for mds.mds1: (2) No such file or directory
> 2018-04-20 06:21:20.050889 7fca14809700  0 ms_deliver_dispatch: unhandled
> message 0x55bf897eac00 mgrreport(mds.mds1 +24-0 packed 214) v5 from mds.0
> 10.100.100.114:6800/4132681434
> 2018-04-20 06:21:20.051351 7fca25102700  1 mgr finish mon failed to return
> metadata for mds.mds1: (2) No such file or directory
> 2018-04-20 06:21:20.784455 7fca238ff700  1 mgr send_beacon active
> 2018-04-20 06:21:21.050968 7fca14809700  0 ms_deliver_dispatch: unhandled
> message 0x55bf897d0d00 mgrreport(mds.mds1 +24-0 packed 214) v5 from mds.0
> 10.100.100.114:6800/4132681434
> 2018-04-20 06:21:21.051441 7fca25102700  1 mgr finish mon failed to return
> metadata for mds.mds1: (2) No such file or directory
> 2018-04-20 06:21:22.051254 7fca14809700  0 ms_deliver_dispatch: unhandled
> message 0x55bf897ec100 mgrreport(mds.mds1 +24-0 packed 214) v5 from mds.0
> 10.100.100.114:6800/4132681434
> 2018-04-20 06:21:22.051704 7fca25102700  1 mgr finish mon failed to return
> metadata for mds.mds1: (2) No such file or directory
> 2018-04-20 06:21:22.786656 7fca238ff700  1 mgr send_beacon active
> 2018-04-20 06:21:23.051235 7fca14809700  0 ms_deliver_dispatch: unhandled
> message 0x55bf897d0400 mgrreport(mds.mds1 +24-0 packed 214) v5 from mds.0
> 10.100.100.114:6800/4132681434
> 2018-04-20 06:21:23.051712 7fca25102700  1 mgr finish mon failed to return
> metadata for mds.mds1: (2) No such file or directory
> 2018-04-20 06:21:24.051353 7fca14809700  0 ms_deliver_dispatch: unhandled
> message 0x55bf897e6000 mgrreport(mds.mds1 +24-0 packed 214) v5 from mds.0
> 10.100.100.114:6800/4132681434
> 2018-04-20 06:21:24.051971 7fca25102700  1 mgr finish mon failed to return
> metadata for mds.mds1: (2) No such file or directory
> 2018-04-20 06:21:24.788228 7fca238ff700  1 mgr send_beacon active
> 2018-04-20 06:21:25.051642 7fca14809700  0 ms_deliver_dispatch: unhandled
> message 0x55bf897d1900 mgrreport(mds.mds1 +24-0 packed 214) v5 from mds.0
> 10.100.100.114:6800/4132681434
> 2018-04-20 06:21:25.052182 7fca25102700  1 mgr finish mon failed to return
> metadata for mds.mds1: (2) No such file or directory
> 2018-04-20 06:21:26.051641 7fca14809700  0 ms_deliver_dispatch: unhandled
> message 0x55bf89835600 mgrreport(mds.mds1 +24-0 packed 214) v5 from mds.0
> 10.100.100.114:6800/4132681434
> 2018-04-20 06:21:26.052169 7fca25102700  1 mgr finish mon failed to return
> metadata for mds.mds1: (2) No such file or directory
> ...


Kind regards,

Charles Alva
Sent from Gmail Mobile

On Fri, Apr 20, 2018 at 10:57 AM, Marc Roos 
wrote:

>
> Hi Charles,
>
> I am more or less responding to your syslog issue. I don’t have the
> experience on cephfs to give you a reliable advice. So lets wait for the
> experts to reply. But I guess you have to give a little more background
> info, like
>
> This happened to running cluster, you didn’t apply any changes to?
> Looks like your dashboard issue is not related to "1 mgr finish mon
> failed to return metadata for mds.mds1"
>
>
> -Original Message-
> From: Charles Alva [mailto:charlesa...@gmail.com]
> Sent: vrijdag 20 april 2018 10:33
> To: Marc Roos
> Cc: ceph-users
> Subject: Re: [ceph-users] Ceph 12.2.4 MGR spams syslog with "mon failed
> to return metadata for mds"
>
> Hi Marc,
>
> I'm using CephFS and mgr could not get the metadata of the mds. I
> enabled the dashboard module and everytime I visit the ceph filesystem
> page, it got internal error 500.
>
> Kind regards,
>
> Charles Alva
> Sent from Gmail Mobile
>
>
> On Fri, Apr 20, 2018 at 9:24 AM, Marc Roos 
> wrote:
>
>
>
> Remote syslog server, and buffering writes to the log?
>
>
> Actually this is another

Re: [ceph-users] Using ceph deploy with mon.a instead of mon.hostname?

[Stefan Kooman]

Quoting Oliver Schulz (oliver.sch...@tu-dortmund.de):
> Dear Ceph Experts,
> 
> I'm try to switch an old Ceph cluster from manual administration to
> ceph-deploy, but I'm running into the following error:
> 
> # ceph-deploy gatherkeys HOSTNAME
> 
> [HOSTNAME][INFO  ] Running command: /usr/bin/ceph --connect-timeout=25 
> --cluster=ceph --admin-daemon=/var/run/ceph/ceph-mon.HOSTNAME.asok mon_status
> [HOSTNAME][ERROR ] "ceph mon_status HOSTNAME" returned 22
> [HOSTNAME][DEBUG ] admin_socket: exception getting command descriptions: 
> [Errno 2] No such file or directory
> 
> The problem seems to stem from the fact that the cluster still uses
> the naming scheme "mon.a", "mon.b", etc., while ceph-deploy assumes
> "mon.HOSTNAME".
> 
> Is the a workaround for this, resp. a way to fix the cluster? Or
> will it be necessary to recreate all mons?

I believe it might have to do with the requirement for "ceph-deploy" to
have a "mon_initial_members = your MONs here". Do you have that in your
ceph.conf file on the node you are deploying from? 

Gr. Stefan


-- 
| BIT BV  http://www.bit.nl/Kamer van Koophandel 09090351
| GPG: 0xD14839C6   +31 318 648 688 / i...@bit.nl
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Using ceph deploy with mon.a instead of mon.hostname?

[Oliver Schulz]

Dear Stefan,

thanks, I tried your suggestion. Unfortunately no matter whether I put

mon_initial_members = hostname1,hostname2,hostname3

or

mon_initial_members = a,b,c

into ceph.conf (both on deployment and mon host),
"ceph-deploy gatherkeys" still tries to use
"--admin-daemon=/var/run/ceph/ceph-mon.ceph-node10.asok"
instead of "--admin-daemon=/var/run/ceph/ceph-mon.a.asok".

When I create "/var/run/ceph/ceph-mon.ceph-node10.asok"
on the node (as a symlink), I get another error instead, one
step later:

[ceph-node10][ERROR ] could not find 'MON_HOSTNAME' in monmap

Any ideas?


Cheers,

Oliver


On 04/20/2018 11:46 AM, Stefan Kooman wrote:

Quoting Oliver Schulz (oliver.sch...@tu-dortmund.de):

Dear Ceph Experts,

I'm try to switch an old Ceph cluster from manual administration to
ceph-deploy, but I'm running into the following error:

# ceph-deploy gatherkeys HOSTNAME

[HOSTNAME][INFO  ] Running command: /usr/bin/ceph --connect-timeout=25 
--cluster=ceph --admin-daemon=/var/run/ceph/ceph-mon.HOSTNAME.asok mon_status
[HOSTNAME][ERROR ] "ceph mon_status HOSTNAME" returned 22
[HOSTNAME][DEBUG ] admin_socket: exception getting command descriptions: [Errno 
2] No such file or directory

The problem seems to stem from the fact that the cluster still uses
the naming scheme "mon.a", "mon.b", etc., while ceph-deploy assumes
"mon.HOSTNAME".

Is the a workaround for this, resp. a way to fix the cluster? Or
will it be necessary to recreate all mons?


I believe it might have to do with the requirement for "ceph-deploy" to
have a "mon_initial_members = your MONs here". Do you have that in your
ceph.conf file on the node you are deploying from?

Gr. Stefan




___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] OSDs not starting if the cluster name is not ceph

[Robert Stanford]

 If I use another cluster name (other than the default "ceph"), I've
learned that I have to create symlinks in /var/lib/ceph/osd/ with
[cluster-name]-[osd-num] that symlink to ceph-[osd-num].  The ceph-disk
command doesn't seem to take a --cluster argument like other commands.

 Is this a known issue, or am I missing something?
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] OSDs not starting if the cluster name is not ceph

[Gregory Farnum]

Not sure about this specific issue, but I believe we've deprecated the use
of cluster names due to (very) low usage and trouble reliably testing for
all the little things like this. :/
-Greg

On Fri, Apr 20, 2018 at 10:18 AM Robert Stanford 
wrote:

>
>  If I use another cluster name (other than the default "ceph"), I've
> learned that I have to create symlinks in /var/lib/ceph/osd/ with
> [cluster-name]-[osd-num] that symlink to ceph-[osd-num].  The ceph-disk
> command doesn't seem to take a --cluster argument like other commands.
>
>  Is this a known issue, or am I missing something?
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] OSDs not starting if the cluster name is not ceph

[Robert Stanford]

 Thanks Gregory.  How much trouble I'd have saved if I'd only known this...

On Fri, Apr 20, 2018 at 3:41 PM, Gregory Farnum  wrote:

> Not sure about this specific issue, but I believe we've deprecated the use
> of cluster names due to (very) low usage and trouble reliably testing for
> all the little things like this. :/
> -Greg
>
> On Fri, Apr 20, 2018 at 10:18 AM Robert Stanford 
> wrote:
>
>>
>>  If I use another cluster name (other than the default "ceph"), I've
>> learned that I have to create symlinks in /var/lib/ceph/osd/ with
>> [cluster-name]-[osd-num] that symlink to ceph-[osd-num].  The ceph-disk
>> command doesn't seem to take a --cluster argument like other commands.
>>
>>  Is this a known issue, or am I missing something?
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Acl's set on bucket, but bucket not visible in users account

[Marc Roos]

I have set acl's on a bucket via cyberduck.

I can see them being set via s3cmd, yet I don’t see the bucket in Test2 
user's account. Should I do more than add just an acl to a bucket? Has 
this to do with multi-tenancy users "test$tester1", "test$tester2"?

[@~]$ s3cmd info s3://test
s3://test/ (bucket):
   Location:  us-east-1
   Payer: BucketOwner
   Expiration Rule: none
   policy:none
   cors:  none
   ACL:   Test1 User: FULL_CONTROL
   ACL:   Test2 User: WRITE

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com