Hi all,
we have a new issue in our Nautilus cluster.
The large omap warning seems to be more common for RGW usage, but we
currently only use CephFS and RBD. I found one thread [1] regarding
metadata pool, but it doesn't really help in our case.
The deep-scrub of PG 36.6 brought up this mess
The thresholds were recently reduced by a factor of 10. I guess you
have a lot of (open) files? Maybe use more active MDS servers?
Or increase the thresholds, I wouldn't worry at all about 200k omap
keys if you are running on reasonable hardware.
The usual argument for a low number of omap keys is
Hi. I am new to ceph but have set it up on my homelab and started using it. It
seemed very good intil I desided to try pg autoscale.
After enabling autoscale to 3 of my pools, autoscale tried(?) to reduce the
number of PGs and the pools are now unaccessible.
I have tried to turn it off again, but
On 10/1/19 12:16 PM, Raymond Berg Hansen wrote:
> Hi. I am new to ceph but have set it up on my homelab and started using it.
> It seemed very good intil I desided to try pg autoscale.
> After enabling autoscale to 3 of my pools, autoscale tried(?) to reduce the
> number of PGs and the pools a
Yes I am sure, tried to restart the whole cluster.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
Thank you, Paul.
The thresholds were recently reduced by a factor of 10. I guess you
have a lot of (open) files? Maybe use more active MDS servers?
We'll consider adding more MDS servers, although the workload hasn't
been an issue yet.
Or increase the thresholds, I wouldn't worry at all a
Hi,
we had a problem with the autoscaler just recently, we had to turn it
off because the MONs suddenly became laggy [1]. Did you check the MON
processes? Try disabling autoscaler, wait until that change is applied
and then restart MONs one by one.
Regards,
Eugen
[1]
https://lists.ceph
Thanks Poul!
For reference to everyone finding this thread, this procedure works indeed as
intended:
ceph osd getcrushmap -o crush.map
crushtool -d crush.map -o crush.txt
# edit crush rule: "step take ServerRoom class hdd" --> "step take ServerRoom
class ssd"
crushtool -o crush-new.map -c crush
Thanks for the tip, but I already tried that to. First thing I tried actually.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
It seems that the metadata pool is not affected, those PGs are all
active+clean. Are different rulesets applied to 'cephfs_data' and
'cephfs_metadata'?
Have you changed anything regarding crush rules or device classes or
something related to the ceph osd tree?
Zitat von Raymond Berg Hans
I'm running a cepf fs with an 8+2 EC data pool. Disks are on 10 hosts and
failure domain is host. Version is mimic 13.2.2. Today I added a few OSDs to
one of the hosts and observed that a lot of PGs became inactive even though 9
out of 10 hosts were up all the time. After getting the 10th host a
You are absolutly right, I had made a crush rule for device class hdd. Did not
put this in connection with this problem. When I put the pools back in the
default crush rule things are starting to fix itself it seems.
Have I done something wrong with this crush rule?
# rules
rule replicated_rule
Thank you everyone for the explanations.
Still on this subject: I created an host and attached a disk. For a second
host, for use a shared iscsi storage, I just need add disk to second client?
I tried this:
> disk add pool1/vmware_iscsi1
Warning: 'pool1/vmware_iscsi1' mapped to 1 other client(s)
o
Some time ago on Luminous I also had to change the crush rules on a all
hdd cluster to hdd (to prepare for adding ssd's and ssd pools). And pg's
started migrating while everything already was on hdd's, looks like this
is still not fixed?
-Original Message-
From: Raymond Berg Hans
Some time ago on Luminous I also had to change the crush rules on a all
hdd cluster to hdd (to prepare for adding ssd's and ssd pools). And pg's
started migrating while everything already was on hdd's, looks like this
is still not fixed?
Sage responded to a thread yesterday, how to change crush
Hi,
We have a ceph+cephfs cluster runing nautilus version 14.2.4
We have debian buster/ubuntu bionic clients mounting cephfs in kernel mode
without problems.
We now want to mount cephfs from our new centos 8 clients. Unfortunately,
ceph-common is needed but there are no packages available for el8
On Tue, 1 Oct 2019, f...@lpnhe.in2p3.fr wrote:
> Hi,
> We have a ceph+cephfs cluster runing nautilus version 14.2.4
> We have debian buster/ubuntu bionic clients mounting cephfs in kernel mode
> without problems.
> We now want to mount cephfs from our new centos 8 clients. Unfortunately,
> ceph-c
Thanks. Happy to ear that el8 packages will soon be available.
F.
By the way, it seems that mount.ceph is called by mount.
I already tryed that :
mount -t ceph 123.456.789.000:6789:/ /data -o
name=xxx_user,secretfile=/etc/ceph/client.xxx_user.keyring
and get
mount: /data: wrong fs type, bad op
On Tue, Oct 1, 2019 at 4:14 PM wrote:
>
> Thanks. Happy to ear that el8 packages will soon be available.
> F.
>
>
> By the way, it seems that mount.ceph is called by mount.
> I already tryed that :
> mount -t ceph 123.456.789.000:6789:/ /data -o
> name=xxx_user,secretfile=/etc/ceph/client.xxx_us
Hi Raymond,
I believe the "type datacenter" bit in your replicated-hdd rule should
read "type host" instead, as in the original replicated rule.
Best
Mattia
On 10/1/19 2:31 PM, Raymond Berg Hansen wrote:
> You are absolutly right, I had made a crush rule for device class hdd. Did
> not put this
On Tue, Oct 1, 2019 at 5:25 AM Frank Schilder wrote:
>
> I'm running a cepf fs with an 8+2 EC data pool. Disks are on 10 hosts and
> failure domain is host. Version is mimic 13.2.2. Today I added a few OSDs to
> one of the hosts and observed that a lot of PGs became inactive even though 9
> out
Thanks a lot ! That was the trick !!! It works.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
On 10/01/2019 07:32 AM, Gesiel Galvão Bernardes wrote:
> Thank you everyone for the explanations.
>
> Still on this subject: I created an host and attached a disk. For a
> second host, for use a shared iscsi storage, I just need add disk to
> second client?
> I tried this:
>> disk add pool1/vmwar
Hi,
Is it possible to do this scenario :
If one open a file first , he will get read/write permissions and other
will get read-only permission if they open the file after the first one.
Thanks
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubs
The standard advice is "1GB RAM per 1TB of OSD". Does this actually still hold
with large OSDs on bluestore? Can it be reasonably reduced with tuning?
>From the docs, it looks like bluestore should target the "osd_memory_target"
>value by default. This is a fixed value (4GB by default), which do
Hi list,
After the nodes ran OOM and after reboot, we are not able to restart the
ceph-osd@x services anymore. (Details about the setup at the end).
I am trying to do this manually, so we can see the error but all i see is
several crash dumps - this is just one of the OSDs which is not starting
Hi list,
After the nodes ran OOM and after reboot, we are not able to restart the
ceph-osd@x services anymore. (Details about the setup at the end).
I am trying to do this manually, so we can see the error but all i see is
several crash dumps - this is just one of the OSDs which is not starting
On Tue, Oct 1, 2019 at 9:10 AM khaled atteya wrote:
>
> Hi,
>
> Is it possible to do this scenario :
> If one open a file first , he will get read/write permissions and other will
> get read-only permission if they open the file after the first one.
CephFS will follow POSIX. If a client request
For some reason the active MGR process just resets the connection after
failover. Nothing really sticks out in the logs to explain this. However if I
restart the MGR process, it will start responding just fine.
Thoughts?
Thanks,
Matt
CONFIDENTIALITY NOTICE: Th
On Tue, Oct 1, 2019 at 6:12 PM Darrell Enns wrote:
>
> The standard advice is “1GB RAM per 1TB of OSD”. Does this actually still
> hold with large OSDs on bluestore?
No
> Can it be reasonably reduced with tuning?
Yes
> From the docs, it looks like bluestore should target the “osd_memory_targ
The problem with lots of OSDs per node is that this usually means you
have too few nodes. It's perfectly fine to run 60 OSDs per node if you
got a total of 1000 OSDs or so.
But I've seen too many setups with 3-5 nodes where each node runs 60
OSDs which makes no sense (and usually isn't even cheaper
There is a lock for object exists. If the file was not writing close, other
ones can read only.
Regards
> Am Oct 2, 2019 - 12:09 AM schrieb khaled.att...@gmail.com:
>
>
> Hi,
>
> Is it possible to do this scenario :
> If one open a file first , he will get read/write permissions and other wil
32 matches
Mail list logo
