rmail/ceph-users-ceph.com/2017-January/015550.html
пн, 1 апр. 2019 г. в 12:23, Vladimir Prokofev :
> As we fixed failed node next day, cluster rebalanced to it's original
> state without any issues, so crush dump would be irrelevant at this point I
> guess. Will have to wait for ne
uot;: 1,
"require_feature_tunables2": 1,
"has_v2_rules": 0,
"require_feature_tunables3": 1,
"has_v3_rules": 0,
"has_v4_buckets": 0,
"require_feature_tunables5": 0,
"has_v5_rules": 0
CEPH 12.2.11, pool size 3, min_size 2.
One node went down today(private network interface started flapping, and
after a while OSD processes crashed), no big deal, cluster recovered, but
not completely. 1 PG stuck in active+clean+remapped state.
PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACE
Your output looks a bit weird, but still, this is normal for bluestore. It
creates small separate data partition that is presented as XFS mounted in
/var/lib/ceph/osd, while real data partition is hidden as raw(bluestore)
block device.
It's no longer possible to check disk utilisation with df using
Hello list.
Today while redeploying an OSD I've noticed that links to DB/WAL devices
are pointing to partitions themselves, not to the partition UUID how it was
before.
I think that changed with latest ceph-deploy.
I'm using 12.2.2 on my mon/osd nodes.
ceph-deploy is 2.0.1 on admin node.
All node
Thank you all for your input.
My best guess at the moment is that deep-scrub performs as it should, and
the issue is that it just has no limits on its performance, so it uses all
the OSD time it can. Even if it has lower priority than client IO, it still
can fill disk queue, and effectively bottlen
Hello list.
Deep scrub totally kills cluster performance.
First of all, it takes several minutes to complete:
2018-12-09 01:39:53.857994 7f2d32fde700 0 log_channel(cluster) log [DBG] :
4.75 deep-scrub starts
2018-12-09 01:46:30.703473 7f2d32fde700 0 log_channel(cluster) log [DBG] :
4.75 deep-scr
I'm running small CEPH cluster of 9 OSD nodes, with systems hosted on USB
sticks exactly for the same reason - not enough disk slots. Works fine for
almost 2 years now.
2018-08-15 1:13 GMT+03:00 Paul Emmerich :
> I've seen the OS running on SATA DOMs and cheap USB sticks.
> It works well for some
on check if this issue is reproducible? If so
- this seems to be a ticket-worthy.
2018-03-30 17:40 GMT+03:00 Vladimir Prokofev :
> CEPH 12.2.2, RGW.
> I'm using it as an object storage endpoint for Openstack.
>
> Recently while browsing an object storage from Horizon, I accident
CEPH 12.2.2, RGW.
I'm using it as an object storage endpoint for Openstack.
Recently while browsing an object storage from Horizon, I accidently marked
container as public. The issue is - I can't make it private again!
Docs state that to do it I should simply delete X-Container-Read metadata,
but
My first guess would be PG overdose protection kicked in [1][2]
You can try fixing it by increasing allowed number of PG per OSD with
ceph tell mon.* injectargs '--mon_max_pg_per_osd 500'
ceph tell osd.* injectargs '--mon_max_pg_per_osd 500'
and then triggering CRUSH algorithm update by restarting
e_Celsius 0x0022 031 017 000Old_age Always
> - 31 (0 17 0 0 0)
> 195 Hardware_ECC_Recovered 0x001a 005 001 000Old_age Always
> - 224621855
> 197 Current_Pending_Sector 0x0012 100 100 000Old_age Always
> - 0
> candidate had a read error
speaks for itself - while scrubbing it coudn't read data.
I had similar issue, and it was just OSD dying - errors and relocated
sectors in SMART, just replaced the disk. But in your case it seems that
errors are on different OSDs? Are your OSDs all healthy?
You can use
I'll pitch in my personal expirience.
When single OSD in a pool becomes full(95% used), then all client IO writes
to this pool must stop, even if other OSDs are almost free. This is done
for the purpose of data intergity. [1]
To avoid this you need to balance your failure domains.
For example, ass
In my case we don't even use "default" ruleset in CRUSH - no pool has this
ruleset associated with it. Adding OSDs doesn't lead to any PG
recalculation or data movement. It's triggered only after modifying CRUSH
map and placing OSDs in appropriate failure domain.
This way you can add any number of
CEPH 12.2.2
So I have a snapshot of an image:
$ rbd du ssd-volumes/volume-4fed3f31-0802-4850-91bc-17e6da05697d
NAME
PROVISIONED USED
volume-4fed3f31-0802-4850-91bc-17e6da05697d@snapshot-d2145e21-99a7-4e2e-9138-ab3e975f8113
20480M 8088M
volume-4fed3f31-0802-4850-91bc-17e6da05697
Hello.
In short: what are the consequence of loosing external WAL/DB
device(assuming it’s SSD) in bluestore?
In comparison with filestore - we used to have an external SSD for
journaling multiple HDD OSDs. Hardware failure of such a device would not
be that big of a deal, as we can quickly use xf
Increased number of PGs in multiple pools in a production cluster on 12.2.2
recently - zero issues.
CEPH claims that increasing pg_num and pgp_num are safe operations, which
are essential for it's ability to scale, and this sounds pretty reasonable
to me. [1]
[1] https://www.sebastien-han.fr/blog
ad no real impact. The fix was also quite easy to figure out.
2017-12-20 4:03 GMT+03:00 Brad Hubbard :
> Version?
>
> See http://tracker.ceph.com/issues/22346 for a (limited) explanation.
>
> On Tue, Dec 19, 2017 at 6:35 PM, Vladimir Prokofev wrote:
> > Took a little walk
OSDs.
Case closed.
2017-12-19 10:39 GMT+03:00 Vladimir Prokofev :
> Hello.
>
> After some furious "ceph-deploy osd prepare/osd zap" cycles to figure out
> a correct command for ceph-deploy to create a bluestore HDD with wal/db
> SSD, I now have orphant OSDs, which are now
Hello.
After some furious "ceph-deploy osd prepare/osd zap" cycles to figure out a
correct command for ceph-deploy to create a bluestore HDD with wal/db SSD,
I now have orphant OSDs, which are nowhere to be found in CRUSH map!
$ ceph health detail
HEALTH_WARN 4 osds exist in the crush map but not
> So, filestore:
> - write data to the journal
> - write metadata
> - move data from journal to definitive storage
>
> Bluestore:
> - write data to definitive storage (free space, not overwriting anything)
> - write metadata
Wait, doesn't that mean that when I move to Bluestore from Filestore I'll
ion more than ceph question, but still.
Surely "ceph tell" is a goto solution for any configuration updates.
2017-05-12 17:42 GMT+03:00 mj :
> Hi,
>
> On 05/12/2017 03:35 PM, Vladimir Prokofev wrote:
>
>> My best guess is that using systemd you can write some basic sc
Intersting question. AFAIK, there's no built-in solution.
Also, if you think about it, restarting whole cluster at once can lead to
service interruption, as you can easily bring all PG copies in stale state
for a short time, and even longer if some OSDs won't go up for some reason.
You should reall
Hello ceph-users.
Short description: during snapshot removal osd usilisation goes up to 100%,
which leads to slow requests and VM failures due to IOPS stall.
We're using Openstack Cinder with CEPH cluster as a volume backend. CEPH
version is 10.2.6.
We also using cinder-backup to create backups o
25 matches
Mail list logo
