On 09/14/2017 12:44 AM, Florian Haas wrote:
On Thu, Sep 14, 2017 at 2:47 AM, Josh Durgin wrote:
On 09/13/2017 03:40 AM, Florian Haas wrote:
So we have a client that is talking to OSD 30. OSD 30 was never down;
OSD 17 was. OSD 30 is also the preferred primary for this PG (via
primary affinity)
Hi,
This's just a test cluster so, I'm just testing the relationship
between these ratios.
I did changes as such --
failsafe_full = 1 (osd failsafe full ratio = 1 in ceph.conf)
backfillfull = 0.99
nearfull = 0.95
full = 0.96
But the ceph health detail output shows a different story (different
ID CLASS WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS
8 hdd 0.18549 1.0 189G 150G 40397M 79.23 1.00 1
1 hdd 0.18549 1.0 189G 150G 40397M 79.23 1.00 1
TOTAL 379G 300G 80794M 79.23
MIN/MAX VAR: 1.00/1.00 STDDEV: 0
On Thu, Sep 14, 2017 at 7:33 PM, Ronny A
Hi，all
My cluster running 12.2.0 with bluestore， we used fio tool with librbd
ioengine make io test yesterday, and serval osds crash one after another.
3 * node, 30 OSD, 1TB SATA HDD for OSD data, 1GB SATA SSD partition for db,
576 MB SATA SSD partition for wal.
ceph options:
I just did this and backfilling started. Let's see where this takes me. ceph
osd lost 0 --yes-i-really-mean-it
Regards,Hong
On Friday, September 15, 2017 12:44 AM, hjcho616 wrote:
Ronny,
Working with all of the pgs shown in the "ceph health detail", I ran below for
each PG to export.c
Ronny,
Working with all of the pgs shown in the "ceph health detail", I ran below for
each PG to export.ceph-objectstore-tool --op export --pgid 0.1c --data-path
/var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal
--skip-journal-replay --file 0.1c.export
I have all PGs ex
Hi,
A few weeks ago after running the command to fix my object index for a
particular bucket with a lot of data (~26TB) and about 50k multipart
objects (~1800 S3 objects), the index lost track of all previous objects
and started tracking only new ones. The radosgw zone was set to
index_type: 1 wh
A serious problem of mds I think.
Anyone to fix it?
Regards.
On Thu, Sep 14, 2017 at 19:55 Micha Krause wrote:
> Hi,
>
> looking at the code, and running with debug mds = 10 it looks like I have
> an inode with negative link count.
>
> -2> 2017-09-14 13:28:39.249399 7f3919616700 10 mds.0.c
Your weights should more closely represent the size of the OSDs. OSD3 and
OSD6 are weighted properly, but your other 3 OSDs have the same weight even
though OSD0 is twice the size of OSD2 and OSD4.
Your OSD weights is what I thought you were referring to when you said you
set the crush map to 1.
Hello,
I was on a old version of ceph. And it showed a warning saying:
/crush map/ has straw_calc_version=/0/
I rode that adjusting it will only rebalance all so admin should select
when to do it. So I went straigth and ran:
ceph osd crush tunables optimal
/
/It rebalanced as it said but t
The warning you are seeing is because those settings are out of order and
it's showing you which ones are greater than the ones they should be.
backfillfull_ratio is supposed to be higher than nearfull_ratio and
osd_failsafe_full_ratio is supposed to be higher than full_ratio.
nearfull_ratio is a
Back in March 2013, there was an email thread regarding the max number of
buckets limit per account.
In summary, a guess was that millions of buckets per account would be okay,
but because the list of buckets resides on the same object, the main issue
is with concurrent writers.
There were future
This isn't a great solution, but something you could try. If you stop all
of the daemons via systemd and start them all in a screen as a manually
running daemon in the foreground of each screen... I don't think that yum
updating the packages can stop or start the daemons. You could copy and
paste
Did you configure your crush map to have that hierarchy of region,
datacenter, room, row, rack, and chassis? If you're using the default
crush map, then it has no idea about any of those places/locations. I
don't know what the crush map would look like after using that syntax if
the crush map did
Thanks Ric, thanks again Ronny.
I have a lot of good info now! I am going to try ocfs2.
Thanks
-- Jim
-Original Message-
From: Ric Wheeler [mailto:rwhee...@redhat.com]
Sent: Thursday, September 14, 2017 4:35 AM
To: Ronny Aasen; James Okken; ceph-users@lists.ceph.com
Subject: Re: [ceph-
Hi,
maybe someone has a hint: I do have a cephalopod cluster (6 nodes, 144 OSDs),
Cents 7.3 ceph 10.2.7.
I did a kernel update to the recent centos 7.3 one on a node and did a reboot.
After that, 10 OSDs did not came up as the others. The disk did not get mounted
and the OSD processes did noth
What do you mean by "updated crush map to 1"? Can you please provide a
copy of your crush map and `ceph osd df`?
On Wed, Sep 13, 2017 at 6:39 AM Gonzalo Aguilar Delgado <
gagui...@aguilardelgado.com> wrote:
> Hi,
>
> I'recently updated crush map to 1 and did all relocation of the pgs. At
> the e
I do run with osd_max_backfills and osd_recovery_max_active turned up quite a
bit from the defaults, I'm trying for as much recovery throughput as possible.
I would hazard a guess that the impact seen from the sleep settings is
proportionally much smaller if your other recovery-related parameter
On 14. sep. 2017 11:58, dE . wrote:
Hi,
I got a ceph cluster where I'm getting a OSD_OUT_OF_ORDER_FULL
health error, even though it appears that it is in order --
full_ratio 0.99
backfillfull_ratio 0.97
nearfull_ratio 0.98
These don't seem like a mistake to me but ceph is complaining --
I'm really glad to hear that it wasn't bluestore! :)
It raises another concern though. We didn't expect to see that much of a
slowdown with the current throttle settings. An order of magnitude
slowdown in recovery performance isn't ideal at all.
I wonder if we could improve things dramatical
Hello,
I'm using ceph since long time ago. A day ago added jewel requirement
for OSD. And upgraded crush map.
From this time I had all kind of errors, maybe because disks failing
because rebalances or because there's a problem I don't know.
I have some pg active+clean+inconsistent, from di
On Wed, 2017-09-13 at 14:49 +0200, Laurent GUERBY wrote:
> Hi,
>
> ceph pg repair is currently not fixing three "inconsistent" objects
> on one of our pg on a replica 3 pool. (...)
Ticket created:
http://tracker.ceph.com/issues/21388
Laurent
___
ceph
Hi,
looking at the code, and running with debug mds = 10 it looks like I have an
inode with negative link count.
-2> 2017-09-14 13:28:39.249399 7f3919616700 10 mds.0.cache.strays
eval_stray [dentry #100/stray7/17aa2f6 [2,head] auth (dversion lock) pv=0
v=23058565 inode=0x7f394b7e0730
Yeah, that hit the nail on the head. Significantly reducing/eliminating the
recovery sleep times increases the recovery speed back up (and beyond!) the
levels I was expecting to see - recovery is almost an order of magnitude faster
now. Thanks for educating me about those changes!
Rich
On 14/0
Hi Mark,
No, I wasn't familiar with that work. I am in fact comparing speed of recovery
to maintenance work I did while the cluster was in Jewel; I haven't manually
done anything to sleep settings, only adjusted max backfills OSD settings. New
options that introduce arbitrary slowdown to recove
Hi,
I got a ceph cluster where I'm getting a OSD_OUT_OF_ORDER_FULL health
error, even though it appears that it is in order --
full_ratio
0.99
backfillfull_ratio 0.97
nearfull_ratio 0.98
These don't seem like a mistake to me but ceph is complaining --
OSD_OUT_OF_ORDER_FULL full ratio(s) out o
Hi All,
We finished installed a fresh new cluster version 12 (Luminous). we used
ceph ansible.
I know since version 12 we need to associate pools to application.
I have only openstack related pools in this configuration:
images
volumes
vms
backup.
I just wanted to make sure... All this pools shoud
On 09/14/2017 11:17 AM, Ronny Aasen wrote:
On 14. sep. 2017 00:34, James Okken wrote:
Thanks Ronny! Exactly the info I need. And kinda of what I thought the answer
would be as I was typing and thinking clearer about what I was asking. I just
was hoping CEPH would work like this since the openst
On 14. sep. 2017 00:34, James Okken wrote:
Thanks Ronny! Exactly the info I need. And kinda of what I thought the answer
would be as I was typing and thinking clearer about what I was asking. I just
was hoping CEPH would work like this since the openstack fuel tools deploy CEPH
storage nodes e
On Thu, Sep 14, 2017 at 5:42 PM, Florian Haas wrote:
> On Thu, Sep 14, 2017 at 3:15 AM, Brad Hubbard wrote:
>> On Wed, Sep 13, 2017 at 8:40 PM, Florian Haas wrote:
>>> Hi everyone,
>>>
>>>
>>> disclaimer upfront: this was seen in the wild on Hammer, and on 0.94.7
>>> no less. Reproducing this on
On Thu, Sep 14, 2017 at 2:47 AM, Josh Durgin wrote:
> On 09/13/2017 03:40 AM, Florian Haas wrote:
>>
>> So we have a client that is talking to OSD 30. OSD 30 was never down;
>> OSD 17 was. OSD 30 is also the preferred primary for this PG (via
>> primary affinity). The OSD now says that
>>
>> - it
On Thu, Sep 14, 2017 at 3:15 AM, Brad Hubbard wrote:
> On Wed, Sep 13, 2017 at 8:40 PM, Florian Haas wrote:
>> Hi everyone,
>>
>>
>> disclaimer upfront: this was seen in the wild on Hammer, and on 0.94.7
>> no less. Reproducing this on 0.94.10 is a pending process, and we'll
>> update here with f
32 matches
Mail list logo
