Hi Stefan,
its all at the defaults it seems:
[root@gnosis ~]# ceph config get osd.243 bluestore_cache_size
0
[root@gnosis ~]# ceph config get osd.243 bluestore_cache_size_ssd
3221225472
I explicitly removed the old settings with commands like
ceph config rm osd.243 bluestore_cache_size
Best re
Dear Stefan,
is it possible that there is a mistake in the documentation or a bug? Out of
curiosity, I restarted one of these OSDs and the memory usage starts going up:
ceph 881203 15.4 4.0 6201580 5344764 ? Sl 09:18 6:38
/usr/bin/ceph-osd --cluster ceph -f -i 243 --setuser ceph -
Quoting Frank Schilder (fr...@dtu.dk):
> Dear Stefan,
>
> is it possible that there is a mistake in the documentation or a bug? Out of
> curiosity, I restarted one of these OSDs and the memory usage starts going up:
>
> ceph 881203 15.4 4.0 6201580 5344764 ? Sl 09:18 6:38
> /usr/b
Hello,
I'm a beginner on ceph. I set up three ceph clusters on google cloud.
Cluster1 has three nodes and each node has three disks. Cluster2 has three
nodes and each node has two disks. Cluster3 has five nodes and each node
has five disks.
All disks are HDD. Disk speed shown by `dd if=/dev/zero o
Hi Sage,
Just read the news on Cancellation of Cephlacon 2020, although the site is
still status quo/ Double checking that we can proceed with the
cancellation of logistics for South Korea
Thanks
Romit
On Tue, Feb 4, 2020 at 11:02 PM wrote:
> Send ceph-users mailing list submissions to
>
min_alloc_size can't be changed after formatting an OSD, and yes,
bluestore defers all writes that are < min_alloc_size. And default
min_alloc_size_ssd is 16KB.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-user
Hi Vitality,
I completely destroyed the test cluster and re-deployed it after changing
these settings but it did not make a difference - there are still a high
number of deferred writes.
Regards
--
Brad.
On Wed, 5 Feb 2020 at 10:55, wrote:
> min_alloc_size can't be changed after formatting an
Hi Boris,
not sure I understand why do you think it needs correction.
IMO _use_rotational_settings() function does what you're suggesting plus
it has some additional logic to be able to enforce specific device type.
The latter is intended primarily for testing/debugging and enabled via
'blue
I have found that if I set norecovery then I can get almost all OSDs to come up
and stay up, but the moment I unset norecovery so that the cluster can heal
itself, tons of OSDs go offline again.
The OSD host servers have plenty of available RAM, they are not maxing on CPU
or I/O near as I can t
Hi,
This helped to disable deferred writes in my case:
bluestore_min_alloc_size=4096
bluestore_prefer_deferred_size=0
bluestore_prefer_deferred_size_ssd=0
If you already deployed your OSDs with min_alloc_size=4K then you don't
need to redeploy them again.
Hi Vitality,
I completely destroye
[Moving this to ceph-users@ceph.io]
This looks like https://tracker.ceph.com/issues/43365, which *looks* like
it is an issue with the standard libraries in ubuntu 18.04. One user
said: "After upgrading our monitor Ubuntu 18.04 packages (apt-get upgrade)
with the 5.3.0-26-generic kernel, it see
Hi,
Do you have any suggestions on where I can look?
Regards,
Gesiel
Em dom., 2 de fev. de 2020 às 00:37, Gesiel Galvão Bernardes <
gesiel.bernar...@gmail.com> escreveu:
> Hi,
>
> Just now was possible continue this. Below is the information required.
> Thanks advance,
>
> Gesiel
>
> Em seg., 2
Cluster upgraded from 12.2.12 to 14.2.5. All went smooth, except BlueFS
spillover warning.
We create OSDs with ceph-deploy, command goes like this:
ceph-deploy osd create --bluestore --data /dev/sdf --block-db /dev/sdb5
--block-wal /dev/sdb6 ceph-osd3
where block-db and block-wal are SSD partitions
Hi Vladimir,
there were a plenty of discussions/recommendations around db volume size
selection here.
In short it's advised to have DB volume of 30 - 64GB for most of use cases.
Thanks,
Igor
On 2/5/2020 4:21 PM, Vladimir Prokofev wrote:
Cluster upgraded from 12.2.12 to 14.2.5. All went smo
Den ons 5 feb. 2020 kl 11:14 skrev quexian da :
> Hello,
>
> I'm a beginner on ceph. I set up three ceph clusters on google cloud.
> Cluster1 has three nodes and each node has three disks. Cluster2 has three
> nodes and each node has two disks. Cluster3 has five nodes and each node
> has five disk
Thanks for your valuable answer!
Is the write cache specific to ceph? Could you please provide some links to
the documentation about the write cache? Thanks!
Do you have any idea about the slow oss speed? Is it normal that the write
performance of object gateway is slower than that of rados clust
Den ons 5 feb. 2020 kl 16:19 skrev quexian da :
> Thanks for your valuable answer!
> Is the write cache specific to ceph? Could you please provide some links
> to the documentation about the write cache? Thanks!
>
>
It is all the possible caches used by ceph, by the device driver, the
filesystem (
Hi,
Our large Luminous cluster still has around 2k FileStore OSDs (35% of OSDs). We
haven't had any particular need to move these over to BlueStore yet, as the
performance is fine for our use case. Obviously, it would be easiest if we
could let the FileStore OSDs stay in the cluster until the h
Hi Vladimir,
If you're using the default options for rocksdb, then the size of L3 will be
25GB. Since your block-db is only 20GB and L3 can only be filled if the entire
level's size is available, bluefs will begin spillover. Like Igor said, having
30GB+ is recommended if you want to host up to
Hi Chad,
In case it's relevant we are on Nautilus 14.2.6, not Mimic.
I've followed Paul's advice and issued a "ceph osd down XXX" command for
the primary osd in each affected pg. I've also tried doing a systemctl
restart for several of the primary osd's, again with no apparent effect.
Unfortunate
Thanks for your valuable answer about write cache!
For object gateway, the performance is got by `swift-bench -t 64` which
uses 64 threads concurrently. Will the radosgw and http overhead be so
significant (94.5MB/s to 26MB/s for cluster1) when multiple threads are
used? Thanks in advance!
On We
Thank you for the insight.
> If you're using the default options for rocksdb, then the size of L3 will
be 25GB
Where this number comes from? Any documentation I can read?
I want to have a better understanding on how DB size is calculated.
ср, 5 февр. 2020 г. в 18:53, Moreno, Orlando :
> Hi Vladim
I didn't have such drop in performance testing 'rados bench 360 write -p
rbd' on 3x replicated (slow)hdd pool. Sort of near the average,
sometimes drops to 90. But I guess the test hits than an osd that is
scrubbing and being used by other processes.
-Original Message-
Sent: 05 Febr
Hi Jake,
In case it's relevant we are on Nautilus 14.2.6, not Mimic.
Yeah, my guess is that it is multiversion. Also, my scenario simply
should not have lost any data, so don't kick yourself too hard.
This command supposedly check cephfs for damaged files. It came back
with nothing for me
What do you guys Think about
Ceph osd Set noout/Down
And See if the osd will become healthy?
Another idea which is in my mind is to remove the sayed osd from the Cluster...
As long as the other osds on the same node dont have an issue i guess the disk
has a Problem...
Just my 2 Cents
- Mehmet
On 2/5/20 2:21 PM, Vladimir Prokofev wrote:
> Cluster upgraded from 12.2.12 to 14.2.5. All went smooth, except BlueFS
> spillover warning.
> We create OSDs with ceph-deploy, command goes like this:
> ceph-deploy osd create --bluestore --data /dev/sdf --block-db /dev/sdb5
> --block-wal /dev/sdb6
Hi,
because load to my OSD's stopped approximately after 12 hours and everything
returned to normal, I did another test.
I have chosen another rbd image and some old snapshot and I tried to mount it -
now without cloning it.
rbd-nbd map succeeded, but situation from my first test repeated, on
On Mon, 3 Feb 2020, Paul Emmerich wrote:
On Sun, Feb 2, 2020 at 9:35 PM Håkan T Johansson wrote:
Changing cp (or whatever standard tool is used) to call fsync() before
each close() is not an option for a user. Also, doing that would lead to
terrible performance generally. Just tested
Quoting vita...@yourcmc.ru (vita...@yourcmc.ru):
> SSD (block.db) partition contains object metadata in RocksDB so it probably
> loads the metadata before modifying objects (if it's not in cache yet). Also
> it sometimes performs compaction which also results in disk reads and
> writes. There are o
Thanks Vitaliy
Posting here for the archives or if anyone else sees the same problem it
might save them some work.
After going through the code and logs (debug bluestore 20/5) it actually
looks like the write-small-pre-read counter increases every time the WAL
gets appended to (it reads the previ
Hi, everyone.
The problem was solved.
A PG's epoch of active osds was dfferent from one of acting osds.
When I removed head and TEMP directory of the PG from active osds, blocked
request vanished.
Thanks to your kindness.
2020年2月6日(木) 4:19 :
> What do you guys Think about
>
> Ceph osd Set noo
Den ons 5 feb. 2020 kl 17:27 skrev Vladimir Prokofev :
> Thank you for the insight.
> > If you're using the default options for rocksdb, then the size of L3 will
> be 25GB
> Where this number comes from? Any documentation I can read?
> I want to have a better understanding on how DB size is calcul
32 matches
Mail list logo
