Try comparing results from something like this test
[global]
ioengine=posixaio
invalidate=1
ramp_time=30
iodepth=1
runtime=180
time_based
direct=1
filename=/mnt/cephfs/ssd/fio-bench.img
[write-4k-seq]
stonewall
bs=4k
rw=write
#write_bw_log=sdx-4k-write-seq.results
#write_iops_log=sdx-4k-write
Thanks Mokhtar! This is what I am looking for, thanks for your explanation!
Best Regards,
Dave Chen
From: Maged Mokhtar
Sent: Wednesday, November 14, 2018 3:36 PM
To: Chen2, Dave; ceph-users@lists.ceph.com
Subject: Re: [ceph-users] Benchmark performance when using SSD as the journal
[EXTERNAL
Hi Roos,
I will try with the configuration, thank you very much!
Best Regards,
Dave Chen
-Original Message-
From: Marc Roos
Sent: Wednesday, November 14, 2018 4:37 PM
To: ceph-users; Chen2, Dave
Subject: RE: [ceph-users] Benchmark performance when using SSD as the journal
[EXTERNAL E
Hi Dave,
The main line in SSD specs you should look at is
Enhanced Power Loss Data Protection: Yes
This makes SSD cache nonvolatile and makes SSD ignore fsync()s so
transactional performance becomes equal to non-transactional. So your
SSDs should be OK for journal.
rados bench is a bad to
Hi,
I copied my custom module in /usr/lib64/ceph/mgr and run "ceph mgr module
enable --force" to enable plugin. It's plug and print some
message in plugin but it's not print any log in ceph-mgr log file.
Thanks,
Amit G
___
ceph-users mailing list
ceph-
On Wed, Nov 14, 2018 at 5:11 PM Amit Ghadge wrote:
> Hi,
> I copied my custom module in /usr/lib64/ceph/mgr and run "ceph mgr module
> enable --force" to enable plugin. It's plug and print some
> message in plugin but it's not print any log in ceph-mgr log file.
>
>
> Thanks,
> Amit G
>
Yes, it
Hi,
We currently deploy our filestore OSDs with ceph-disk (via
ceph-ansible), and I was looking at using ceph-volume as we migrate to
bluestore.
Our servers have 60 OSDs and 2 NVME cards; each OSD is made up of a
single hdd, and an NVME partition for journal.
If, however, I do:
ceph-volume lvm b
Hi,
I'm in the middle of expanding a Ceph cluster and while having 'ceph -s'
open I suddenly saw a bunch of Placement Groups go undersized.
My first hint was that one or more OSDs have failed, but none did.
So I checked and I saw these Placement Groups undersized:
11.3b54 active+undersized+degr
On Wed, Nov 14, 2018 at 9:10 AM Matthew Vernon wrote:
>
> Hi,
>
> We currently deploy our filestore OSDs with ceph-disk (via
> ceph-ansible), and I was looking at using ceph-volume as we migrate to
> bluestore.
>
> Our servers have 60 OSDs and 2 NVME cards; each OSD is made up of a
> single hdd, a
Hello everyone,
we encountered an error with the Prometheus plugin for Ceph mgr:
One osd was down and (therefore) it had no class:
```
sudo ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
28 hdd 7.27539 osd.28 up 1.0 1.0
6
Hi Eric,
Please take a look at the new Foundation site's FAQ for answers to
these questions:
https://ceph.com/foundation/
On Tue, Nov 13, 2018 at 11:51 AM Smith, Eric wrote:
>
> https://techcrunch.com/2018/11/12/the-ceph-storage-project-gets-a-dedicated-open-source-foundation/
>
>
>
> What does
So, I’ve issued the deep-scrub command (and the repair command)
and nothing seems to happen.
Unrelated to this issue, I have to take down some OSD to prepare
a host for RMA. One of them happens to be in the replication
group for this PG. So, a scrub happened indirectly. I now have
this from “ceph -
Hello
I have a ceph 13.2.2 cluster comprised of 5 hosts, each with 16 HDDs and
4 SSDs. HDD OSDs have about 50 PGs each, while SSD OSDs have about 400
PGs each (a lot more pools use SSDs than HDDs). Servers are fairly
powerful: 48 HT cores, 192GB of RAM, and 2x25Gbps Ethernet.
The impression
On Wed, Nov 14, 2018 at 3:32 PM Gökhan Kocak
wrote:
>
> Hello everyone,
>
> we encountered an error with the Prometheus plugin for Ceph mgr:
> One osd was down and (therefore) it had no class:
> ```
> sudo ceph osd tree
> ID CLASS WEIGHTTYPE NAME STATUS REWEIGHT PRI-AFF
> 28 hdd
Hi Dave
Have you looked at the Intel P4600 vsd the P4500
The P4600 has better random writes and a better drive writes per day I
believe
Thanks Joe
>>> 11/13/2018 8:45 PM >>>
Thanks Merrick!
I checked with Intel spec [1], the performance Intel said is,
· Sequential Read (up to) 500 MB
This may be less of an issue now - the most traumatic experience for us,
back around hammer, memory usage under recovery+load ended up with OOM kill
of osds, needing more recovery, a pretty vicious cycle.
-KJ
On Wed, Nov 14, 2018 at 11:45 AM Vladimir Brik <
vladimir.b...@icecube.wisc.edu> wrote:
You could try a 'rados get' and then a 'rados put' on the object to start with.
On Thu, Nov 15, 2018 at 4:07 AM K.C. Wong wrote:
>
> So, I’ve issued the deep-scrub command (and the repair command)
> and nothing seems to happen.
> Unrelated to this issue, I have to take down some OSD to prepare
> a
On 11/14/18 1:45 PM, Vladimir Brik wrote:
Hello
I have a ceph 13.2.2 cluster comprised of 5 hosts, each with 16 HDDs
and 4 SSDs. HDD OSDs have about 50 PGs each, while SSD OSDs have about
400 PGs each (a lot more pools use SSDs than HDDs). Servers are fairly
powerful: 48 HT cores, 192GB of R
Thanks Konstantin, I already tried accessing it in different ways and best
I got is bulk renamed files and other non presentable data.
Maybe to solve this I can create overlapping osd pools?
Like one pool includes all 3 osd for replication, and 3 more include one
osd at each site with same blocks?
Hi cephers,
All our cluster osds are deployed in armhf.
Could someone say something about what is the rational performance rates for
librbd VS KRBD ?
Or rational performance loss range when we use librbd compare to KRBD.
I googled a lot, but I could not find a solid criterion.
In fact , it con
On 11/15/18 9:31 AM, Vlad Kopylov wrote:
Thanks Konstantin, I already tried accessing it in different ways and
best I got is bulk renamed files and other non presentable data.
Maybe to solve this I can create overlapping osd pools?
Like one pool includes all 3 osd for replication, and 3 more in
This is weird. Can you capture the pg query for one of them and narrow down
in which epoch it “lost” the previous replica and see if there’s any
evidence of why?
On Wed, Nov 14, 2018 at 8:09 PM Wido den Hollander wrote:
> Hi,
>
> I'm in the middle of expanding a Ceph cluster and while having 'cep
You'll need to provide more data about how your test is configured and run
for us to have a good idea. IIRC librbd is often faster than krbd because
it can support newer features and things, but krbd may have less overhead
and is not dependent on the VM's driver configuration in QEMU...
On Thu, No
On Mon, Nov 12, 2018 at 2:46 PM Hector Martin wrote:
> On 10/11/2018 06:35, Gregory Farnum wrote:
> > Yes, do that, don't try and back up your monitor. If you restore a
> > monitor from backup then the monitor — your authoritative data source —
> > will warp back in time on what the OSD peering i
Thank you for your attention.
Our test are in run in physical machine environments.
Fio for KRBD:
[seq-write]
description="seq-write"
direct=1
ioengine=libaio
filename=/dev/rbd0
numjobs=1
iodepth=256
group_reporting
rw=write
bs=4M
size=10T
runtime=180
*/dev/rbd0 mapped by rbd_pool/image2, so KRB
Attempting to send 256 concurrent 4MiB writes via librbd will pretty
quickly hit the default "objecter_inflight_op_bytes = 100 MiB" limit,
which will drastically slow (stall) librados. I would recommend
re-testing librbd w/ a much higher throttle override.
On Thu, Nov 15, 2018 at 11:34 AM 赵赵贺东 wro
Thanks you for your suggestion.
It really give me a lot of inspirations.
I will test as your suggestion, and browse through src/common/config_opts.h to
see if I can find some configs performance related.
But, our osd nodes hardware itself is very poor, that is the truth…we have to
face it.
Two
True, sorry and many thanks!
Gökhan
On 14.11.18 21:03, John Spray wrote:
> On Wed, Nov 14, 2018 at 3:32 PM Gökhan Kocak
> wrote:
>> Hello everyone,
>>
>> we encountered an error with the Prometheus plugin for Ceph mgr:
>> One osd was down and (therefore) it had no class:
>> ```
>> sudo ceph osd
28 matches
Mail list logo
