thanks nick,
filestore-> journal_latency: ~1.1ms
214.0/180611
0.0011848669239415096
seems ssd write is ok, any other idea is highly appreciated!
"filestore": {
"journal_queue_max_ops": 300,
"journal_queue_ops": 0,
"journal_ops": 180611,
"journal_queue_max_bytes": 33554432,
"journal_queue_bytes": 0,
"journal_bytes": 32637888155,
"journal_latency": {
"avgcount": 180611,
"sum": 214.095788552
},
"journal_wr": 176801,
"journal_wr_bytes": {
"avgcount": 176801,
"sum": 33122885632
},
"journal_full": 0,
"committing": 0,
"commitcycle": 14648,
"commitcycle_interval": {
"avgcount": 14648,
"sum": 73299.187956076
},
2016-02-12 18:04 GMT+08:00 Nick Fisk <n...@fisk.me.uk>:
>
>
> > -----Original Message-----
> > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
> > Huan Zhang
> > Sent: 12 February 2016 10:00
> > To: Irek Fasikhov <malm...@gmail.com>
> > Cc: ceph-users <ceph-us...@ceph.com>
> > Subject: Re: [ceph-users] ceph 9.2.0 SAMSUNG ssd performance issue?
> >
> > "op_w_latency":
> > "avgcount": 42991,
> > "sum": 402.804741329
> >
> > 402.0/42991
> > 0.009350794352306296
> >
> > ~9ms latency, that means this ssd not suitable for journal device?
>
> I believe that counter includes lots of other operations in the OSD
> including the journal write. If you want pure journal stats, I would under
> the Filestore->journal_latency counter
>
> >
> >
> > "osd": {
> > "op_wip": 0,
> > "op": 58683,
> > "op_in_bytes": 7309042294,
> > "op_out_bytes": 507137488,
> > "op_latency": {
> > "avgcount": 58683,
> > "sum": 484.302231121
> > },
> > "op_process_latency": {
> > "avgcount": 58683,
> > "sum": 323.332046552
> > },
> > "op_r": 902,
> > "op_r_out_bytes": 507137488,
> > "op_r_latency": {
> > "avgcount": 902,
> > "sum": 0.793759596
> > },
> > "op_r_process_latency": {
> > "avgcount": 902,
> > "sum": 0.619918138
> > },
> > "op_w": 42991,
> > "op_w_in_bytes": 7092142080,
> > "op_w_rlat": {
> > "avgcount": 38738,
> > "sum": 334.643717526
> > },
> > "op_w_latency": {
> > "avgcount": 42991,
> > "sum": 402.804741329
> > },
> > "op_w_process_latency": {
> > "avgcount": 42991,
> > "sum": 260.489972416
> > },
> > ...
> >
> >
> > 2016-02-12 15:56 GMT+08:00 Irek Fasikhov <malm...@gmail.com>:
> > Hi.
> > You need to read : https://www.sebastien-han.fr/blog/2014/10/10/ceph-
> > how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
> >
> >
> > С уважением, Фасихов Ирек Нургаязович
> > Моб.: +79229045757
> >
> > 2016-02-12 10:41 GMT+03:00 Huan Zhang <huan.zhang...@gmail.com>:
> > Hi,
> >
> > ceph VERY SLOW with 24 osd(SAMSUNG ssd).
> > fio /dev/rbd0 iodepth=1 direct=1 IOPS only ~200
> > fio /dev/rbd0 iodepth=32 direct=1 IOPS only ~3000
> >
> > But test single ssd deive with fio:
> > fio iodepth=1 direct=1 IOPS ~15000
> > fio iodepth=32 direct=1 IOPS ~30000
> >
> > Why ceph SO SLOW? Could you give me some help?
> > Appreciated!
> >
> >
> > My Enviroment:
> > [root@szcrh-controller ~]# ceph -s
> > cluster eb26a8b9-e937-4e56-a273-7166ffaa832e
> > health HEALTH_WARN
> > 1 mons down, quorum 0,1,2,3,4
> ceph01,ceph02,ceph03,ceph04,ceph05
> > monmap e1: 6 mons at {ceph01=
> > 10.10.204.144:6789/0,ceph02=10.10.204.145:6789/0,ceph03=10.10.204.146:67
> > 89/0,ceph04=10.10.204.147:6789/0,ceph05=10.10.204.148:6789/0,ceph06=0.0
> > .0.0:0/5
> > }
> > election epoch 6, quorum 0,1,2,3,4
> > ceph01,ceph02,ceph03,ceph04,ceph05
> > osdmap e114: 24 osds: 24 up, 24 in
> > flags sortbitwise
> > pgmap v2213: 1864 pgs, 3 pools, 49181 MB data, 4485 objects
> > 144 GB used, 42638 GB / 42782 GB avail
> > 1864 active+clean
> >
> > [root@ceph03 ~]# lsscsi
> > [0:0:6:0] disk ATA SAMSUNG MZ7KM1T9 003Q /dev/sda
> > [0:0:7:0] disk ATA SAMSUNG MZ7KM1T9 003Q /dev/sdb
> > [0:0:8:0] disk ATA SAMSUNG MZ7KM1T9 003Q /dev/sdc
> > [0:0:9:0] disk ATA SAMSUNG MZ7KM1T9 003Q /dev/sdd
> >
> > _______________________________________________
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
>
>
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
