Most likely you need to apply a patch to the kernel. http://www.theirek.com/blog/2014/02/16/patch-dlia-raboty-s-enierghoniezavisimym-keshiem-ssd-diskov
2014-04-28 15:20 GMT+04:00 Indra Pramana <in...@sg.or.id>: > Hi Udo and Irek, > > Good day to you, and thank you for your emails. > > > >perhaps due IOs from the journal? > >You can test with iostat (like "iostat -dm 5 sdg"). > > Yes, I have shared the iostat result earlier on this same thread. At times > the utilisation of the 2 journal drives will hit 100%, especially when I > simulate writing data using rados bench command. Any suggestions what could > be the cause of the I/O issue? > > > ==== > avg-cpu: %user %nice %system %iowait %steal %idle > 1.85 0.00 1.65 3.14 0.00 93.36 > > > Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz > avgqu-sz await r_await w_await svctm %util > sdg 0.00 0.00 0.00 55.00 0.00 25365.33 > 922.38 34.22 568.90 0.00 568.90 17.82 98.00 > sdf 0.00 0.00 0.00 55.67 0.00 25022.67 > 899.02 29.76 500.57 0.00 500.57 17.60 98.00 > > > avg-cpu: %user %nice %system %iowait %steal %idle > 2.10 0.00 1.37 2.07 0.00 94.46 > > > Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz > avgqu-sz await r_await w_await svctm %util > sdg 0.00 0.00 0.00 56.67 0.00 25220.00 > 890.12 23.60 412.14 0.00 412.14 17.62 99.87 > sdf 0.00 0.00 0.00 52.00 0.00 24637.33 > 947.59 33.65 587.41 0.00 587.41 19.23 100.00 > > > avg-cpu: %user %nice %system %iowait %steal %idle > 2.21 0.00 1.77 6.75 0.00 89.27 > > > Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz > avgqu-sz await r_await w_await svctm %util > sdg 0.00 0.00 0.00 54.33 0.00 24802.67 > 912.98 25.75 486.36 0.00 486.36 18.40 100.00 > sdf 0.00 0.00 0.00 53.00 0.00 24716.00 > 932.68 35.26 669.89 0.00 669.89 18.87 100.00 > > > avg-cpu: %user %nice %system %iowait %steal %idle > 1.87 0.00 1.67 5.25 0.00 91.21 > > > Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz > avgqu-sz await r_await w_await svctm %util > sdg 0.00 0.00 0.00 94.33 0.00 26257.33 > 556.69 18.29 208.44 0.00 208.44 10.50 99.07 > sdf 0.00 0.00 0.00 51.33 0.00 24470.67 > 953.40 32.75 684.62 0.00 684.62 19.51 100.13 > > > avg-cpu: %user %nice %system %iowait %steal %idle > 1.51 0.00 1.34 7.25 0.00 89.89 > > > Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz > avgqu-sz await r_await w_await svctm %util > sdg 0.00 0.00 0.00 52.00 0.00 22565.33 > 867.90 24.73 446.51 0.00 446.51 19.10 99.33 > sdf 0.00 0.00 0.00 64.67 0.00 24892.00 > 769.86 19.50 330.02 0.00 330.02 15.32 99.07 > ==== > > >You what model SSD? > > For this one, I am using Seagate 100GB SSD, model: HDS-2TM-ST100FM0012 > > >Which version of the kernel? > > Ubuntu 13.04, Linux kernel version: 3.8.0-19-generic #30-Ubuntu SMP Wed > May 1 16:35:23 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux > > Looking forward to your reply, thank you. > > Cheers. > > > > On Mon, Apr 28, 2014 at 4:45 PM, Irek Fasikhov <malm...@gmail.com> wrote: > >> You what model SSD? >> Which version of the kernel? >> >> >> >> 2014-04-28 12:35 GMT+04:00 Udo Lembke <ulem...@polarzone.de>: >> >>> Hi, >>> perhaps due IOs from the journal? >>> You can test with iostat (like "iostat -dm 5 sdg"). >>> >>> on debian iostat is in the package sysstat. >>> >>> Udo >>> >>> Am 28.04.2014 07:38, schrieb Indra Pramana: >>> > Hi Craig, >>> > >>> > Good day to you, and thank you for your enquiry. >>> > >>> > As per your suggestion, I have created a 3rd partition on the SSDs and >>> did >>> > the dd test directly into the device, and the result is very slow. >>> > >>> > ==== >>> > root@ceph-osd-08:/mnt# dd bs=1M count=128 if=/dev/zero of=/dev/sdg3 >>> > conv=fdatasync oflag=direct >>> > 128+0 records in >>> > 128+0 records out >>> > 134217728 bytes (134 MB) copied, 19.5223 s, 6.9 MB/s >>> > >>> > root@ceph-osd-08:/mnt# dd bs=1M count=128 if=/dev/zero of=/dev/sdf3 >>> > conv=fdatasync oflag=direct >>> > 128+0 records in >>> > 128+0 records out >>> > 134217728 bytes (134 MB) copied, 5.34405 s, 25.1 MB/s >>> > ==== >>> > >>> > I did a test onto another server with exactly similar specification and >>> > similar SSD drive (Seagate SSD 100 GB) but not added into the cluster >>> yet >>> > (thus no load), and the result is fast: >>> > >>> > ==== >>> > root@ceph-osd-09:/home/indra# dd bs=1M count=128 if=/dev/zero >>> of=/dev/sdf1 >>> > conv=fdatasync oflag=direct >>> > 128+0 records in >>> > 128+0 records out >>> > 134217728 bytes (134 MB) copied, 0.742077 s, 181 MB/s >>> > ==== >>> > >>> > Is the Ceph journal load really takes up a lot of the SSD resources? I >>> > don't understand how come the performance can drop significantly. >>> > Especially since the two Ceph journals are only taking the first 20 GB >>> out >>> > of the 100 GB of the SSD total capacity. >>> > >>> > Any advice is greatly appreciated. >>> > >>> > Looking forward to your reply, thank you. >>> > >>> > Cheers. >>> > >>> > >>> > >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@lists.ceph.com >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >> >> >> >> -- >> С уважением, Фасихов Ирек Нургаязович >> Моб.: +79229045757 >> > > -- С уважением, Фасихов Ирек Нургаязович Моб.: +79229045757
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
