Seeing some performance issues on my ceph cluster with *qemu-img
convert *directly
writing to ceph against normal rbd import command.
*Direct data copy (without qemu-img convert) took 5 hours 43 minutes for
465GB data.*
[root@cephlarge vm_res_id_24291e4b-93d2-47ad-80a8-bf3c395319b9_vdb]# time
rbd import 66582225-6539-4e5e-9b7a-59aa16739df1 -p volumes
66582225-6539-4e5e-9b7a-59aa16739df1_directCopy --image-format 2
rbd: --pool is deprecated for import, use --dest-pool
Importing image: 100% complete...done.
real *343m38.028s*
user 4m40.779s
sys 7m18.916s
[root@cephlarge vm_res_id_24291e4b-93d2-47ad-80a8-bf3c395319b9_vdb]# rbd
info volumes/66582225-6539-4e5e-9b7a-59aa16739df1_directCopy
rbd image '66582225-6539-4e5e-9b7a-59aa16739df1_directCopy':
size 465 GB in 119081 objects
order 22 (4096 kB objects)
block_name_prefix: rbd_data.373174b0dc51
format: 2
features: layering, exclusive-lock, object-map, fast-diff,
deep-flatten
flags:
[root@cephlarge vm_res_id_24291e4b-93d2-47ad-80a8-bf3c395319b9_vdb]#
*Qemu-img convert is still in progress and completed merely 10% in more
than 40 hours. (for 465GB data)*
[root@cephlarge mnt]# time qemu-img convert -p -t none -O raw
/mnt/data/workload_326e8a43-a90a-4fe9-8aab-6d33bcdf5a05/
snapshot_9f0cee13-8200-4562-82ec-1fb9f234bcd8/vm_id_05e9534e-5c84-4487-9613-
1e0e227e4c1a/vm_res_id_24291e4b-93d2-47ad-80a8-bf3c395319b9_vdb/66582225-6539-4e5e-9b7a-59aa16739df1
rbd:volumes/24291e4b-93d2-47ad-80a8-bf3c395319b9
(0.00/100%)
(10.00/100%)
*Rbd bench-write shows speed of ~21MB/s.*
[root@cephlarge ~]# rbd bench-write image01 --pool=rbdbench
bench-write io_size 4096 io_threads 16 bytes 1073741824 pattern sequential
SEC OPS OPS/SEC BYTES/SEC
2 6780 3133.53 12834946.35
3 6831 1920.65 7866998.17
4 8896 2040.50 8357871.83
5 13058 2562.61 10496432.34
6 17225 2836.78 11619432.99
7 20345 2736.84 11210076.25
8 23534 3761.57 15407392.94
9 25689 3601.35 14751109.98
10 29670 3391.53 13891695.57
11 33169 3218.29 13182107.64
12 36356 3135.34 12842344.21
13 38431 2972.62 12175863.99
14 47780 4389.77 17980497.11
15 55452 5156.40 21120627.26
16 59298 4772.32 19547440.33
17 61437 5151.20 21099315.94
18 67702 5861.64 24009295.97
19 77086 5895.03 24146032.34
20 85474 5936.09 24314243.88
21 93848 7499.73 30718898.25
22 100115 7783.39 31880760.34
23 105405 7524.76 30821410.70
24 111677 6797.12 27841003.78
25 116971 6274.51 25700386.48
26 121156 5468.77 22400087.81
27 126484 5345.83 21896515.02
28 137937 6412.41 26265239.30
29 143229 6347.28 25998461.13
30 149505 6548.76 26823729.97
31 159978 7815.37 32011752.09
32 171431 8821.65 36133479.15
33 181084 8795.28 36025472.27
35 182856 6322.41 25896605.75
36 186891 5592.25 22905872.73
37 190906 4876.30 19973339.07
38 190943 3076.87 12602853.89
39 190974 1536.79 6294701.64
40 195323 2344.75 9604081.07
41 198479 2703.00 11071492.89
42 208893 3918.55 16050365.70
43 214172 4702.42 19261091.89
44 215263 5167.53 21166212.98
45 219435 5392.57 22087961.94
46 225731 5242.85 21474728.85
47 234101 5009.43 20518607.70
48 243529 6326.00 25911280.08
49 254058 7944.90 32542315.10
elapsed: 50 ops: 262144 ops/sec: 5215.19 bytes/sec: 21361431.86
[root@cephlarge ~]#
This CEPH deployment has 2 OSDs.
It would be of great help if anyone can give me pointers.
--
Regards,
mahesh j
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
