Sorry ,the capture for targetcli seems can not display correctly.Paste it again.
o- / ......................................................................................................................... [...] o- backstores .............................................................................................................. [...] | o- block .................................................................................................. [Storage Objects: 1] | | o- rbd_iblock1 ..................................................... [/dev/rbd/rbd_pool/image2 (50.0TiB) write-thru activated] | | o- alua ................................................................................................... [ALUA Groups: 1] | | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized] | o- fileio ................................................................................................. [Storage Objects: 0] | o- pscsi .................................................................................................. [Storage Objects: 0] | o- ramdisk ................................................................................................ [Storage Objects: 0] | o- user:glfs .............................................................................................. [Storage Objects: 0] | o- user:rbd ............................................................................................... [Storage Objects: 1] | | o- my_replicated_test ..................................................... [rbd_pool/replicated_image1 (1000.0GiB) activated] | | o- alua ................................................................................................... [ALUA Groups: 1] | | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized] | o- user:zbc ............................................................................................... [Storage Objects: 0] o- iscsi ............................................................................................................ [Targets: 1] | o- iqn.2018-09.com.test:target1 ...................................................................................... [TPGs: 1] | o- tpg1 ..................................................................................... [gen-acls, tpg-auth, 1-way auth] | o- acls .......................................................................................................... [ACLs: 0] | o- luns .......................................................................................................... [LUNs: 2] | | o- lun0 ..................................................................... [user/my_replicated_test (default_tg_pt_gp)] | | o- lun1 ................................................ [block/rbd_iblock1 (/dev/rbd/rbd_pool/image2) (default_tg_pt_gp)] | o- portals .................................................................................................... [Portals: 1] | o- 10.0.1.111:3260 .................................................................................................. [OK] o- loopback ......................................................................................................... [Targets: 0] o- vhost ............................................................................................................ [Targets: 0] o- xen-pvscsi ....................................................................................................... [Targets: 0] 在 2018年9月6日星期四 UTC+8上午10:50:59，[email protected]写道： > > > Thank you very much, Mike! > > 在 2018年9月5日星期三 UTC+8上午5:16:21，Mike Christie写道： >> >> On 09/03/2018 02:31 AM, [email protected] wrote: >> > Hello Mike, >> > >> > I am very appreciate for your instruction. >> > Now I can set hw_max_sectors through targcli when i create the device. >> > I set it to 8192 same as raw rbd device. >> > The performance improve a little, 4M seq write increase from 24MB/s to >> > 40MB/s.(hw_max_sectors 64->8192, it is a /backstore/user:rbd/disk_xxx >> > device) >> > But it is far away from block device, if I usr /backstore/block 4M seq >> > write it will be 500MB/s, performance is still a big problem.(first I >> > should map a rbd device , then export it by targetcli) >> > >> > The performance difference between /backstore/user:rbd/ and >> > /backstore/block is so big, is it normal? >> >> backstore/user will be lower than block, but I do not think it would a >> difference like you are seeing. >> > ok, it is a good news for me. > It is still possible for me to improve my iscsi disk performance. > >> >> What lio fabric driver are you using? iSCSI? What kernel version and >> what version of tcmu-runner? >> > io fabric driver : iscsi > > iscsid version: 2.0-873 > > OS version: CentOS Linux release 7.5.1804 (Core) > > kernel version: 3.10.0-862.el7.x86_64 > > tcmu-runner version: 1.4.0-rc1 > > > Target Build: > > targetcli /iscsi create iqn.2018-09.com.test:target1 > > targetcli /backstores/user:rbd create name=my_replicated_test size=1000G > cfgstring=rbd_pool/replicated_image1 hw_max_sectors=8192 > > targetcli /iscsi/iqn.2018-09.com.test:target1/tpg1/luns create > /backstores/user:rbd/my_replicated_test > > targetcli /iscsi/iqn.2018-09.com.test:target1/tpg1/portals create > 10.0.1.111 > > targetcli /iscsi/iqn.2018-09.com.test:target1/tpg1 set auth userid=****** > password=****** > > targetcli /iscsi/iqn.2018-09.com.test:target1/tpg1 set attribute > authentication=1 demo_mode_write_protect=0 generate_node_acls=1 > > > Target Setting: > > >> [image: 屏幕快照 2018-09-06 上午10.43.11.png] >> >> > > > > > > > > > >> > Could you give me some suggestions about improving the performance of >> > /backstore/user:rbd/ device. >> > >> > Thanks very much! >> > >> > 在 2018年9月3日星期一 UTC+8上午10:31:48，[email protected]写道： >> > >> > Hello Mike, >> > >> > Thank you for your informative response. >> > >> > >> > 在 2018年8月28日星期二 UTC+8上午8:49:46，Mike Christie写道： >> > >> > On 08/21/2018 08:52 PM, [email protected] wrote: >> > > Hi folks, >> > > >> > > I am newbie to open-iscsi. >> > > My case is I export ceph rbd by open-iscsi. >> > > >> > > I found the max_sectors_kb is 64, the value is so small, and >> > 4M sequence >> > > write is only about 10MB/s. >> > > I can not increase max_sectors_kb, if I do it will return >> > "bash: echo: >> > > write error: Invalid argument"(But I can change the value to >> a >> > small one >> > > < 64, max_hw_sectors_kb is 32767) >> > > >> > >> > In new version of the linux kernel the initiator will use the >> > optimal >> > value reported by the target and then uses the max reported as >> > the limit >> > that the user can set. It sounds like you are using rbd/ceph >> with >> > tcmu-runner which has a default limits of 64K. >> > >> > Yes, I am using tcmu-runner and gwcli. >> > >> > >> > If you are using targetcli/lio directly then you can set >> > hw_max_sectors >> > through targcli when you create the device or in the >> > saveconfig.json file. >> > >> > If you are using the ceph-iscsi tools then I am actually >> working >> > on a >> > gwcli command to configure this right now. >> > >> > Yes, I am using ceph-iscsi tools.How can I change the limit by >> > ceph-iscsi tools? >> > >> > Thanks. >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "open-iscsi" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an email to [email protected] >> > <mailto:[email protected]>. >> > To post to this group, send email to [email protected] >> > <mailto:[email protected]>. >> > Visit this group at https://groups.google.com/group/open-iscsi. >> > For more options, visit https://groups.google.com/d/optout. >> >> -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout.
