> Hello , Hello and thank you for the reply.Bellow are the answers to your questions. > > Let me ask some questions: > 1. What is the scheduler for your PV ?
On the Raid Controller device where the SSD disks are in Raid 0 (device sda) it is set to "deadline". But on the lvm volume logical volume dm-7, where the logical block is set for "data" volunr it is set to none.(i think this is ok ) [root@host1 ~]# ls -al /dev/mapper/gluster_vg_sd v_data ter_l lrwxrwxrwx. 1 root root 7 Jun 28 14:14 /dev/mapper/gluster_v g_sda3-gluster_lv_data -> ../dm-7 [root@host1 ~]# cat /sys/block/dm-7/queue/scheduler none root@host1:~[root@host1 ~]# cat /sys/block/dm-7/queue/schedu [root@host1 ~]# cat /sys/block/sda/queue/scheduler noop [deadline] cfq > 2. Have you aligned your PV during the setup 'pvcreate --dataalignment > alignment_value > device' I did not made other alignment then the default.Bellow are the partitions on /dev/sda. Can i enable partition alignment now, if yes how ? sfdisk -d /dev/sda # partition table of /dev/sda unit: sectors /dev/sda1 : start= 2048, size= 487424, Id=83, bootable /dev/sda2 : start= 489472, size= 95731712, Id=8e /dev/sda3 : start= 96221184, size=3808675840, Id=83 /dev/sda4 : start= 0, size= 0, Id= 0 > 3. What is your tuned profile ? Do you use rhgs-random-io from > the ftp://ftp.redhat.com/redhat/linux/enterprise/7Server/en/RHS/SRPMS/red... > ? My tuned active profile is virtual-host Current active profile: virtual-host No i dont use any of the rhgs-random-io profiles > 4. What is the output of "xfs_info /path/to/your/gluster/brick" ? xfs_info /gluster_bricks/data meta-data=/dev/mapper/gluster_vg_sda3-gluster_lv_data isize= 512 agcount=32, agsize=6553600 blks = sectsz=512 attr=2, projid 32bit=1 = crc=1 finobt=0 spino des=0 data = bsize=4096 blocks=2097152 00, imaxpct=25 = sunit=64 swidth=64 blks naming =version 2 bsize=8192 ascii-ci=0 fty pe=1 log =internal bsize=4096 blocks=102400, version=2 = sectsz=512 sunit=64 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtex tents=0 > 5. Are you using Jumbo Frames ? Does your infra support them? > Usually MTU of 9k is standard, but some switches and NICs support up to 16k. > Unfortunately I can not enable MTU to 9000 and Jumbo Frames on these Cisco SG350X switches to specific ports.The switches dont suport Jumbo Frames enable to a single port, only on all ports . I have others devices connected to the switches on the remaining 48 ports that have 1Gb/s. > All the options for "optimize for virt...." are located > at /var/lib/glusterd/groups/virt on each gluster node. I have already looked previously at that file, but not all the volume settings that are set by "Optime for Virt Store" are stored there. For example "Optimize for Virt Store " sets network.remote.dio to disable and in the glusterd/groups/virt is set to enabled.Or cluster.granular-entry-heal: enable is not present there, bit it is set by "Optimize for Virt Store" > > Best Regards, > Strahil Nikolov > > > > > В неделя, 28 юни 2020 г., 22:13:09 Гринуич+3, jury cat > <shadow.emy1(a)gmail.com> > написа: > > > > > > Hello all, > > I am using Ovirt 4.3.10 on Centos 7.8 with glusterfs 6.9 . > My Gluster setup is of 3 hosts in replica 3 (2 hosts + 1 arbiter). > All the 3 hosts are Dell R720 with Perc Raid Controller H710 mini(that has > maximim > throughtout 6Gbs) and with 2×1TB samsumg SSD in RAID 0. The volume is > partitioned using > LVM thin provision and formated XFS. > The hosts have separate 10GE network cards for storage traffic. > The Gluster Network is connected to this 10GE network cards and is mounted > using Fuse > Glusterfs(NFS is disabled).Also Migration Network is activated on the same > storage > network. > > > The problem is that the 10GE network is not used at full potential by the > Gluster. > If i do live Migration of Vms i can see speeds of 7GB/s ~ 9GB/s. > The same network tests using iperf3 reported 9.9GB/s , these exluding the > network setup > as a bottleneck(i will not paste all the iperf3 tests here for now). > I did not enable all the Volume options from "Optimize for Virt Store", > because > of the bug that cant set volume cluster.granural-heal to enable(this was > fixed in vdsm-4 > 40, but that is working only on Centos 8 with ovirt 4.4 ) . > i whould be happy to know what are all these "Optimize for Virt Store" > options, > so i can set them manually. > > > The speed on the disk inside the host using dd is b etween 1GB/s to 700Mbs. > > > [root@host1 ~]# dd if=/dev/zero of=test bs=100M count=40 cou nt=80 > status=progress > 8074035200 bytes (8.1 GB) copied, 11.059372 s, 730 MB/s 80+0 records in 80+0 > records out > 8388608000 bytes (8.4 GB) copied, 11.9928 s, 699 MB/s > > > The dd write test on the gluster volme inside the host is poor only ~ > 120MB/s . > During the dd test, if i look at Networks->Gluster network ->Hosts at Tx and > Rx the > network speed barerly reaches over 1Gbs (~1073 Mbs) out of maximum of 10000 > Mbs. > > > dd if=/dev/zero of=/rhev/data-center/mnt/glu sterSD/gluster1.domain.local\: > _data/test > bs=100M count=80 status=progress 8283750400 bytes (8.3 GB) copied, 71.297942 > s, 116 MB/s > 80+0 records in 80+0 records out 8388608000 bytes (8.4 GB) copied, 71.9545 s, > 117 MB/s > > > I have attached my Gluster volume settings and mount options. > > Thanks, > Emy > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/7BR6TZQ4EXS... _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/R5XPLZ7QARJG2TYRTOK6BADJDNONPOPQ/
