Hi,
I'm using Linstor on 5 node Proxmox 7.x cluster. After recent upgrade
I've got some performance issues with some linstor queries.
Upgrade: linstor-common:amd64 (1.20.2-1, 1.22.1-1), python-linstor:amd64
(1.16.0-1, 1.18.0-1), linstor-client:amd64 (1.16.0-1, 1.18.0-1),
linstor-controller:amd64 (1.20.2-1, 1.22.1-1), linstor-satellite:amd64
(1.20.2-1, 1.22.1-1), linstor-proxmox:amd64 (6.1.0-1, 7.0.0-1)
Some queries work fast:
# time linstor node list
+------------------------------------------------------------------------------+
| Node | NodeType| Addresse s | State |
|================================================|
| node1 | SATELLITE | xxx.xxx.xxx.xx1:3366 (PLAIN) | Online |
| node2 | SATELLITE | xxx.xxx.xxx.xx2:3366 (PLAIN) | Online |
| node3 | SATELLITE | xxx.xxx.xxx.xx3:3366 (PLAIN) | Online |
| node4 | SATELLITE | xxx.xxx.xxx.xx4:3366 (PLAIN) | Online |
| node5 | SATELLITE | xxx.xxx.xxx.xx5:3366 (PLAIN) | Online |
+------------------------------------------------------------------------------+
real 0m0.294s
user 0m0.259s
sys 0m0.033s
But some got very slow, up to a minute. I'm 100% sure it was a lot
faster before this upgrade:
# time linstor volume list
+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Node | Resource | StoragePool | VolNr |
MinorNr | DeviceName | Allocated | InUse | State |
|=================================================================================================|
| node3 | linstor_db | pool-drbd-mgmt | 0 |
1000 | /dev/drbd1000 | 136.80 MiB | InUse | UpToDate |
| node4 | linstor_db | pool-drbd-mgmt | 0 |
1000 | /dev/drbd1000 | 141.82 MiB | Unused | UpToDate |
| node5 | linstor_db | pool-drbd-mgmt | 0 |
1000 | /dev/drbd1000 | 36.43 MiB | Unused | UpToDate |
| node3 | vm-100-disk-2 | DfltDisklessStorPool | 0 | 1012 |
/dev/drbd1012 | | Unused | TieBreaker |
| node4 | vm-100-disk-2 | pool-drbd-4-5 | 0 |
1012 | /dev/drbd1012 | 29.90 GiB | InUse | UpToDate |
| node5 | vm-100-disk-2 | pool-drbd-4-5 | 0 |
1012 | /dev/drbd1012 | 29.90 GiB | Unused | UpToDate |
| node1 | vm-101-disk-3 | DfltDisklessStorPool | 0 | 1048 |
/dev/drbd1048 | | Unused | TieBreaker |
| node4 | vm-101-disk-3 | pool-drbd-4-5 | 0 |
1048 | /dev/drbd1048 | 146.09 GiB | InUse | UpToDate |
| node5 | vm-101-disk-3 | pool-drbd-4-5 | 0 |
1048 | /dev/drbd1048 | 147.09 GiB | Unused | UpToDate |
| node1 | vm-102-disk-2 | pool-drbd-1-2 | 0 |
1021 | /dev/drbd1021 | 16 MiB | InUse | UpToDate |
| node2 | vm-102-disk-2 | pool-drbd-1-2 | 0 |
1021 | /dev/drbd1021 | 16 MiB | Unused | UpToDate |
| node3 | vm-102-disk-2 | DfltDisklessStorPool | 0 | 1021 |
/dev/drbd1021 | | Unused | TieBreaker |
| node1 | vm-102-disk-3 | pool-drbd-1-2 | 0 |
1044 | /dev/drbd1044 | 10.25 GiB | InUse | UpToDate |
| node2 | vm-102-disk-3 | pool-drbd-1-2 | 0 |
1044 | /dev/drbd1044 | 10.25 GiB | Unused | UpToDate |
| node3 | vm-102-disk-3 | DfltDisklessStorPool | 0 | 1044 |
/dev/drbd1044 | | Unused | TieBreaker |
[...]
+-------------------------------------------------------------------------------------------------------------------------------------------------------------+
real 0m58.888s
user 0m0.229s
sys 0m0.047s
There is 113 volumes (with TieBreakers):
# linstor volume list | grep -c /dev/drbd
113
Performance graphs in Proxmox are all scattered after upgrade, I suppose
it's related to long query times. It looks like this - top image is
before upgrade, bottom one is after upgrade: https://imgur.com/a/c5FrS0y
I've changed node on which linstor-controller runs but it didn't help.
Is there anything I can do to make it work like before this upgrade? I
will gladly provide additional info if needed.
--
Best regards
Łukasz Wąsikowski
_______________________________________________
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user
