Hi,
Op 28/04/2023 om 10:45 schreef Pierre Le Fevre:
Hi all,
We're working on upgrading our storage solution to a proper network
attached storage.
Before this, we had an NFS share on some mounted disks on the management
server. Our new setup is a NAS running TrueNAS (zfs) with 64 GB ram and
8x8TB, 7200 rpm hard disks mounted to cloudstack over NFS.
It seems the performance is however much lower than before, resulting in
somewhat unusable virtual machines. In VMs, IOPS can be as low as <10 IOPS,
vs 100 with the management server setup.
Is there a recommended setup for primary storage to yield better
performance?
NFS served from TrueNAS with ZFS can work just fine and very fast. We
use this as well with CloudStack and we get more then 10k IOps with QD=1
and BS=4k.
Now, this isn't a CloudStack question as this is mainly about NFS and
ZFS performance.
A few things to think about:
- Use a ZIL device with ZFS, Flash based
- vdevs should be not to big
- Make sure that your vdev size aligns with 4k block.
- RAIDZ1 you want 5 drives
- RAIDZ2 you want 6 drives
- RAIDZ3 you want 7 drives
- Use nconnect feature of NFS
- Latency is what matters! Not bandwidth
- Network
- Storage
Search around on the internet, because a lot has been written about this
already.
Wido
All the best
Pierre,
kthcloud
