[Bug 258755] vmxnet3: low throughput
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258755 Michal Vanco changed: What|Removed |Added Version|13.0-RELEASE|13.2-RELEASE --- Comment #4 from Michal Vanco --- This issue is still present in 13.2-RELEASE-p10. Is really nobody interested in solving this? -- You are receiving this mail because: You are the assignee for the bug.
[Bug 258755] vmxnet3: low throughput
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258755 --- Comment #5 from Michal Vanco --- This issue is still present in 13.2-RELEASE-p10. Is really nobody interested in solving this? -- You are receiving this mail because: You are the assignee for the bug.
problems booting netbsd guest on bhyve host
Hi, I'm trying to boot most recent netbsd9 into bhyve. Trying to start the install like this # bhyve -AHP -s 0:0,amd_hostbridge -s 1:0,lpc \ -s 2:0,virtio-net,tap6 \ -s 3:0,virtio-blk,./netbsd9.img \ -s 4:0,ahci-cd,/isos/NetBSD/9/NetBSD-9.3-amd64.iso -c 4 -m 8G \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ -l com1,/dev/nmdm0A \ netbsd9 then, in another terminal # cu -l /dev/nmdm0B result: NetBSD/x86 EFI Boot (x64), Revision 1.1 (Thu Aug 4 15:30:37 UTC 2022) (from NetBSD 9.3) Memory: 640/3137664 k 1. Boot normally 2. Boot single user 3. Drop to boot prompt Choose an option; RETURN for default; SPACE to stop countdown. Option 1 will be chosen in 0 seconds. command(s): rndseed /var/db/entropy-file;boot default boot twice, skipping... 18476680+668736+1428416 [869943+1252488+894273]=0x16e11b0 WARNING: couldn't open /var/db/entropy-file Loading /stand/amd64/9.3/modules/cd9660/cd9660.kmod WARNING: 1 module failed to load On a linux box I can create a netbsd vm in virtualbox. Why not in bhyve? thanks in advance for any help/assistance --
Re: bhyve disk performance issue
On 2/18/24 09:21, Matthew Grooms wrote: On 2/17/24 15:53, Matthew Grooms wrote: Unfortunately same story with 13.2. I'm going to try and downgrade to 12.4 and see if it gets any better. Begin @ Sat Feb 17 11:00:01 CST 2024 Version 2.00 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP localhost.lo 63640M 690k 99 1.5g 97 727m 78 950k 99 1.3g 68 + +++ Latency 11759us 29114us 8098us 8649us 23413us 4540us Version 2.00 --Sequential Create-- Random Create localhost.localdoma -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ + +++ + +++ + +++ Latency 7791us 131us 1671us 464us 15us 1811us End @ Sat Feb 17 11:03:13 CST 2024 Begin @ Sat Feb 17 11:10:01 CST 2024 Version 2.00 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP localhost.lo 63640M 667k 99 449m 99 313m 94 940k 99 398m 99 16204 563 Latency 12147us 1079us 24470us 8795us 17853us 4384us Version 2.00 --Sequential Create-- Random Create localhost.localdoma -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 0 93 + +++ + +++ 0 96 + +++ + +++ Latency 118us 159us 269us 164us 54us 844us End @ Sat Feb 17 11:18:43 CST 2024 I wasn't able to get a working 12.4 system in place due to lack of packages. However, I did fire up a FreeBSD 14 VM and let it run overnight on the same SSD array. It consistently ran at a much higher speed for 50+ runs @ 10m intervals ... Begin @ Sun Feb 18 15:00:00 UTC 2024 Version 1.98 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP freebsd.shrew.l 64G 628k 99 1.6g 98 831m 60 1278k 99 1.1g 42 + +++ Latency 13447us 68490us 207ms 7187us 195ms 17665us Version 1.98 --Sequential Create-- Random Create freebsd.shrew.lab -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ + +++ + +++ + +++ Latency 18225us 18us 28us 18812us 18us 25us End @ Sun Feb 18 15:03:11 UTC 2024 I used identical options to run both that VM and the Linux VM I've been testing. The backing store for each VM has a 1TB partition and the guest interfaces are NVME. Now I'm really scratching my head. Chuck, were you testing disk performance in Linux VMs or only FreeBSD? Anyone have ideas on why Linux disk performance would drop off a cliff over time? I took a detour trying out Xen but apparently that has it's own set of performance issues related to the FreeBSD port missing newer features. I did install KVM on the same hardware for comparison. I then tested a guest provisioned identically to the bhyve VM with a virtio blk device which ran for 2.5 hours with consistently impressive output ... Begin @ Thu Feb 22 20:55:01 CST 2024 Version 2.00 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP linux2.shrew.pr 31G 2191k 99 1.4g 55 1.1g 59 3484k 99 2.5g 83 7817 127 Latency 4480us 2528us 17656us 2650us 102us 3568us Version 2.00 --Sequential Create-- Random Create linux2.shrew.prv -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ + +++ + +++ + +++ Latency 9722us 90us 123us 61us 20us 998us End @ Thu Feb 22 20:56:11 CST 2024 =
Re: bhyve disk performance issue
On 23/02/2024 4:51 pm, Matthew Grooms wrote: On 2/18/24 09:21, Matthew Grooms wrote: On 2/17/24 15:53, Matthew Grooms wrote: Unfortunately same story with 13.2. I'm going to try and downgrade to 12.4 and see if it gets any better. Begin @ Sat Feb 17 11:00:01 CST 2024 Version 2.00 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP localhost.lo 63640M 690k 99 1.5g 97 727m 78 950k 99 1.3g 68 + +++ Latency 11759us 29114us 8098us 8649us 23413us 4540us Version 2.00 --Sequential Create-- Random Create localhost.localdoma -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ + +++ + +++ + +++ Latency 7791us 131us 1671us 464us 15us 1811us End @ Sat Feb 17 11:03:13 CST 2024 Begin @ Sat Feb 17 11:10:01 CST 2024 Version 2.00 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP localhost.lo 63640M 667k 99 449m 99 313m 94 940k 99 398m 99 16204 563 Latency 12147us 1079us 24470us 8795us 17853us 4384us Version 2.00 --Sequential Create-- Random Create localhost.localdoma -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 0 93 + +++ + +++ 0 96 + +++ + +++ Latency 118us 159us 269us 164us 54us 844us End @ Sat Feb 17 11:18:43 CST 2024 I wasn't able to get a working 12.4 system in place due to lack of packages. However, I did fire up a FreeBSD 14 VM and let it run overnight on the same SSD array. It consistently ran at a much higher speed for 50+ runs @ 10m intervals ... Begin @ Sun Feb 18 15:00:00 UTC 2024 Version 1.98 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP freebsd.shrew.l 64G 628k 99 1.6g 98 831m 60 1278k 99 1.1g 42 + +++ Latency 13447us 68490us 207ms 7187us 195ms 17665us Version 1.98 --Sequential Create-- Random Create freebsd.shrew.lab -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ + +++ + +++ + +++ Latency 18225us 18us 28us 18812us 18us 25us End @ Sun Feb 18 15:03:11 UTC 2024 I used identical options to run both that VM and the Linux VM I've been testing. The backing store for each VM has a 1TB partition and the guest interfaces are NVME. Now I'm really scratching my head. Chuck, were you testing disk performance in Linux VMs or only FreeBSD? Anyone have ideas on why Linux disk performance would drop off a cliff over time? I took a detour trying out Xen but apparently that has it's own set of performance issues related to the FreeBSD port missing newer features. I did install KVM on the same hardware for comparison. I then tested a guest provisioned identically to the bhyve VM with a virtio blk device which ran for 2.5 hours with consistently impressive output ... Begin @ Thu Feb 22 20:55:01 CST 2024 Version 2.00 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP linux2.shrew.pr 31G 2191k 99 1.4g 55 1.1g 59 3484k 99 2.5g 83 7817 127 Latency 4480us 2528us 17656us 2650us 102us 3568us Version 2.00 --Sequential Create-- Random Create linux2.shrew.prv -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ + +++ + +++ + +++ Latency 9722us 90us 123us 61us 20us 998us End @ Thu Feb 22 20:5
Re: bhyve disk performance issue
On 2/23/24 00:25, Jason Tubnor wrote: On 23/02/2024 4:51 pm, Matthew Grooms wrote: On 2/18/24 09:21, Matthew Grooms wrote: On 2/17/24 15:53, Matthew Grooms wrote: Unfortunately same story with 13.2. I'm going to try and downgrade to 12.4 and see if it gets any better. Begin @ Sat Feb 17 11:00:01 CST 2024 Version 2.00 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP localhost.lo 63640M 690k 99 1.5g 97 727m 78 950k 99 1.3g 68 + +++ Latency 11759us 29114us 8098us 8649us 23413us 4540us Version 2.00 --Sequential Create-- Random Create localhost.localdoma -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ + +++ + +++ + +++ Latency 7791us 131us 1671us 464us 15us 1811us End @ Sat Feb 17 11:03:13 CST 2024 Begin @ Sat Feb 17 11:10:01 CST 2024 Version 2.00 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP localhost.lo 63640M 667k 99 449m 99 313m 94 940k 99 398m 99 16204 563 Latency 12147us 1079us 24470us 8795us 17853us 4384us Version 2.00 --Sequential Create-- Random Create localhost.localdoma -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 0 93 + +++ + +++ 0 96 + +++ + +++ Latency 118us 159us 269us 164us 54us 844us End @ Sat Feb 17 11:18:43 CST 2024 I wasn't able to get a working 12.4 system in place due to lack of packages. However, I did fire up a FreeBSD 14 VM and let it run overnight on the same SSD array. It consistently ran at a much higher speed for 50+ runs @ 10m intervals ... Begin @ Sun Feb 18 15:00:00 UTC 2024 Version 1.98 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP freebsd.shrew.l 64G 628k 99 1.6g 98 831m 60 1278k 99 1.1g 42 + +++ Latency 13447us 68490us 207ms 7187us 195ms 17665us Version 1.98 --Sequential Create-- Random Create freebsd.shrew.lab -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ + +++ + +++ + +++ Latency 18225us 18us 28us 18812us 18us 25us End @ Sun Feb 18 15:03:11 UTC 2024 I used identical options to run both that VM and the Linux VM I've been testing. The backing store for each VM has a 1TB partition and the guest interfaces are NVME. Now I'm really scratching my head. Chuck, were you testing disk performance in Linux VMs or only FreeBSD? Anyone have ideas on why Linux disk performance would drop off a cliff over time? I took a detour trying out Xen but apparently that has it's own set of performance issues related to the FreeBSD port missing newer features. I did install KVM on the same hardware for comparison. I then tested a guest provisioned identically to the bhyve VM with a virtio blk device which ran for 2.5 hours with consistently impressive output ... Begin @ Thu Feb 22 20:55:01 CST 2024 Version 2.00 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP linux2.shrew.pr 31G 2191k 99 1.4g 55 1.1g 59 3484k 99 2.5g 83 7817 127 Latency 4480us 2528us 17656us 2650us 102us 3568us Version 2.00 --Sequential Create-- Random Create linux2.shrew.prv -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 + +++ + +++ + +++ + +++ + +++ + +++ Latency 9722us 90us 123us 6
