Re[2]: Re[2]: Re[2]: Migrate from real server to virtual
--- Original message --- From: "Rodney W. Grimes" Date: 27 February 2018, 09:13:56 > ... > > > > Boot Failed? So windows never started. > > > Your not having a shutdown problem your having a start up failure. > > > > > > What is the output of: > > > file /vm/img/disk1.raw > > > file /VM/distro/null.iso > > > > > > And do you really have /vm and /VM? Or is that a typo? > > > You have not answered these 3 questions > what is output of "file /vm/img/disk1.raw" > what is output of "file /VM/distro/null.iso" > > Is there a possible typo here in your command line??? vm vs VM? This files are present and without typo as I have copied path and filenames via clipboard. Next I will try with sysutils/uefi-edk2-bhyve-csm. -- Vitaly ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: superfluous host interfaces
Bezüglich Paul Vixie's Nachricht vom 27.02.2018 07:14 (localtime): > > > Harry Schmalzbauer wrote: >> Bezüglich Ruben's Nachricht vom 26.02.2018 11:34 (localtime): >>> On 26/02/2018 10:56, Harry Schmalzbauer wrote: >> … Another, personally very significant, reason is that you'll get a superfluous host interface for each if_bridge(4), which makes the output of plain ifconfig(8) kind of unreadable. >> … >>> By superflous host interfaces, do you mean the tap interfaces configured >>> for each vm together with the bridge interfaces they are "bundled" in? >> >> Additionally to the if_tap(4) ethernet host interfaces, you also get >> if_bridge(4) ethernet interfaces, named bridgeX if I remember correctly. > > you do not remember correctly. Please see next para. >> [mm1.redbarn:amd64] egrep 'bridge|tap' /etc/rc.conf >> autobridge_interfaces="bridge0" >> autobridge_bridge0="tap* igb1" >> cloned_interfaces="bridge0 tap0 tap1 tap2 tap3 tap4 tap5 tap6 tap7" >> ifconfig_bridge0="inet 24.104.150.210/27" >> ifconfig_bridge0_ipv6="inet6 2001:559:8000:cd::2/64 auto_linklocal up" >> ifconfig_tap0="up" >> ifconfig_tap1="up" >> ifconfig_tap2="up" >> ifconfig_tap3="up" >> ifconfig_tap4="up" >> ifconfig_tap5="up" >> ifconfig_tap6="up" >> ifconfig_tap7="up" >> [mm1.redbarn:amd64] ifconfig | egrep '^(bridge|tap)' >> bridge0: flags=8843 metric 0 >> mtu 1500 That's what I mean; and it's named bridgeX, so memeory works in that case ;-) If you have only one "LAN" sharing all VMs, the one additional interface is neglectable. But my setups are different. I have almost as many different 802.11q separated ethernet collsion domains (VLANs) as VMs. That's what ESXi's portgroup is used for. I need a separate switch for each VLAN (guests mustn't be able to sniff traffic etc.). … >> And using ng_bridge(4) instead of if_bridge(4) doesn't change the need >> for if_tap(4). Only with vale(4) switches, bhyve(8) was able to provide >> virtio-net connection wihtout "spamming" the host's ethernet interface >> list (no tapX, no bridgeX). > > how did you get bhyve to use the netmap API rather than the tap > character special device? Not me, peter commited the following: https://svnweb.freebsd.org/base/head/usr.sbin/bhyve/pci_virtio_net.c?r1=288470&r2=293459 >From that an, you don't need to add any host ethernet devices/interfaces, simply start your VM with e.g. "-s 1:0,virtio-net,vale0:1,mac=02:03:04:05:06:07". As long as you created vale0, port [:]1 will be created dynamically and also destroyed after shutdown. Again, to mimic ESXi's portgroups, you need one vale(4) switch for each VLAN. And to uplink, you need to utilize if_valn(4), which forces netmap emulated mode. If you don't have/need VLANs, you can uplink a supported NIC via native netmap support, and additionally gain significatn efficiency improvements. Unfortunately, at least if_vlan(4) uplinks, don't work reliably. After some short time, the complete network netmap(4) subsystem locks up. I talked with Vincenco Maffione (a member of Liugi Rizzo's netmap(4) team of University of Pisa) and fixing emulated netpmap mode on FreeBSD doesn't have really high priority there, since such a setup is considered as weak design. For sure, it's a hack/workarround, but we don't have VLAN/portgroup support in vale(4) nor in byhve(8), and writing my own userland filter is beyond my scope. -harry ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Re[2]: Re[2]: Re[2]: Migrate from real server to virtual
[ Charset UTF-8 unsupported, converting... ] > > > > --- Original message --- > From: "Rodney W. Grimes" > Date: 27 February 2018, 09:13:56 > > > > > ... > > > > > > Boot Failed? So windows never started. > > > > Your not having a shutdown problem your having a start up failure. > > > > > > > > What is the output of: > > > > file /vm/img/disk1.raw > > > > file /VM/distro/null.iso > > > > > > > > And do you really have /vm and /VM? Or is that a typo? > > > > > > You have not answered these 3 questions > > what is output of "file /vm/img/disk1.raw" > > what is output of "file /VM/distro/null.iso" > > > > Is there a possible typo here in your command line??? vm vs VM? > > This files are present and without typo as I have copied path and filenames > via clipboard. > Next I will try with sysutils/uefi-edk2-bhyve-csm. It would still be helpful to know what the host system thinks of these files, please output of: file /vm/img/disk1.raw file /VM/distory/null.iso -- Rod Grimes rgri...@freebsd.org ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re[2]: Re[2]: Re[2]: Re[2]: Migrate from real server to virtual
--- Original message --- From: "Rodney W. Grimes" Date: 27 February 2018, 10:28:37 > [ Charset UTF-8 unsupported, converting... ] > > > > > > > > --- Original message --- > > From: "Rodney W. Grimes" > > Date: 27 February 2018, 09:13:56 > > > > > > > > > ... > > > > > > > > Boot Failed? So windows never started. > > > > > Your not having a shutdown problem your having a start up failure. > > > > > > > > > > What is the output of: > > > > > file /vm/img/disk1.raw > > > > > file /VM/distro/null.iso > > > > > > > > > > And do you really have /vm and /VM? Or is that a typo? > > > > > > > > > You have not answered these 3 questions > > > what is output of "file /vm/img/disk1.raw" > > > what is output of "file /VM/distro/null.iso" > > > > > > Is there a possible typo here in your command line??? vm vs VM? > > > > This files are present and without typo as I have copied path and filenames > > via clipboard. > > Next I will try with sysutils/uefi-edk2-bhyve-csm. > > It would still be helpful to know what the host system thinks of > these files, please output of: > file /vm/img/disk1.raw > file /VM/distory/null.iso Ok, # file ./null.iso ./null.iso: empty # file ./disk1.raw ./disk1.raw: DOS/MBR boot sector MS-MBR Windows 7 english at offset 0x163 "Invalid partition table" at offset 0x17b "Error loading operating system" at offset 0x19a "Missing operating system", disk signature 0xbaf010d7; partition 1 : ID=0x7, active, start-CHS (0x0,32,33), end-CHS (0xc,223,19), startsector 2048, 204800 sectors; partition 2 : ID=0x7, start-CHS (0xc,223,20), end-CHS (0x3ff,254,63), startsector 206848, 253745152 sectors; partition 3 : ID=0x7, start-CHS (0x3ff,254,63), end-CHS (0x3ff,254,63), startsector 253953468, 722810844 sectors May be I incorrect have converted this image?... I have just tested with UEFI_CSM. I am able manage BHYVE UEFI via F2 key pressing and there are Legacy CD-ROM and Legacy Hard Drive options. If choosing HDD and continue to booting just black screen... ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 226142] [bhyve] [panic] mtx_lock_spin: recursed on non-recursive mutex vcpu lock @ /usr/src/sys/amd64/vmm/vmm.c:2246
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226142 Mark Linimon changed: What|Removed |Added Assignee|freebsd-b...@freebsd.org|freebsd-virtualization@Free ||BSD.org -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 226142] [bhyve] [panic] mtx_lock_spin: recursed on non-recursive mutex vcpu lock @ /usr/src/sys/amd64/vmm/vmm.c:2246
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226142 Peter Grehan changed: What|Removed |Added CC||gre...@freebsd.org --- Comment #1 from Peter Grehan --- The issue (and a proposed fix) is discussed here https://reviews.freebsd.org/D14382 -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve manager
On Sun, Feb 25, 2018 at 01:14:01PM +, tech-lists wrote: Hi, What do folks use for their bhyve guest management? I have always spun bhyve guests up by hand but now I'm considering streamlining the process. What do you use? thanks for all the suggestions. I wasn't sure where to start before. Now I know. -- J. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 225588] VirtualBox 5.2.6 guests freeze after a while, sometimes with I/O errors
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225588 Mark Linimon changed: What|Removed |Added Assignee|freebsd-ports-bugs@FreeBSD. |freebsd-virtualization@Free |org |BSD.org -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Re[2]: Re[2]: Re[2]: Re[2]: Migrate from real server to virtual
> --- Original message --- > From: "Rodney W. Grimes" > Date: 27 February 2018, 10:28:37 > > > --- Original message --- > > > From: "Rodney W. Grimes" > > > Date: 27 February 2018, 09:13:56 > > > > ... > > > > > > > > > > Boot Failed? So windows never started. > > > > > > Your not having a shutdown problem your having a start up failure. > > > > > > > > > > > > What is the output of: > > > > > > file /vm/img/disk1.raw > > > > > > file /VM/distro/null.iso Your command in the original email had these 2 strings, please correct that error in your command. VM != vm in unix world. > > > > > > And do you really have /vm and /VM? Or is that a typo? > > > > > > > > > > > > You have not answered these 3 questions > > > > what is output of "file /vm/img/disk1.raw" > > > > what is output of "file /VM/distro/null.iso" > > > > > > > > Is there a possible typo here in your command line??? vm vs VM? > > > > > > This files are present and without typo as I have copied path and > > > filenames via clipboard. > > > Next I will try with sysutils/uefi-edk2-bhyve-csm. > > > > It would still be helpful to know what the host system thinks of > > these files, please output of: > > file /vm/img/disk1.raw > > file /VM/distory/null.iso > > Ok, > > # file ./null.iso > ./null.iso: empty > > # file ./disk1.raw > ./disk1.raw: DOS/MBR boot sector MS-MBR Windows 7 english at offset 0x163 > "Invalid partition table" at offset 0x17b "Error loading operating system" at > offset 0x19a "Missing operating system", disk signature 0xbaf010d7; partition > 1 : ID=0x7, active, start-CHS (0x0,32,33), end-CHS (0xc,223,19), startsector > 2048, 204800 sectors; partition 2 : ID=0x7, start-CHS (0xc,223,20), end-CHS > (0x3ff,254,63), startsector 206848, 253745152 sectors; partition 3 : ID=0x7, > start-CHS (0x3ff,254,63), end-CHS (0x3ff,254,63), startsector 253953468, > 722810844 sectors Ok so infact these 2 files are in the same directory, that needs to be spelled exactly the same on the command line of bhyyve. But I think the bigger problem is that this is an MBR disk with Windows 7 in it, I do not believe that bhyve can run Windows 7 at this time. > May be I incorrect have converted this image?... The results above look ok, though the alignment is a bit odd. You man want to play with mdconfig and fusefs to see if you can read the converted image: pkg install fusefs-ntfs mdconfig -f disk1.raw ntfs-3g /dev/md0s1 /mnt find /mnt That should atleast validate the file system a little. > I have just tested with UEFI_CSM. > I am able manage BHYVE UEFI via F2 key pressing and there are Legacy CD-ROM > and Legacy Hard Drive options. If choosing HDD and continue to booting just > black screen... Peter pointed out to us that UEFI_CSM does not work with windows. I think the bigger problem here is this appears to be a windows 7 image? -- Rod Grimes rgri...@freebsd.org ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Ryzen lockup on bhyve was (Re: new Ryzen lockup issue ?)
On 2/25/2018 12:51 PM, Peter Grehan wrote: > Hi Mike, > > (moving this to -virtualization) > >> Actually I can confirm the same sort of hard lockup happens on my Epyc >> board with RELENG11. It also happens in current. I will file a PR and >> post on freebsd-current in case someone has any suggestions on how to >> try and figure out whats going on. > > I could reproduce this on a Ryzen 1700 with HT enabled - it took ~700 > seconds, and the entire machine locked up. > > An interesting point is a single VM was able to run to completion. > > I'll keep testing with various combinations of numbers of VMs, > different vCPUs per VM, and pinning guest vCPUs. Hi Peter, I upgraded both boxes (Ryzen and Epyc) to HEAD. For the record, the Epyc CPU is CPU1 Information AMD EPYC 7281 16-Core Family: 17h Model: 00h-0Fh Microcode Patch Level: 8001207 Bios Release Date: 01/08/2018 SuperMicro: H11SSL-i Not sure if its related to core count or not, but it takes several instances on the Epyc to lock up and takes a lot longer. I enabled IOMMU on it as well, and it's dmesg shows below. But the lockups are still there with or without it enabled. ivhd0: on acpi0 ivhd0: Flag:b0 ivhd0: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0 ivhd0: Extended features[31:0]:22294ada HATS = 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1 DualPortLogSup = 0x2 DualEventLogSup = 0x2 ivhd0: Extended features[62:32]:f77ef Max PASID: 0x2f DevTblSegSup = 0x3 MarcSup = 0x1 ivhd0: supported paging level:7, will use only: 4 ivhd0: device range: 0x0 - 0x ivhd0: PCI cap 0x190b640f@0x40 feature:19 ivhd1: on acpi0 ivhd1: Flag:b0 ivhd1: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0 ivhd1: Extended features[31:0]:22294ada HATS = 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1 DualPortLogSup = 0x2 DualEventLogSup = 0x2 ivhd1: Extended features[62:32]:f77ef Max PASID: 0x2f DevTblSegSup = 0x3 MarcSup = 0x1 ivhd1: supported paging level:7, will use only: 4 ivhd1: device range: 0x0 - 0x ivhd1: PCI cap 0x190b640f@0x40 feature:19 ivhd2: on acpi0 ivhd2: Flag:b0 ivhd2: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0 ivhd2: Extended features[31:0]:22294ada HATS = 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1 DualPortLogSup = 0x2 DualEventLogSup = 0x2 ivhd2: Extended features[62:32]:f77ef Max PASID: 0x2f DevTblSegSup = 0x3 MarcSup = 0x1 ivhd2: supported paging level:7, will use only: 4 ivhd2: device range: 0x0 - 0x3ffe ivhd2: PCI cap 0x190b640f@0x40 feature:19 ivhd3: on acpi0 ivhd3: Flag:b0 ivhd3: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0 ivhd3: Extended features[31:0]:22294ada HATS = 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1 DualPortLogSup = 0x2 DualEventLogSup = 0x2 ivhd3: Extended features[62:32]:f77ef Max PASID: 0x2f DevTblSegSup = 0x3 MarcSup = 0x1 ivhd3: supported paging level:7, will use only: 4 ivhd3: device range: 0x0 - 0x3ffe ivhd3: PCI cap 0x190b640f@0x40 feature:19 ---Mike -- --- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, m...@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Migrate from real server to virtual
But I think the bigger problem is that this is an MBR disk with Windows 7 in it, I do not believe that bhyve can run Windows 7 at this time. Bhyve has been able to run Win7 since 2015, though x64 only and in UEFI mode. later, Peter. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
bhyve bombed out..
environment: HOST: freebsd 12 (a couple of weeks ago) on ESX 6.0.0 yes.. nested Virtualisation Client: slightly modified freebsd current (from the 20th) Bhyve dies afer a while with: vm exit[0] reason VMX rip 0x0008003e4b03 inst_length 0 status 0 exit_reason 48 qualification 0x0001 inst_type 0 inst_error 0 Abort trap *** Error code 99 Any suggestions? I'm working towards having a hardware host. This doesn't work at all on esx 5.1.1 so I'm actually amazed it runs for as long as it does on 6.0.0... (days if not doing much). ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 226142] [bhyve] [panic] mtx_lock_spin: recursed on non-recursive mutex vcpu lock @ /usr/src/sys/amd64/vmm/vmm.c:2246
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226142 --- Comment #2 from Dave Cottlehuber --- Thanks Peter! giving it a whirl now; https://github.com/skunkwerks/freebsd/commit/e1f1fbb91c80d7f60801e2487c3e8761d61f7864 for a git-friendly branch. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Differential] D14473: userboot: add callbacks to set unrestricted guest mode
fabian.freyer_physik.tu-berlin.de added a comment. ping? Would these changes be welcome in base? If yes, what would still need to be done to this review in order to get it into a committable state? REVISION DETAIL https://reviews.freebsd.org/D14473 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: fabian.freyer_physik.tu-berlin.de, imp, rgrimes, #bhyve, grehan Cc: grehan, imp, freebsd-virtualization-list, #contributor_reviews_base ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Differential] D14473: userboot: add callbacks to set unrestricted guest mode
rgrimes added a comment. If Peter accepts this and says I can go ahead with a commit I would do that, but I need his approval to commit it. REVISION DETAIL https://reviews.freebsd.org/D14473 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: fabian.freyer_physik.tu-berlin.de, imp, rgrimes, #bhyve, grehan Cc: grehan, imp, freebsd-virtualization-list, #contributor_reviews_base ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: superfluous host interfaces
On 27/02/2018 08:17, Harry Schmalzbauer wrote: > Bezüglich Paul Vixie's Nachricht vom 27.02.2018 07:14 (localtime): >> >> >> Harry Schmalzbauer wrote: >>> Bezüglich Ruben's Nachricht vom 26.02.2018 11:34 (localtime): On 26/02/2018 10:56, Harry Schmalzbauer wrote: >>> … > Another, personally very significant, reason is that you'll get a > superfluous host interface for each if_bridge(4), which makes the > output > of plain ifconfig(8) kind of unreadable. >>> … By superflous host interfaces, do you mean the tap interfaces configured for each vm together with the bridge interfaces they are "bundled" in? >>> >>> Additionally to the if_tap(4) ethernet host interfaces, you also get >>> if_bridge(4) ethernet interfaces, named bridgeX if I remember correctly. >> >> you do not remember correctly. > > Please see next para. > > >>> [mm1.redbarn:amd64] egrep 'bridge|tap' /etc/rc.conf >>> autobridge_interfaces="bridge0" >>> autobridge_bridge0="tap* igb1" >>> cloned_interfaces="bridge0 tap0 tap1 tap2 tap3 tap4 tap5 tap6 tap7" >>> ifconfig_bridge0="inet 24.104.150.210/27" >>> ifconfig_bridge0_ipv6="inet6 2001:559:8000:cd::2/64 auto_linklocal up" >>> ifconfig_tap0="up" >>> ifconfig_tap1="up" >>> ifconfig_tap2="up" >>> ifconfig_tap3="up" >>> ifconfig_tap4="up" >>> ifconfig_tap5="up" >>> ifconfig_tap6="up" >>> ifconfig_tap7="up" >>> [mm1.redbarn:amd64] ifconfig | egrep '^(bridge|tap)' >>> bridge0: flags=8843 metric 0 >>> mtu 1500 > > That's what I mean; and it's named bridgeX, so memeory works in that > case ;-) > If you have only one "LAN" sharing all VMs, the one additional interface > is neglectable. > But my setups are different. > I have almost as many different 802.11q separated ethernet collsion > domains (VLANs) as VMs. > That's what ESXi's portgroup is used for. I need a separate switch for > each VLAN (guests mustn't be able to sniff traffic etc.). > Untested by me with bhyve but it looks like net/openvswitch (http://www.openvswitch.org/) could be useful to you. As I say though untested by me so cant speak for performance etc. Vince > > … > >>> And using ng_bridge(4) instead of if_bridge(4) doesn't change the need >>> for if_tap(4). Only with vale(4) switches, bhyve(8) was able to provide >>> virtio-net connection wihtout "spamming" the host's ethernet interface >>> list (no tapX, no bridgeX). >> >> how did you get bhyve to use the netmap API rather than the tap >> character special device? > > Not me, peter commited the following: > https://svnweb.freebsd.org/base/head/usr.sbin/bhyve/pci_virtio_net.c?r1=288470&r2=293459 > > From that an, you don't need to add any host ethernet > devices/interfaces, simply start your VM with e.g. "-s > 1:0,virtio-net,vale0:1,mac=02:03:04:05:06:07". > > As long as you created vale0, port [:]1 will be created dynamically and > also destroyed after shutdown. > > Again, to mimic ESXi's portgroups, you need one vale(4) switch for each > VLAN. And to uplink, you need to utilize if_valn(4), which forces > netmap emulated mode. > If you don't have/need VLANs, you can uplink a supported NIC via native > netmap support, and additionally gain significatn efficiency improvements. > Unfortunately, at least if_vlan(4) uplinks, don't work reliably. After > some short time, the complete network netmap(4) subsystem locks up. > I talked with Vincenco Maffione (a member of Liugi Rizzo's netmap(4) > team of University of Pisa) and fixing emulated netpmap mode on FreeBSD > doesn't have really high priority there, since such a setup is > considered as weak design. For sure, it's a hack/workarround, but we > don't have VLAN/portgroup support in vale(4) nor in byhve(8), and > writing my own userland filter is beyond my scope. > > -harry > ___ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" > ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: superfluous host interfaces
On 26/2/18 6:34 pm, Ruben wrote: On 26/02/2018 10:56, Harry Schmalzbauer wrote: Hi Harry, What are your reasons for preferring ng_bridge over the "normal" bridge? Two very different main reasons: if_bridge(4) is very standards compliant (e.g. that different reserved MAC addresses won't get forwarded – don't know any explicit examples out of mind), which was problematic for some SDN setups (software defined networking, in means of sharing a PHY for multiple VMs and simultaniously interconnect VMs to VMs) Another, personally very significant, reason is that you'll get a superfluous host interface for each if_bridge(4), which makes the output of plain ifconfig(8) kind of unreadable. For VM SDN, I don't need/want those host interfaces, despite they don't do any harm. vale(4) was extremely convinient. Simply create a switch, then each VM attaches on the fly :-) Unfortunately, I'm unable to debug the lockups and my setups was kind of hacky, since I haven't used NIC's native netmap(4) support, but used emulated netmap(4) for if_vlan(4). This leads to loss of almost all performance advantages, but left convinience advantages. Unfortunately, emulated netmap(4) is supposed to have some unresolved problems on FreeBSD and upstream hackers consider my hacky setup as wrong by nature – which it is technically speaking. For real-world usagen, one would need to code a VLAN filter between bhyve(4) and vale(4). Skillwise, I'm not the one :-( -harry Hi Harry, Thank you for elaborating on that. I took the liberty of creating a new mailthread as my questions are kind of off-topic to the original thread. By superflous host interfaces, do you mean the tap interfaces configured for each vm together with the bridge interfaces they are "bundled" in? Overall I'm very happy with my bhyve setups atm. If there are any speed-/administrative-advantages that come with bridge_ng however, I'm very interested in switching to such a setup (or at least play with it). I'm running my vm's without any helper project so I'm flexible enough to do some fiddling :P Do you know of any documentation on using bridge_ng together with bhyve? My search-engines don't turn up much Im affraid and I haven't stumbled on it before. I will add another postive to ng_bridge and a negative.. using ng_bridge makes it easier to do funk things with your network if you want to but it requires you to do your own scripting. see the jail examples in /usr/share/examples and replace the jail with bhyve. Kind regards, Ruben ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: superfluous host interfaces
On 26/2/18 8:13 pm, Harry Schmalzbauer wrote: Bezüglich Ruben's Nachricht vom 26.02.2018 11:34 (localtime): On 26/02/2018 10:56, Harry Schmalzbauer wrote: … Another, personally very significant, reason is that you'll get a superfluous host interface for each if_bridge(4), which makes the output of plain ifconfig(8) kind of unreadable. … By superflous host interfaces, do you mean the tap interfaces configured for each vm together with the bridge interfaces they are "bundled" in? Additionally to the if_tap(4) ethernet host interfaces, you also get if_bridge(4) ethernet interfaces, named bridgeX if I remember correctly. The if_bridge(4) host interface is for control purposes only on a VM-SDN host – at least with my setups. I never needed to make use of IP numbered bridges. And I don't need to utilize any if_bridge(4) features like STP, so I consider the bridgeX host interfaces as superfluous in the VM-SDN use case. I'd call the if_tap(4) host interfaces likewise superfluous – you would only need the corresponding character devices – but that's been implemented long before the need for SDN setups, so it is like it is. And using ng_bridge(4) instead of if_bridge(4) doesn't change the need for if_tap(4). Only with vale(4) switches, bhyve(8) was able to provide virtio-net connection wihtout "spamming" the host's ethernet interface list (no tapX, no bridgeX). Overall I'm very happy with my bhyve setups atm. If there are any speed-/administrative-advantages that come with bridge_ng however, I'm very interested in switching to such a setup (or at least play with it). I'm running my vm's without any helper project so I'm flexible enough to do some fiddling :P Do you know of any documentation on using bridge_ng together with bhyve? My search-engines don't turn up much Im affraid and I haven't stumbled on it before. Unfortunately it's not too easy to get started with netgraph. Besides numerous man pages for the different nodes (ng_bridge(4) e.g.), I only know the following source for a good overview: http://www.netbsd.org/gallery/presentations/ast/2012_AsiaBSDCon/Tutorial_NETGRAPH.pdf https://people.freebsd.org/~julian/netgraph.html dated but not too much One convenience disadvantage with ng_bridge(4) is that you have to assign MACs manually, while if_bridge(4) does that itself (adjustable by sysctl net.link.bridge.inherit_mac). And you need to script all setups yourself. Almost all of my setups seem to be awkward enough that I always had to do some local scripting, so that wasn't really a disadvantage for me. If you're happy with your setup, I don't think you gain anything from switching to ng_bridge(4), besides learning to control netgraph(4) (which is very desirable imho). I haven't had time left to do useful benchmarking regarding ng_bridge(4) vs. if_bridge(4). I even don't know if netgraph nodes are still limited to single threads. But rough load comparings on a IvyBride machine showed similar resource usage for both bridges, both easy capable of 1GbE saturation with small frames (while I remember one run with ng_bridge(4) and if_vmnet(4), which couldn't deliver 1GbE speed, and I wanted to falsify for vmnet/tap difference... just ran out of time :-( ). -harry ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: superfluous host interfaces
On 26/2/18 8:13 pm, Harry Schmalzbauer wrote: If you're happy with your setup, I don't think you gain anything from switching to ng_bridge(4), besides learning to control netgraph(4) (which is very desirable imho). I haven't had time left to do useful benchmarking regarding ng_bridge(4) vs. if_bridge(4). I even don't know if netgraph nodes are still limited to single threads. depends on the node.. teh framework allows many threads to traverse it at a time but some nodes have resources that need guarding. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: superfluous host interfaces
Hi Julian, Thank you for your feedback. I'm not affraid of some scripting, ill try to make some time and dig into the ng stuff :) Regards, Ruben On 27/02/2018 22:16, Julian Elischer wrote: > On 26/2/18 8:13 pm, Harry Schmalzbauer wrote: >> >> >> If you're happy with your setup, I don't think you gain anything from >> switching to ng_bridge(4), besides learning to control netgraph(4) >> (which is very desirable imho). >> I haven't had time left to do useful benchmarking regarding ng_bridge(4) >> vs. if_bridge(4). I even don't know if netgraph nodes are still limited >> to single threads. > > depends on the node.. teh framework allows many threads to traverse it > at a time > but some nodes have resources that need guarding. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Ryzen lockup on bhyve was (Re: new Ryzen lockup issue ?)
Hi Mike, I could reproduce this on a Ryzen 1700 with HT enabled - it took ~700 seconds, and the entire machine locked up. An interesting point is a single VM was able to run to completion. I'll keep testing with various combinations of numbers of VMs, different vCPUs per VM, and pinning guest vCPUs. 3 4*vCPU guests locked the machine up in ~170 secs, 4 guests ran to completion :( Not sure if its related to core count or not, but it takes several instances on the Epyc to lock up and takes a lot longer. I enabled IOMMU on it as well, and it's dmesg shows below. But the lockups are still there with or without it enabled. Would you be able to try on the Epyc with vCPUs pinned, and all guest vCPUs on different physical CPUs ? i.e. for a configuration that will lock up, run the same configuration but with vCPUs pinned. For example, a 4 vCPU guest that will have vCPUs 0-3 pinned to host physical CPUs 7-10 would look like: -c 4 -p 0:7 -p 1:8 -p 2:9 -p 3:10 later, Peter. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re[2]: Migrate from real server to virtual
--- Original message --- From: "Peter Grehan" Date: 27 February 2018, 20:40:52 > > But I think the bigger problem is that this is an MBR disk with Windows 7 > > in it, I > > do not believe that bhyve can run Windows 7 at this time. > > Bhyve has been able to run Win7 since 2015, though x64 only and in > UEFI mode. > Yes, Windows 7 is from the same "brunch" as Windows 2008. And I think my problem is that hardware server has been installed without UEFI, just in Legacy mode. So, in this situation it is unable run this server in Bihyve with UEFI. Am I right? -- Vitaly ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
