I ran into the same issue with the "2048.00 MB AMD High Performance, 2
vCPU" on my end. Fortunately I had a snapshot and I was able to roll back.
Here's my dmesg output if that's helpful:
OpenBSD 7.3 (GENERIC.MP) #1125: Sat Mar 25 10:36:29 MDT 2023
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 2130558976 (2031MB)
avail mem = 2046648320 (1951MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5b70 (9 entries)
bios0: vendor Vultr
bios0: Vultr VHP
acpi0 at bios0: ACPI 3.0
acpi0: sleep states S3 S4 S5
acpi0: tables DSDT FACP APIC HPET MCFG WAET
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD EPYC-Rome Processor, 1996.53 MHz, 17-31-00
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB
64b/line 8-way L2 cache, 16MB 64b/line 16-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 1000MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD EPYC-Rome Processor, 1996.51 MHz, 17-31-00
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB
64b/line 8-way L2 cache, 16MB 64b/line 16-way L3 cache
cpu1: smt 1, core 0, package 0
ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
acpihpet0 at acpi0: 100000000 Hz
acpimcfg0 at acpi0
acpimcfg0: addr 0xb0000000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
"ACPI0006" at acpi0 not configured
acpipci0 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"QEMU0002" at acpi0 not configured
acpicmos0 at acpi0
"ACPI0010" at acpi0 not configured
acpicpu0 at acpi0: C1(@1 halt!)
acpicpu1 at acpi0: C1(@1 halt!)
pvbus0 at mainbus0: KVM
pvclock0 at pvbus0
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82G33 Host" rev 0x00
vga1 at pci0 dev 1 function 0 "Bochs VGA" rev 0x02
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ppb0 at pci0 dev 2 function 0 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 22
pci1 at ppb0 bus 1
virtio0 at pci1 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01
vio0 at virtio0: address 56:00:04:81:d8:d4
virtio0: msix shared
ppb1 at pci0 dev 2 function 1 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 22
pci2 at ppb1 bus 2
ppb2 at pci2 dev 0 function 0 vendor "Red Hat", unknown product 0x000e
rev 0x00
pci3 at ppb2 bus 3
"Intel 6300ESB WDT" rev 0x00 at pci3 dev 1 function 0 not configured
ppb3 at pci0 dev 2 function 2 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 22
pci4 at ppb3 bus 4
xhci0 at pci4 dev 0 function 0 vendor "Red Hat", unknown product 0x000d
rev 0x01: apic 0 int 22, xHCI 0.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Red Hat xHCI root hub" rev
3.00/1.00 addr 1
ppb4 at pci0 dev 2 function 3 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 22
pci5 at ppb4 bus 5
virtio1 at pci5 dev 0 function 0 "Qumranet Virtio 1.x Storage" rev 0x01
vioblk0 at virtio1
scsibus1 at vioblk0: 1 targets
sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, >
sd0: 61440MB, 512 bytes/sector, 125829120 sectors
virtio1: msix per-VQ
ppb5 at pci0 dev 2 function 4 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 22
pci6 at ppb5 bus 6
virtio2 at pci6 dev 0 function 0 vendor "Qumranet", unknown product
0x1045 rev 0x01
viomb0 at virtio2
virtio2: apic 0 int 22
ppb6 at pci0 dev 2 function 5 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 22
pci7 at ppb6 bus 7
virtio3 at pci7 dev 0 function 0 "Qumranet Virtio 1.x RNG" rev 0x01
viornd0 at virtio3
virtio3: msix shared
ppb7 at pci0 dev 2 function 6 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 22
pci8 at ppb7 bus 8
ppb8 at pci0 dev 2 function 7 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 22
pci9 at ppb8 bus 9
ppb9 at pci0 dev 3 function 0 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 23
pci10 at ppb9 bus 10
ppb10 at pci0 dev 3 function 1 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 23
pci11 at ppb10 bus 11
ppb11 at pci0 dev 3 function 2 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 23
pci12 at ppb11 bus 12
ppb12 at pci0 dev 3 function 3 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 23
pci13 at ppb12 bus 13
ppb13 at pci0 dev 3 function 4 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 23
pci14 at ppb13 bus 14
ppb14 at pci0 dev 3 function 5 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 23
pci15 at ppb14 bus 15
ppb15 at pci0 dev 3 function 6 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 23
pci16 at ppb15 bus 16
ppb16 at pci0 dev 3 function 7 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 23
pci17 at ppb16 bus 17
ppb17 at pci0 dev 4 function 0 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 20
pci18 at ppb17 bus 18
ppb18 at pci0 dev 4 function 1 vendor "Red Hat", unknown product 0x000c
rev 0x00: apic 0 int 20
pci19 at ppb18 bus 19
azalia0 at pci0 dev 27 function 0 "Intel 82801I HD Audio" rev 0x03: msi
azalia0: No codecs found
pcib0 at pci0 dev 31 function 0 "Intel 82801IB LPC" rev 0x02
ahci0 at pci0 dev 31 function 2 "Intel 82801I AHCI" rev 0x02: msi, AHCI 1.0
ahci0: port 2: 1.5Gb/s
scsibus2 at ahci0: 32 targets
cd0 at scsibus2 targ 2 lun 0: <QEMU, QEMU DVD-ROM, 2.5+> removable
ichiic0 at pci0 dev 31 function 3 "Intel 82801I SMBus" rev 0x02: apic 0
int 16
iic0 at ichiic0
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
uhidev0 at uhub0 port 5 configuration 1 interface 0 "QEMU QEMU USB
Tablet" rev 2.00/0.00 addr 2
uhidev0: iclass 3/0
ums0 at uhidev0: 3 buttons, Z dir
wsmouse1 at ums0 mux 0
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
root on sd0a (50928d3f17d03654.a) swap on sd0b dump on sd0b
On 7/25/23 2:37 AM, Kevin wrote:
Hey gang,
After applying today's zenbleed patches and running fw_update and
installboot -v sd0, ALL of our AMD servers running 7.3 at Vultr that
were--as part of the patch process--rebooted are now dead in the water and
won't boot.
Intel Vultr virtual machines appear to be unaffected, as are all of my
Intel bare metal machines, laptops, desktops, and servers alike.
(Thankfully, with one exception everything running an AMD cpu is redundant
OR the patches have been applied but haven't yet been rebooted, so we're
not completely dead in the water.... just close.
I was waiting for the machines to come up after reboot to reboot the
sibling servers, which are now patched but not being rebooted 'til we have
a fix.)
All machines are single CPU and were rebooted via SSH (and when that didn't
work via the Vultr console).
All ddbs look similar to these. (Sorry for the screenshots, but it's the
best I can do given these are remote virtual instances.)
[image: Screen Shot 2023-07-24 at 23.18.40.png]
[image: Screen Shot 2023-07-24 at 23.19.10.png]
[image: Screen Shot 2023-07-24 at 23.19.27.png]
[image: Screen Shot 2023-07-24 at 23.19.40.png]
Any developer who wants access to one of our machines is welcome to it.
We can also easily provision a new instance with 7.3 so someone can get in
and break shit as needed.
All affected machines are Vultr's "1024.00 MB AMD High Performance"
instances.
Available to help any way I can.
Thanks,
Kevin
P.S. If this is a PEBKAC on my part, I'm happy to take the requisite 50+
lashes with the clue stick.
