bugs@ - post upgrade to 7.5, I have lost websockets functionality via relayd for app Vaultwarden. Websockets is used in package - vaultwarden-1.30.5 - in an advanced feature that pushes data to client browsers and mobile apps in real time.
Note - the application has basic functionality without websockets via polling of the server, so at a cursory glance the app appears to work fine. So, use step (6) to test websockets. Steps to reproduce: 1 - pkg_add vaultwarden-1.30.5 2 - rcctl enable vaultwarden && rcctl start vaultwarden 3 - configure relayd with below config 4 - point web browser to the host and register for a new vaultwarden user account 5 - open a second browser session incognito / private 6 - in the first browser, create a new secure note - when websockets is working the data should show up in near real time in the other browser 7 - watch WS activity in the dev console, and note that although the WS session is established successfully, no payload data is ever received from the server - this set of screenshots shows proper operation without relayd in the first screenshot, and then failure of WS with relayd in the second screenshot - https://imgur.com/a/msvyXbX 8 - note that if you turn relayd off and use pf to send inbound web traffic to Vaultwarden's Rocket server on port 8000, then websockets works Ollie Strickland ------------------------- relayd.conf: ------------------------- table <vaultwarden-default-host> { 127.0.0.1 } # protocol definition for vaultwarden with tls http protocol vaultwarden-https { # forward connections to vaultwarden rocket match request path "/*" forward to <vaultwarden-default-host> # add headers vaultwarden may need match request header append "Host" value "$HOST" match request header append "X-Real-IP" value "$REMOTE_ADDR" match request header append "X-Forwarded-For" value "$REMOTE_ADDR" match request header append "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT" match request header append "CF-Connecting-IP" value "$REMOTE_ADDR" # various TCP options tcp { nodelay, sack, backlog 128 } # tls config tls keypair vault.example.com tls { no tlsv1.0, ciphers HIGH } # allow websockets http websockets } # relay definition for vaultwarden - forward inbound 443 tls on the egress interface to rocket on default port 8000 relay vaultwarden-https-relay { listen on egress port 443 tls protocol vaultwarden-https forward to <vaultwarden-default-host> port 8000 } ------------------------- dmesg: ------------------------- OpenBSD 7.4 (GENERIC.MP) #3: Wed Feb 28 06:23:33 MST 2024 r...@syspatch-74-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 4278042624 (4079MB) avail mem = 4128661504 (3937MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0 acpi0 at bios0: ACPI 3.0 acpi0: sleep states S3 S4 S5 acpi0: tables DSDT FACP APIC HPET MCFG WAET acpi0: wakeup devices acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: AMD EPYC-Milan Processor, 3250.37 MHz, 19-01-01 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,CLFLUSHOPT,CLWB,SHA,UMIP,PKU,IBRS,IBPB,SSBD,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVES cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line 8-way L2 cache, 32MB 64b/line 16-way L3 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 1000MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: AMD EPYC-Milan Processor, 3250.49 MHz, 19-01-01 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,CLFLUSHOPT,CLWB,SHA,UMIP,PKU,IBRS,IBPB,SSBD,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVES cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line 8-way L2 cache, 32MB 64b/line 16-way L3 cache cpu1: smt 1, core 0, package 0 ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins acpihpet0 at acpi0: 100000000 Hz acpimcfg0 at acpi0 acpimcfg0: addr 0xb0000000, bus 0-255 acpiprt0 at acpi0: bus 0 (PCI0) "ACPI0006" at acpi0 not configured acpipci0 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000 "PNP0A06" at acpi0 not configured "PNP0A06" at acpi0 not configured "PNP0A06" at acpi0 not configured "QEMU0002" at acpi0 not configured acpicmos0 at acpi0 "ACPI0010" at acpi0 not configured acpicpu0 at acpi0: C1(@1 halt!) acpicpu1 at acpi0: C1(@1 halt!) pvbus0 at mainbus0: KVM pvclock0 at pvbus0 pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel 82G33 Host" rev 0x00 vga1 at pci0 dev 1 function 0 "Bochs VGA" rev 0x02 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) ppb0 at pci0 dev 2 function 0 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci1 at ppb0 bus 1 virtio0 at pci1 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 vio0 at virtio0: address 56:00:04:d9:bf:ac virtio0: msix per-VQ ppb1 at pci0 dev 2 function 1 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci2 at ppb1 bus 2 ppb2 at pci2 dev 0 function 0 vendor "Red Hat", unknown product 0x000e rev 0x00 pci3 at ppb2 bus 3 "Intel 6300ESB WDT" rev 0x00 at pci3 dev 1 function 0 not configured ppb3 at pci0 dev 2 function 2 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci4 at ppb3 bus 4 xhci0 at pci4 dev 0 function 0 vendor "Red Hat", unknown product 0x000d rev 0x01: msix, xHCI 0.0 usb0 at xhci0: USB revision 3.0 uhub0 at usb0 configuration 1 interface 0 "Red Hat xHCI root hub" rev 3.00/1.00 addr 1 ppb4 at pci0 dev 2 function 3 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci5 at ppb4 bus 5 virtio1 at pci5 dev 0 function 0 "Qumranet Virtio 1.x Storage" rev 0x01 vioblk0 at virtio1 scsibus1 at vioblk0: 1 targets sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > sd0: 51200MB, 512 bytes/sector, 104857600 sectors virtio1: msix per-VQ ppb5 at pci0 dev 2 function 4 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci6 at ppb5 bus 6 virtio2 at pci6 dev 0 function 0 vendor "Qumranet", unknown product 0x1045 rev 0x01 viomb0 at virtio2 virtio2: apic 0 int 22 ppb6 at pci0 dev 2 function 5 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci7 at ppb6 bus 7 virtio3 at pci7 dev 0 function 0 "Qumranet Virtio 1.x RNG" rev 0x01 viornd0 at virtio3 virtio3: msix per-VQ ppb7 at pci0 dev 2 function 6 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci8 at ppb7 bus 8 ppb8 at pci0 dev 2 function 7 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci9 at ppb8 bus 9 ppb9 at pci0 dev 3 function 0 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci10 at ppb9 bus 10 ppb10 at pci0 dev 3 function 1 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci11 at ppb10 bus 11 ppb11 at pci0 dev 3 function 2 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci12 at ppb11 bus 12 ppb12 at pci0 dev 3 function 3 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci13 at ppb12 bus 13 ppb13 at pci0 dev 3 function 4 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci14 at ppb13 bus 14 ppb14 at pci0 dev 3 function 5 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci15 at ppb14 bus 15 ppb15 at pci0 dev 3 function 6 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci16 at ppb15 bus 16 ppb16 at pci0 dev 3 function 7 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci17 at ppb16 bus 17 ppb17 at pci0 dev 4 function 0 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 20 pci18 at ppb17 bus 18 ppb18 at pci0 dev 4 function 1 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 20 pci19 at ppb18 bus 19 azalia0 at pci0 dev 27 function 0 "Intel 82801I HD Audio" rev 0x03: msi azalia0: No codecs found pcib0 at pci0 dev 31 function 0 "Intel 82801IB LPC" rev 0x02 ahci0 at pci0 dev 31 function 2 "Intel 82801I AHCI" rev 0x02: msi, AHCI 1.0 ahci0: port 2: 1.5Gb/s scsibus2 at ahci0: 32 targets cd0 at scsibus2 targ 2 lun 0: <QEMU, QEMU DVD-ROM, 2.5+> removable ichiic0 at pci0 dev 31 function 3 "Intel 82801I SMBus" rev 0x02: apic 0 int 16 iic0 at ichiic0 isa0 at pcib0 isadma0 at isa0 pckbc0 at isa0 port 0x60/5 irq 1 irq 12 pckbd0 at pckbc0 (kbd slot) wskbd0 at pckbd0: console keyboard, using wsdisplay0 pms0 at pckbc0 (aux slot) wsmouse0 at pms0 mux 0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 uhidev0 at uhub0 port 5 configuration 1 interface 0 "QEMU QEMU USB Tablet" rev 2.00/0.00 addr 2 uhidev0: iclass 3/0 ums0 at uhidev0: 3 buttons, Z dir wsmouse1 at ums0 mux 0 vscsi0 at root scsibus3 at vscsi0: 256 targets softraid0 at root scsibus4 at softraid0: 256 targets root on sd0a (c7f9b5c386becdce.a) swap on sd0b dump on sd0b WARNING: clock gained 3 days WARNING: CHECK AND RESET THE DATE! syncing disks... done rebooting... OpenBSD 7.4 (GENERIC.MP) #3: Wed Feb 28 06:23:33 MST 2024 r...@syspatch-74-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 4278042624 (4079MB) avail mem = 4128665600 (3937MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0 acpi0 at bios0: ACPI 3.0 acpi0: sleep states S3 S4 S5 acpi0: tables DSDT FACP APIC HPET MCFG WAET acpi0: wakeup devices acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: AMD EPYC-Milan Processor, 3250.60 MHz, 19-01-01 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,CLFLUSHOPT,CLWB,SHA,UMIP,PKU,IBRS,IBPB,SSBD,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVES cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line 8-way L2 cache, 32MB 64b/line 16-way L3 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 1000MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: AMD EPYC-Milan Processor, 3250.64 MHz, 19-01-01 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,CLFLUSHOPT,CLWB,SHA,UMIP,PKU,IBRS,IBPB,SSBD,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVES cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line 8-way L2 cache, 32MB 64b/line 16-way L3 cache cpu1: smt 1, core 0, package 0 ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins acpihpet0 at acpi0: 100000000 Hz acpimcfg0 at acpi0 acpimcfg0: addr 0xb0000000, bus 0-255 acpiprt0 at acpi0: bus 0 (PCI0) "ACPI0006" at acpi0 not configured acpipci0 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000 "PNP0A06" at acpi0 not configured "PNP0A06" at acpi0 not configured "PNP0A06" at acpi0 not configured "QEMU0002" at acpi0 not configured acpicmos0 at acpi0 "ACPI0010" at acpi0 not configured acpicpu0 at acpi0: C1(@1 halt!) acpicpu1 at acpi0: C1(@1 halt!) pvbus0 at mainbus0: KVM pvclock0 at pvbus0 pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel 82G33 Host" rev 0x00 vga1 at pci0 dev 1 function 0 "Bochs VGA" rev 0x02 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) ppb0 at pci0 dev 2 function 0 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci1 at ppb0 bus 1 virtio0 at pci1 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 vio0 at virtio0: address 56:00:04:d9:bf:ac virtio0: msix per-VQ ppb1 at pci0 dev 2 function 1 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci2 at ppb1 bus 2 ppb2 at pci2 dev 0 function 0 vendor "Red Hat", unknown product 0x000e rev 0x00 pci3 at ppb2 bus 3 "Intel 6300ESB WDT" rev 0x00 at pci3 dev 1 function 0 not configured ppb3 at pci0 dev 2 function 2 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci4 at ppb3 bus 4 xhci0 at pci4 dev 0 function 0 vendor "Red Hat", unknown product 0x000d rev 0x01: msix, xHCI 0.0 usb0 at xhci0: USB revision 3.0 uhub0 at usb0 configuration 1 interface 0 "Red Hat xHCI root hub" rev 3.00/1.00 addr 1 ppb4 at pci0 dev 2 function 3 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci5 at ppb4 bus 5 virtio1 at pci5 dev 0 function 0 "Qumranet Virtio 1.x Storage" rev 0x01 vioblk0 at virtio1 scsibus1 at vioblk0: 1 targets sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > sd0: 51200MB, 512 bytes/sector, 104857600 sectors virtio1: msix per-VQ ppb5 at pci0 dev 2 function 4 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci6 at ppb5 bus 6 virtio2 at pci6 dev 0 function 0 vendor "Qumranet", unknown product 0x1045 rev 0x01 viomb0 at virtio2 virtio2: apic 0 int 22 ppb6 at pci0 dev 2 function 5 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci7 at ppb6 bus 7 virtio3 at pci7 dev 0 function 0 "Qumranet Virtio 1.x RNG" rev 0x01 viornd0 at virtio3 virtio3: msix per-VQ ppb7 at pci0 dev 2 function 6 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci8 at ppb7 bus 8 ppb8 at pci0 dev 2 function 7 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 22 pci9 at ppb8 bus 9 ppb9 at pci0 dev 3 function 0 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci10 at ppb9 bus 10 ppb10 at pci0 dev 3 function 1 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci11 at ppb10 bus 11 ppb11 at pci0 dev 3 function 2 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci12 at ppb11 bus 12 ppb12 at pci0 dev 3 function 3 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci13 at ppb12 bus 13 ppb13 at pci0 dev 3 function 4 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci14 at ppb13 bus 14 ppb14 at pci0 dev 3 function 5 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci15 at ppb14 bus 15 ppb15 at pci0 dev 3 function 6 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci16 at ppb15 bus 16 ppb16 at pci0 dev 3 function 7 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 23 pci17 at ppb16 bus 17 ppb17 at pci0 dev 4 function 0 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 20 pci18 at ppb17 bus 18 ppb18 at pci0 dev 4 function 1 vendor "Red Hat", unknown product 0x000c rev 0x00: apic 0 int 20 pci19 at ppb18 bus 19 azalia0 at pci0 dev 27 function 0 "Intel 82801I HD Audio" rev 0x03: msi azalia0: No codecs found pcib0 at pci0 dev 31 function 0 "Intel 82801IB LPC" rev 0x02 ahci0 at pci0 dev 31 function 2 "Intel 82801I AHCI" rev 0x02: msi, AHCI 1.0 ahci0: port 2: 1.5Gb/s scsibus2 at ahci0: 32 targets cd0 at scsibus2 targ 2 lun 0: <QEMU, QEMU DVD-ROM, 2.5+> removable ichiic0 at pci0 dev 31 function 3 "Intel 82801I SMBus" rev 0x02: apic 0 int 16 iic0 at ichiic0 isa0 at pcib0 isadma0 at isa0 pckbc0 at isa0 port 0x60/5 irq 1 irq 12 pckbd0 at pckbc0 (kbd slot) wskbd0 at pckbd0: console keyboard, using wsdisplay0 pms0 at pckbc0 (aux slot) wsmouse0 at pms0 mux 0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 uhidev0 at uhub0 port 5 configuration 1 interface 0 "QEMU QEMU USB Tablet" rev 2.00/0.00 addr 2 uhidev0: iclass 3/0 ums0 at uhidev0: 3 buttons, Z dir wsmouse1 at ums0 mux 0 vscsi0 at root scsibus3 at vscsi0: 256 targets softraid0 at root scsibus4 at softraid0: 256 targets root on sd0a (c7f9b5c386becdce.a) swap on sd0b dump on sd0b -------------------------
