simple-mtpfs kernel panic

[Christian Schulte]

Hi misc@,

during transferring some files from my laptop to my Android mobile
mounted using simple-mtpfs, the machine rebooted and /var/crash now
contains the following files:

$ ls -h /var/crash/
bounds  bsd.0   bsd.0.core  minfree

I read crash(8) but cannot get any meaningful information from gdb for
me. If someone is interested, I can send those files (amd64). It just
happened once and I cannot reproduce it.

$ dmesg -M /var/crash/bsd.0.core -N /var/crash/bsd.0
ugen2 at uhub0 port 2 "LGE LG-E610v" rev 2.00/2.33 addr 2
fusefs: libfuse vnode reclaim failed
panic: ehci_device_clear_toggle: queue active
Starting stack trace...
panic() at panic+0x10b
ehci_device_clear_toggle() at ehci_device_clear_toggle+0x2b
usbd_clear_endpoint_stall() at usbd_clear_endpoint_stall+0x24
ugen_do_write() at ugen_do_write+0x175
ugenwrite() at ugenwrite+0x48
spec_write() at spec_write+0xbb
VOP_WRITE() at VOP_WRITE+0x3f
vn_write() at vn_write+0x98
dofilewritev() at dofilewritev+0x205
sys_write() at sys_write+0x89
syscall() at syscall+0x2b8
--- syscall (number 4) ---
end of kernel
end trace frame: 0x1, count: 246
0x5c7f4107c9a:
End of stack trace.
syncing disks... 4 done

Regards,
-- 
Christian

Question about i386 COPTS during 'make release'.

[Christian Schulte]

Hello misc@,

are the i386 GENERIC and GENERIC.MP kernels built using '-O2' as is
setup in '/usr/src/sys/arch/i386/conf/Makefile.i386' or is COPTS set to
something else in '/etc/mk.conf' during 'make release' ?

Regards,
-- 
Christian Schulte

Re: Question about i386 COPTS during 'make release'.

[Christian Schulte]

Am 11/09/13 05:44, schrieb Philip Guenther:
> On Fri, Nov 8, 2013 at 7:30 PM, Christian Schulte  wrote:
>> are the i386 GENERIC and GENERIC.MP kernels built using '-O2' as is
>> setup in '/usr/src/sys/arch/i386/conf/Makefile.i386' or is COPTS set to
>> something else in '/etc/mk.conf' during 'make release' ?
> 
> COPTS is overridden when building the ramdisk kernels, but it's left
> alone for the normal GENERIC/GENERIC.* kernels.
> 
> So yes, if your /etc/mk.conf sets COPTS, it'll affect them.
> 
> 
> Philip Guenther
> 

Ok. Reason I am asking is this:

$ cc bcmp.c
$ time ./a.out
0m17.83s real 0m16.92s user 0m0.87s system
$ cc -O2 bcmp.c
$ time ./a.out
1m0.98s real 1m0.17s user 0m0.87s system
$ cc memcmp.c
$ time ./a.out
0m17.41s real 0m16.56s user 0m0.87s system
$ cc -O2 memcmp.c
$ time ./a.out
1m1.03s real 1m0.18s user 0m0.88s system

The difference comes from GCC optimizing away calls to the libc assembly
versions of bcmp/memcmp. For 'len' values greater than or equal to 8
(change BSIZ below), those assembly versions perform way better than
'cmpsb' inlined by GCC so that the following may be useful as most of
the bcmp/memcmp calls are using 'len' values greater than 8 and the
added function call overhead due to GCC no longer inlining these
functions should not be an issue.


Index: sys/arch/i386/conf/Makefile.i386
===
RCS file: /cvs/src/sys/arch/i386/conf/Makefile.i386,v
retrieving revision 1.88
diff -u -p -r1.88 Makefile.i386
--- sys/arch/i386/conf/Makefile.i38615 Oct 2013 19:23:27 -  1.88
+++ sys/arch/i386/conf/Makefile.i3869 Nov 2013 04:55:54 -
@@ -30,7 +30,10 @@ CWARNFLAGS=  -Werror -Wall -Wstrict-proto
 CMACHFLAGS=
 CMACHFLAGS+=   -fno-builtin-printf -fno-builtin-snprintf \
-fno-builtin-vsnprintf -fno-builtin-log \
-   -fno-builtin-log2 -fno-builtin-malloc ${NOPIE_FLAGS}
+   -fno-builtin-log2 -fno-builtin-malloc \
+   -fno-builtin-bcmp -fno-builtin-memcmp \
+   ${NOPIE_FLAGS}
+
 .if ${IDENT:M-DNO_PROPOLICE}
 CMACHFLAGS+=   -fno-stack-protector
 .endif


--
bcmp.c:

#include 
#include 
#include 
#include 
#include 

#define BSIZ(256 * 1024 * 1024)
#define VALUE (0xff)
#define ITERATIONS (100)

int
main(int argc, char *argv[])
{
void   *b1, *b2;
int i;

b1 = malloc(BSIZ);
if (b1 == NULL) {
fprintf(stderr, "unable to allocate memory: %s\n",
strerror(errno));
return errno;
}
if (mlock(b1, BSIZ)) {
fprintf(stderr, "unable to lock memory: %s\n",
strerror(errno));
return errno;
}
memset(b1, VALUE, BSIZ);

b2 = malloc(BSIZ);
if (b2 == NULL) {
fprintf(stderr, "unable to allocate memory: %s\n",
strerror(errno));
return errno;
}
if (mlock(b2, BSIZ)) {
fprintf(stderr, "unable to lock memory: %s\n",
strerror(errno));
return errno;
}
memset(b2, VALUE, BSIZ);

for (i = 0; i < ITERATIONS; i++) {
fprintf(stdout, "bcmp(b1,b2,%d)\t%d\t%d\n", BSIZ, i,
bcmp(b1, b2, BSIZ));
}

if (munlock(b1, BSIZ)) {
fprintf(stderr, "unable to unlock memory: %s\n",
strerror(errno));
}
if (munlock(b2, BSIZ)) {
fprintf(stderr, "unable to unlock memory: %s\n",
strerror(errno));
}
free(b1);
free(b2);

return 0;
}

--
memcmp.c:

#include 
#include 
#include 
#include 
#include 

#define BSIZ(256 * 1024 * 1024)
#define VALUE (0xff)
#define ITERATIONS (100)

int
main(int argc, char *argv[])
{
void   *b1, *b2;
int i;

b1 = malloc(BSIZ);
if (b1 == NULL) {
fprintf(stderr, "unable to allocate memory: %s\n",
strerror(errno));
return errno;
}
if (mlock(b1, BSIZ)) {
fprintf(stderr, "unable to lock memory: %s\n",
strerror(errno));
return errno;
}
memset(b1, VALUE, BSIZ);

b2 = malloc(BSIZ);
if (b2 == NULL) {
fprintf(stderr, "unable to allocate memory: %s\n",
strerror(errno));
return errno;
}
if (mlock(b2, BSIZ)) {
fprintf(stderr, "unable to lock memory: %s\n",
   

Re: Question about i386 COPTS during 'make release'.

[Christian Schulte]

Am 11/09/13 22:21, schrieb Philip Guenther:
> 
> Sorry, but I don't really find your tests convincing.
> 
> * Only test the worst case of a matching buffer.
> * Unreasonably large example used (are there *any* 256MB memcmp or
> bcmp in the kernel?)
> * Use of fprintf in the inner loop adds large fixed costs including
> syscalls to what should be a microbenchmark.
> * Measurements aren't of just the inner loop.
> * No test showing the suggested compiler options actually have the
> suggested effect.
> 
> If you want to show that changing A will have a positive effect, you
> need to have your test be as close a simulation of A as you can.  This
> doesn't seem to be that.
> 

All valid points. I compared things using 'objdump -d' already. As soon
as GCC is told to optimize, it will inline bcmp/memcmp using 'repz
cmpsb' (see '/usr/src/gnu/gcc/gcc/config/i386/i386.md' lines 18622ff and
function 'expand_builtin_memcmp' in file
'/usr/src/gnu/gcc/gcc/builtins.c'). This is slower even when comparing
just a few bytes. The larger the number of bytes to compare gets, the
more significant the difference becomes. See last result for just 128
bytes (0m29.58s vs. 0m5.32s).

$ cc -DBSIZ=4 -DITERATIONS=10 -O2 bcmp.c

0m23.54s real 0m23.24s user 0m0.00s system

$ cc -DBSIZ=4 -DITERATIONS=10 -O2 -fno-builtin-bcmp
-fno-builtin-memcmp bcmp.c

0m18.79s real 0m18.76s user 0m0.00s system

$ cc -DBSIZ=8 -DITERATIONS=10 -O2 bcmp.c

0m32.46s real 0m32.15s user 0m0.00s system

$ cc -DBSIZ=8 -DITERATIONS=10 -O2 -fno-builtin-bcmp
-fno-builtin-memcmp bcmp.c

0m20.03s real 0m20.00s user 0m0.00s system

$ cc -DBSIZ=16 -DITERATIONS=10 -O2 bcmp.c

0m49.81s real 0m49.78s user 0m0.00s system

$ cc -DBSIZ=16 -DITERATIONS=10 -O2 -fno-builtin-bcmp
-fno-builtin-memcmp bcmp.c

0m22.62s real 0m22.57s user 0m0.00s system

$ cc -DBSIZ=128 -DITERATIONS=1 -O2 bcmp.c

0m29.66s real 0m29.58s user 0m0.00s system

$ cc -DBSIZ=128 -DITERATIONS=1 -O2 -fno-builtin-bcmp
-fno-builtin-memcmp bcmp.c

0m5.33s real 0m5.32s user 0m0.00s system


$ cat bcmp.c
#include 
#include 
#include 
#include 
#include 

#define VALUE (0xff)

int
main(int argc, char *argv[])
{
void   *b1, *b2;
int i;

b1 = malloc(BSIZ);
if (b1 == NULL) {
fprintf(stderr, "unable to allocate memory: %s\n",
strerror(errno));
return errno;
}
if (mlock(b1, BSIZ)) {
fprintf(stderr, "unable to lock memory: %s\n",
strerror(errno));
return errno;
}
memset(b1, VALUE, BSIZ);

b2 = malloc(BSIZ);
if (b2 == NULL) {
fprintf(stderr, "unable to allocate memory: %s\n",
strerror(errno));
return errno;
}
if (mlock(b2, BSIZ)) {
fprintf(stderr, "unable to lock memory: %s\n",
strerror(errno));
return errno;
}
memset(b2, VALUE, BSIZ);

for (i = 0; i < ITERATIONS; i++) {
if (bcmp(b1, b2, BSIZ)) {
fprintf(stderr, "buffers do not match\n");
}
}

if (munlock(b1, BSIZ)) {
fprintf(stderr, "unable to unlock memory: %s\n",
strerror(errno));
}
if (munlock(b2, BSIZ)) {
fprintf(stderr, "unable to unlock memory: %s\n",
strerror(errno));
}
free(b1);
free(b2);

return 0;
}

Re: Advice on Security Cameras

[Christian Schulte]

Am 01.01.2019 um 18:46 schrieb Elias M. Mariani:
> Hi list,
> I'm thinking in installing some cameras in my private home, I have
> been looking for solutions, my concern is that I wish to be able to
> look the videos from outside the house and I'm a little paranoid about
> the quality of the software that the different vendors use. I have
> seen clusters of camaras that only work over ActiveX...

I would just pay attention to the cameras to support ONVIF. Most do.
Rest is a matter of using software supporting that and your specific use
cases.

Regards,
-- 
Christian

SSL/TLS troubleshooting

[Christian Schulte]

Hello,

I am facing an issue accessing an SSL/TLS webserver from OpenBSD. I have
another box not running OpenBSD connected to the same router and that
box can connect to that server flawlessly. I already tried to
troubleshoot things with the administrator of that system without
success. Is there something I need to be aware of accessing SSL/TLS
(https) servers with OpenBSD? How can I capture information I can send
to the administrator of that system to help him/her find out what is
special about connections coming from OpenBSD?

Regards,
-- 
Christian

Re: SSL/TLS troubleshooting

[Christian Schulte]

Am 12/10/16 um 21:43 schrieb Kai:
> Am 10. Dezember 2016 21:35:04 MEZ, schrieb Christian Schulte 
> :
>> Hello,
>>
>> I am facing an issue accessing an SSL/TLS webserver from OpenBSD. I
>> have
>> another box not running OpenBSD connected to the same router and that
>> box can connect to that server flawlessly. I already tried to
>> troubleshoot things with the administrator of that system without
>> success. Is there something I need to be aware of accessing SSL/TLS
>> (https) servers with OpenBSD? How can I capture information I can send
>> to the administrator of that system to help him/her find out what is
>> special about connections coming from OpenBSD?
>>
>> Regards,
> 
> I doubt this is special to OpenBSD. But you don't give any information to pin
> point this. What error do you get? 
> What tls version does the server offer?

It's <https://repository.apache.org/>

Operation timed out. Connections are very slow. Too slow so that they
time out. Does not happen using that other box ever. So there is a
difference accessing that server from OpenBSD (tested with Java, Firefox
and Chromium) and from that other box. I am not having any issues
accessing other servers. I created a ticket with them already.

<https://issues.apache.org/jira/browse/INFRA-13074>

> What version is supported by your installation?
> 
> Are you using a current version of OpenBSD?

$ uname -a
OpenBSD t60.schulte.it 6.0 1KHZ.MP#7 amd64

Thanks,
-- 
Christian

Re: SSL/TLS troubleshooting

[Christian Schulte]

Am 12/10/16 um 23:28 schrieb Karel Gardas:
> On Sat, Dec 10, 2016 at 10:56 PM, Christian Schulte  wrote:
>> It's <https://repository.apache.org/>
>>
>> Operation timed out. Connections are very slow. Too slow so that they
> 
> Not sure about the issue, but I've seen that last night too. Generally
> speaking there were too high number of lost packets which made TCP
> slow to crawl or broken. It stayed around 2-3 hours and then suddenly
> resolved. And I was accessing this from Ubuntu 16.04.1 if that matters
> 

Never disappears here. I'd really like to know why I can access that
without any issue using Windows 10 but start running into issues when
using OpenBSD. I also doubt this is affecting OpenBSD users only.

Regards,
-- 
Christian

Re: SSL/TLS troubleshooting

[Christian Schulte]

Am 12/10/16 um 22:57 schrieb Peter Hessler:
> On 2016 Dec 10 (Sat) at 22:56:05 +0100 (+0100), Christian Schulte wrote:
> :$ uname -a
> :OpenBSD t60.schulte.it 6.0 1KHZ.MP#7 amd64
> 
> You broke it.  Please use a GENERIC kernel, and it will work as normal.
> 

This is the configuration in use. Do you really think that HZ=1000 is
causing this? Will give GENERIC.MP a try, of course.

$cat 1KHZ.MP
#   $OpenBSD: GENERIC.MP,v 1.10 2008/12/22 16:35:28 deraadt Exp $

include "arch/amd64/conf/GENERIC"

option  MULTIPROCESSOR
option  HZ=1000
option  BUFCACHEPERCENT=5
rmoptionPOOL_DEBUG
makeoptions DEBUG="-g"
cpu*at mainbus?

Thanks,
-- 
Christian

Re: SSL/TLS troubleshooting

[Christian Schulte]

Am 12/10/16 um 22:57 schrieb Peter Hessler:
> On 2016 Dec 10 (Sat) at 22:56:05 +0100 (+0100), Christian Schulte wrote:
> :$ uname -a
> :OpenBSD t60.schulte.it 6.0 1KHZ.MP#7 amd64
> 
> You broke it.  Please use a GENERIC kernel, and it will work as normal.
> 

This is what I did using a recent source tree:

$ cd /usr/src/sys/arch/amd64/conf
$ config GENERIC.MP
$ cd /usr/src/sys/arch/amd64/compile/GENERIC.MP
$ make
$ make install
$ reboot

$ uname -a
OpenBSD t60.schulte.it 6.0 GENERIC.MP#2 amd64

$ cd /usr/src/lib/libssl
$ make clean
$ make obj
$ make depend
$ make
$ make install
$ cd /usr/src/lib/libcrypto
$ make clean
$ make obj
$ make depend
$ make
$ make install

This does not solve the issue, sadly.

Regards,
-- 
Christian

Re: mprotect W^X violation and JDK

[Christian Schulte]

Am 01/28/17 um 10:04 schrieb Alex McWhirter:
> Java doesn't work with write xor execute and this is the kernels way of
> letting you know. Java still runs because the partition is mounted with
> wxallowed, but the kernel still prints the error to let you know that
> Java isn't respecting a security feature.
> 

What should the VM do instead? It allocates memory, JIT compiles
bytecode to machinecode and then executes that machinecode. Should it
mprotect the memory after generating the machinecode? It would still
execute code from memory it could write to.

Regards,
-- 
Christian

Mouse pointer moving unintentionally.

[Christian Schulte]

Hello,

after upgrading to 5.6, I am experiencing a mouse pointer weirdness.
The X window manager (windowmaker) stops responding to window related
button presses. Switching to the console and back to X (CTRL-F1
followed by CTRL-F5), the window manager starts working again, but the
moise pointer now is slowly moving to the lower left corner of the
screen. I need to reboot to make the mouse pointer stand still again.
Disabling the kernel 'pms' driver makes this issue go away. This happens
with a T60 laptop. dmesg with the 'pms' driver disabled is attached.

Regards,
-- 
Christian Schulte


OpenBSD 5.6-stable (1KHZ.MP) #7: Sun Nov 30 15:45:48 CET 2014
r...@t60.schulte.it:/usr/src/sys/arch/amd64/compile/1KHZ.MP
real mem = 3203203072 (3054MB)
avail mem = 3109216256 (2965MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xe0010 (68 entries)
bios0: vendor LENOVO version "79ETD4WW (2.14 )" date 06/07/2007
bios0: LENOVO 2007FVG
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT ECDT TCPA APIC MCFG HPET BOOT SSDT SSDT SSDT SSDT
acpi0: wakeup devices LID_(S3) SLPB(S3) LURT(S3) DURT(S3) EXP0(S4) EXP1(S4) 
EXP2(S4) EXP3(S4) PCI1(S4) USB0(S3) USB1(S3) USB2(S3) USB7(S3) HDEF(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiec0 at acpi0
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz, 1828.91 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF
cpu0: 2MB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 166MHz
cpu0: mwait min=64, max=64, C-substates=0.2.2.2.2, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz, 1828.76 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF
cpu1: 2MB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec0, version 20, 24 pins
ioapic0: misconfigured as apic 2, remapped to apid 1
acpimcfg0 at acpi0 addr 0xf000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (AGP_)
acpiprt2 at acpi0: bus 2 (EXP0)
acpiprt3 at acpi0: bus 3 (EXP1)
acpiprt4 at acpi0: bus 4 (EXP2)
acpiprt5 at acpi0: bus 12 (EXP3)
acpiprt6 at acpi0: bus 21 (PCI1)
acpicpu0 at acpi0: C3, C2, C1, PSS
acpicpu1 at acpi0: C3, C2, C1, PSS
acpipwrres0 at acpi0: PUBS, resource for USB0, USB2, USB7
acpitz0 at acpi0: critical temperature is 127 degC
acpitz1 at acpi0: critical temperature is 99 degC
acpibtn0 at acpi0: LID_
acpibtn1 at acpi0: SLPB
acpibat0 at acpi0: BAT0 model "92P1139" serial  9496 type LION oem "Panasonic"
acpibat1 at acpi0: BAT1 not present
acpiac0 at acpi0: AC unit online
acpithinkpad0 at acpi0
acpidock0 at acpi0: GDCK not docked (0)
cpu0: Enhanced SpeedStep 1828 MHz: speeds: 1833, 1333, 1000 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82945GM Host" rev 0x03
ppb0 at pci0 dev 1 function 0 "Intel 82945GM PCIE" rev 0x03: msi
pci1 at ppb0 bus 1
radeondrm0 at pci1 dev 0 function 0 "ATI Radeon Mobility X1400" rev 0x00
drm0 at radeondrm0
radeondrm0: apic 1 int 16
azalia0 at pci0 dev 27 function 0 "Intel 82801GB HD Audio" rev 0x02: msi
azalia0: codecs: Analog Devices AD1981HD, Conexant/0x2bfa, using Analog Devices 
AD1981HD
audio0 at azalia0
ppb1 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x02: msi
pci2 at ppb1 bus 2
em0 at pci2 dev 0 function 0 "Intel 82573L" rev 0x00: msi, address 
00:15:58:7c:c0:6c
ppb2 at pci0 dev 28 function 1 "Intel 82801GB PCIE" rev 0x02: msi
pci3 at ppb2 bus 3
athn0 at pci3 dev 0 function 0 "Atheros AR5418" rev 0x01: apic 1 int 17
athn0: MAC AR5418 rev 2, RF AR5133 (2T3R), ROM rev 3, address 00:16:cf:a9:e0:c7
ppb3 at pci0 dev 28 function 2 "Intel 82801GB PCIE" rev 0x02: msi
pci4 at ppb3 bus 4
ppb4 at pci0 dev 28 function 3 "Intel 82801GB PCIE" rev 0x02: msi
pci5 at ppb4 bus 12
uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x02: apic 1 int 16
uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x02: apic 1 int 17
uhci2 at pci0 dev 29 function 2 "Intel 82801GB USB" rev 0x02: apic 1 int 18
uhci3 at pci0 dev 29 function 3 "Intel 82801GB USB" rev 0x02: apic 1 int 19
ehci0 at pci0 dev 29 function 7 "Intel 82801GB USB" rev 0x02: apic 1 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb5 at pci0 dev 30 func

em0 interface hangs in 5.7 (was: Re: Semi-reproduceable em0 network hangs in new snap)

[Christian Schulte]

Hi @misc,

after upgrading my T60 from 5.6-stable to 5.7-stable, I am experiencing the
same issue described above. Should this be filed using sendbug ? I am
currently reverting to the athn0 interface which is working fine. The em0
interface hangs after some minutes or even seconds and isn't usable as of 5.7.

Regards,
--
Christian

Mouse setup question.

[Christian Schulte]

Hi misc@,

I have a question regarding the setup of mice for the following
system. Should wsmoused(8) be enabled or disabled? When enabled,
only the synaptics touchpad is working with X. When disabled,
all mice are working, but all of a sudden, the mouse pointer
starts moving without any mouse beeing in use. A reboot is
needed to get the mouse working again. Following is dmesg
and the X logfile with wsmoused(8) enabled and disabled.
Please note the following lines when wsmoused(8) is enabled:

[  1105.998] (EE) xf86OpenSerial: Cannot open device /dev/wsmouse^M
Ger\xc3\xa4t ist belegt.^M
[  1105.998] (EE) ws: /dev/wsmouse: cannot open input device^M
[  1105.998] (EE) PreInit returned 2 for "/dev/wsmouse"^M
[  1105.998] (II) UnloadModule: "ws"^M

Regards,
Christian

Script started on Fri May 15 22:27:29 2015
$ dmesg
OpenBSD 5.7-stable (1KHZ.MP) #1: Sun May  3 19:25:27 CEST 2015
r...@t60.schulte.it:/usr/src/sys/arch/amd64/compile/1KHZ.MP
real mem = 3203203072 (3054MB)
avail mem = 3114078208 (2969MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xe0010 (68 entries)
bios0: vendor LENOVO version "79ETD4WW (2.14 )" date 06/07/2007
bios0: LENOVO 2007FVG
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT ECDT TCPA APIC MCFG HPET BOOT SSDT SSDT SSDT SSDT
acpi0: wakeup devices LID_(S3) SLPB(S3) LURT(S3) DURT(S3) EXP0(S4) EXP1(S4) 
EXP2(S4) EXP3(S4) PCI1(S4) USB0(S3) USB1(S3) USB2(S3) USB7(S3) HDEF(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiec0 at acpi0
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz, 1828.90 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF
cpu0: 2MB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 166MHz
cpu0: mwait min=64, max=64, C-substates=0.2.2.2.2, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz, 1828.75 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF
cpu1: 2MB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec0, version 20, 24 pins
ioapic0: misconfigured as apic 2, remapped to apid 1
acpimcfg0 at acpi0 addr 0xf000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (AGP_)
acpiprt2 at acpi0: bus 2 (EXP0)
acpiprt3 at acpi0: bus 3 (EXP1)
acpiprt4 at acpi0: bus 4 (EXP2)
acpiprt5 at acpi0: bus 12 (EXP3)
acpiprt6 at acpi0: bus 21 (PCI1)
acpicpu0 at acpi0: C3, C2, C1, PSS
acpicpu1 at acpi0: C3, C2, C1, PSS
acpipwrres0 at acpi0: PUBS, resource for USB0, USB2, USB7
acpitz0 at acpi0: critical temperature is 127 degC
acpitz1 at acpi0: critical temperature is 99 degC
acpibtn0 at acpi0: LID_
acpibtn1 at acpi0: SLPB
acpibat0 at acpi0: BAT0 model "92P1139" serial  9496 type LION oem "Panasonic"
acpibat1 at acpi0: BAT1 not present
acpiac0 at acpi0: AC unit online
acpithinkpad0 at acpi0
acpidock0 at acpi0: GDCK not docked (0)
cpu0: Enhanced SpeedStep 1828 MHz: speeds: 1833, 1333, 1000 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82945GM Host" rev 0x03
ppb0 at pci0 dev 1 function 0 "Intel 82945GM PCIE" rev 0x03: msi
pci1 at ppb0 bus 1
radeondrm0 at pci1 dev 0 function 0 "ATI Radeon Mobility X1400" rev 0x00
drm0 at radeondrm0
radeondrm0: apic 1 int 16
azalia0 at pci0 dev 27 function 0 "Intel 82801GB HD Audio" rev 0x02: msi
azalia0: codecs: Analog Devices AD1981HD, Conexant/0x2bfa, using Analog Devices 
AD1981HD
audio0 at azalia0
ppb1 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x02: msi
pci2 at ppb1 bus 2
em0 at pci2 dev 0 function 0 "Intel 82573L" rev 0x00: msi, address 
00:15:58:7c:c0:6c
ppb2 at pci0 dev 28 function 1 "Intel 82801GB PCIE" rev 0x02: msi
pci3 at ppb2 bus 3
athn0 at pci3 dev 0 function 0 "Atheros AR5418" rev 0x01: apic 1 int 17
athn0: MAC AR5418 rev 2, RF AR5133 (2T3R), ROM rev 3, address 00:16:cf:a9:e0:c7
ppb3 at pci0 dev 28 function 2 "Intel 82801GB PCIE" rev 0x02: msi
pci4 at ppb3 bus 4
ppb4 at pci0 dev 28 function 3 "Intel 82801GB PCIE" rev 0x02: msi
pci5 at ppb4 bus 12
uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x02: apic 1 int 16
uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x02: apic 1 int 17
uhci2 at pci0 dev 29 function 2 "Intel 82801GB USB" rev 0x02: apic 1 int 18
uhci3 at pci0 dev 29 function 3 "Intel 82801GB USB" rev 0x02: apic 1 int 19
ehci0 at pci0 dev 29 function 7 "Intel 82801GB USB" rev 0x02: apic 1 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr

Re: Mouse setup question.

[Christian Schulte]

Here is some more information I can provide. When wsmoused(8) is not 
running and the mouse pointer starts moving to the lower left corner of 
the screen without any mouse being touched, I can stop it by executing:



schu...@t60.schulte.it
2015-05-17T22:52:14+0200 Sunday 137
~
$ xinput --disable 8

So disabling the "ws" driver makes the mouse pointer stop moving (by 
disabling the USB mouse, that is, after executing that command, only the 
synaptics touchpad is working afterwards). Reenabling the USB mouse by 
executing


schu...@t60.schulte.it
2015-05-17T22:54:50+0200 Sunday 137
~
$ xinput --enable 8

the mouse pointer immediately starts moving to the lower left corner of 
the screen.


schu...@t60.schulte.it
2015-05-17T22:54:55+0200 Sunday 137
~
$ xinput --list
⎡ Virtual core pointer  id=2[master pointer  (3)]
⎜   ↳ Virtual core XTEST pointerid=4[slave  pointer  (2)]
⎜   ↳ /dev/wsmouse0 id=7[slave  pointer  (2)]
⎜   ↳ /dev/wsmouse  id=8[slave  pointer  (2)]
⎣ Virtual core keyboard id=3[master keyboard (2)]
↳ Virtual core XTEST keyboard   id=5[slave  keyboard (3)]
↳ /dev/wskbdid=6[slave  keyboard (3)]

Could someone please tell me how to make the "ws" driver produce any 
debugging information so that I can try to find a solution myself and 
maybe provide a patch? The issue started with 5.6 and 5.7 shows the same 
behaviour. It never happened using 5.5.


Regards,
--
Christian

Cannot get sound to work on a T60

[Christian Schulte]

Hello misc@,

please see the output of 'dmesg', 'audioctl -f /dev/audio' and 'mixerctl' 
included in this message. Is it correct that executing 'aucat -i something.wav' 
should produce audible output with things setup like this ?

Regards,
Christian

$ dmesg
OpenBSD 5.5-stable (1KHZ.MP) #18: Sun May  4 15:39:00 CEST 2014
r...@t60.schulte.it:/usr/src/sys/arch/amd64/compile/1KHZ.MP
real mem = 3203203072 (3054MB)
avail mem = 3109384192 (2965MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xe0010 (68 entries)
bios0: vendor LENOVO version "79ETD4WW (2.14 )" date 06/07/2007
bios0: LENOVO 2007FVG
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT ECDT TCPA APIC MCFG HPET BOOT SSDT SSDT SSDT SSDT
acpi0: wakeup devices LID_(S3) SLPB(S3) LURT(S3) DURT(S3) EXP0(S4) EXP1(S4) 
EXP2(S4) EXP3(S4) PCI1(S4) USB0(S3) USB1(S3) USB2(S3) USB7(S3) HDEF(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiec0 at acpi0
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz, 1828.87 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,LONG,LAHF,PERF
cpu0: 2MB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 166MHz
cpu0: mwait min=64, max=64, C-substates=0.2.2.2.2, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz, 1828.76 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,LONG,LAHF,PERF
cpu1: 2MB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec0, version 20, 24 pins
ioapic0: misconfigured as apic 2, remapped to apid 1
acpimcfg0 at acpi0 addr 0xf000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (AGP_)
acpiprt2 at acpi0: bus 2 (EXP0)
acpiprt3 at acpi0: bus 3 (EXP1)
acpiprt4 at acpi0: bus 4 (EXP2)
acpiprt5 at acpi0: bus 12 (EXP3)
acpiprt6 at acpi0: bus 21 (PCI1)
acpicpu0 at acpi0: C3, C2, C1, PSS
acpicpu1 at acpi0: C3, C2, C1, PSS
acpipwrres0 at acpi0: PUBS, resource for USB0, USB2, USB7
acpitz0 at acpi0: critical temperature is 127 degC
acpitz1 at acpi0: critical temperature is 99 degC
acpibtn0 at acpi0: LID_
acpibtn1 at acpi0: SLPB
acpibat0 at acpi0: BAT0 model "92P1139" serial  9496 type LION oem "Panasonic"
acpibat1 at acpi0: BAT1 not present
acpiac0 at acpi0: AC unit online
acpithinkpad0 at acpi0
acpidock0 at acpi0: GDCK not docked (0)
cpu0: Enhanced SpeedStep 1828 MHz: speeds: 1833, 1333, 1000 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82945GM Host" rev 0x03
ppb0 at pci0 dev 1 function 0 "Intel 82945GM PCIE" rev 0x03: msi
pci1 at ppb0 bus 1
radeondrm0 at pci1 dev 0 function 0 "ATI Radeon Mobility X1400" rev 0x00
drm0 at radeondrm0
radeondrm0: apic 1 int 16
azalia0 at pci0 dev 27 function 0 "Intel 82801GB HD Audio" rev 0x02: msi
azalia0: codecs: Analog Devices AD1981HD, Conexant/0x2bfa, using Analog Devices 
AD1981HD
audio0 at azalia0
ppb1 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x02: msi
pci2 at ppb1 bus 2
em0 at pci2 dev 0 function 0 "Intel 82573L" rev 0x00: msi, address 
00:15:58:7c:c0:6c
ppb2 at pci0 dev 28 function 1 "Intel 82801GB PCIE" rev 0x02: msi
pci3 at ppb2 bus 3
athn0 at pci3 dev 0 function 0 "Atheros AR5418" rev 0x01: apic 1 int 17
athn0: MAC AR5418 rev 2, RF AR5133 (2T3R), ROM rev 3, address 00:16:cf:a9:e0:c7
ppb3 at pci0 dev 28 function 2 "Intel 82801GB PCIE" rev 0x02: msi
pci4 at ppb3 bus 4
ppb4 at pci0 dev 28 function 3 "Intel 82801GB PCIE" rev 0x02: msi
pci5 at ppb4 bus 12
uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x02: apic 1 int 16
uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x02: apic 1 int 17
uhci2 at pci0 dev 29 function 2 "Intel 82801GB USB" rev 0x02: apic 1 int 18
uhci3 at pci0 dev 29 function 3 "Intel 82801GB USB" rev 0x02: apic 1 int 19
ehci0 at pci0 dev 29 function 7 "Intel 82801GB USB" rev 0x02: apic 1 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb5 at pci0 dev 30 function 0 "Intel 82801BAM Hub-to-PCI" rev 0xe2
pci6 at ppb5 bus 21
cbb0 at pci6 dev 0 function 0 "TI PCI1510 CardBus" rev 0x00: apic 1 int 16
cardslot0 at cbb0 slot 0 flags 0
cardbus0 at cardslot0: bus 22 device 0 cacheline 0x8, lattimer 0xb0
pcmcia0 at cardslot0
pcib0 at pci0 dev 31 function 0 "Intel 82801GBM LPC" rev 0x02
pciide0 at pci0 dev 31 function 1 "Intel 82801GB IDE" rev 0x02: DMA, channel 0 
configured to compatibility, channel 1 configured to compatibility
atapiscsi0 at pciide0 channel 0 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0:  ATAPI 5/cdrom 
removable
cd

Re: Cannot get sound to work on a T60

[Christian Schulte]

Am 05/25/14 04:21, schrieb Philip Guenther:
> On Sat, May 24, 2014 at 5:59 PM, Christian Schulte  wrote:
>>
>> please see the output of 'dmesg', 'audioctl -f /dev/audio' and 'mixerctl'
>> included in this message. Is it correct that executing 'aucat -i
>> something.wav' should produce audible output with things setup like this ?
>>
> 
> I don't have a .wav file on hand, but my T60 running i386 works Just Fine
> using mplayer.

Internal speakers or do you have any external speakers connected ?

> 
> 
> OpenBSD 5.5-stable (1KHZ.MP) #18: Sun May  4 15:39:00 CEST 2014
>> r...@t60.schulte.it:/usr/src/sys/arch/amd64/compile/1KHZ.MP
> 
> 
> Tried the GENERIC.MP kernel?
> 

Of course. Here is the kernel config I am using.

$ cat /usr/src/sys/arch/amd64/conf/1KHZ.MP
include "arch/amd64/conf/GENERIC"

option  MULTIPROCESSOR
option  HZ=1000
option  BUFCACHEPERCENT=5
rmoptionPOOL_DEBUG
makeoptions DEBUG="-g"
cpu*at mainbus?

-- 
Regargs
Christian

Re: Cannot get sound to work on a T60

[Christian Schulte]

Am 05/25/14 06:29, schrieb Philip Guenther:
> On Sat, May 24, 2014 at 9:03 PM, Christian Schulte  wrote:
> 
>> Am 05/25/14 04:21, schrieb Philip Guenther:
>>> On Sat, May 24, 2014 at 5:59 PM, Christian Schulte 
>> wrote:
>>>>
>>>> please see the output of 'dmesg', 'audioctl -f /dev/audio' and
>> 'mixerctl'
>>>> included in this message. Is it correct that executing 'aucat -i
>>>> something.wav' should produce audible output with things setup like
>> this ?
>>>
>>> I don't have a .wav file on hand, but my T60 running i386 works Just Fine
>>> using mplayer.
>>
>> Internal speakers or do you have any external speakers connected ?
>>
> 
> Works both with internal speakers and with ear buds plugged in.
> 
> So when did audio stop working on your box?
> 

Hard to tell. I have been working with an R60 for a long time (wouldn't
recommend these) and sound was working initially. After some update
sound stopped working there as well (-stable). Should be some 2 years
ago or so already. I have that R60 lying around and sound isn't working
there as well. Sound on the T60 I got a few weeks ago also does not
work. So I am asking. Having read the FAQ, tried this and that, it just
does not provide a single glitch. Still thinking this to be something
really stupid on my side - but - the R60 provided sound and after some
update stopped to do so. The T60 now also does not provide any sound. I
am lost somehow.

Regards,
Christian

Re: Cannot get sound to work on a T60

[Christian Schulte]

Am 05/25/14 06:49, schrieb patrick keshishian:
> On 5/24/14, Philip Guenther  wrote:
>> On Sat, May 24, 2014 at 9:03 PM, Christian Schulte  wrote:
>>
>>> Am 05/25/14 04:21, schrieb Philip Guenther:
>>>> On Sat, May 24, 2014 at 5:59 PM, Christian Schulte 
>>> wrote:
>>>>>
>>>>> please see the output of 'dmesg', 'audioctl -f /dev/audio' and
>>> 'mixerctl'
>>>>> included in this message. Is it correct that executing 'aucat -i
>>>>> something.wav' should produce audible output with things setup like
>>> this ?
>>>>
>>>> I don't have a .wav file on hand, but my T60 running i386 works Just
>>>> Fine
>>>> using mplayer.
>>>
>>> Internal speakers or do you have any external speakers connected ?
>>>
>>
>> Works both with internal speakers and with ear buds plugged in.
>>
>> So when did audio stop working on your box?
> 
> Does your audio not work completely? Does it work
> through headphones? Speakers?
> 

I have not plugged in anything external to the machine. I am just
heading after the internal speakers to provide something. Hmm. It does
not even beep currently.

-- 
Christian

Re: Cannot get sound to work on a T60

[Christian Schulte]

Am 05/25/14 09:39, schrieb patrick keshishian:
> On 5/25/14, Christian Schulte  wrote:
>> Am 05/25/14 06:49, schrieb patrick keshishian:
>>> On 5/24/14, Philip Guenther  wrote:
>>>> On Sat, May 24, 2014 at 9:03 PM, Christian Schulte 
>>>> wrote:
>>>>
>>>>> Am 05/25/14 04:21, schrieb Philip Guenther:
>>>>>> On Sat, May 24, 2014 at 5:59 PM, Christian Schulte 
>>>>> wrote:
>>>>>>>
>>>>>>> please see the output of 'dmesg', 'audioctl -f /dev/audio' and
>>>>> 'mixerctl'
>>>>>>> included in this message. Is it correct that executing 'aucat -i
>>>>>>> something.wav' should produce audible output with things setup like
>>>>> this ?
>>>>>>
>>>>>> I don't have a .wav file on hand, but my T60 running i386 works Just
>>>>>> Fine
>>>>>> using mplayer.
>>>>>
>>>>> Internal speakers or do you have any external speakers connected ?
>>>>>
>>>>
>>>> Works both with internal speakers and with ear buds plugged in.
>>>>
>>>> So when did audio stop working on your box?
>>>
>>> Does your audio not work completely? Does it work
>>> through headphones? Speakers?
>>>
>>
>> I have not plugged in anything external to the machine. I am just
>> heading after the internal speakers to provide something. Hmm. It does
>> not even beep currently.
> 
> I ask to see if it your issue is at all related/similar to mine:
> http://marc.info/?l=openbsd-misc&m=139317761904835&w=2
> 

Don't know. I get no error messages at all and the sample buffer is
increasing during playback as the FAQ suggests to look after. So
something is sending data to something. It just isn't audible.

-- 
Regargs,
Christian

Re: Cannot get sound to work on a T60

[Christian Schulte]

Am 05/25/14 10:27, schrieb Mihai Popescu:
>> Don't know. I get no error messages at all and the sample buffer is
>> increasing during playback as the FAQ suggests to look after. So
>> something is sending data to something. It just isn't audible.
> 
> Do you have some push buttons for volume up/down and mute on your keyboard
> or close to it?
> Try to push them, it may be on mute or very low volume.
> 

Indeed. Never tried those buttons during playback. Always looked at the
output of mixerctl and there nothing changes. It really was that simple.
Damn it.

Regards,
-- 
Christian

iwn0: fatal firmware error after sysupgrade to 7.1

[Christian Schulte]

Please CC me. I am not subscribed to the list.

Hello,

after upgrading from 7.0 to 7.1, the iwn0 interface cannot be used 
anymore, stating iwn0: fatal firmware error. I then checked out 7.1 
stable, build the GENERIC.MP kernel, the system and xenocara. Issue 
remains. Here's what dmesg contains after adding the "debug" option to 
/etc/hostname.iwn0.


Thanks for any help.

splassert: ieee80211_free_node: want 7 have 4
urtw0 detached
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: fatal firmware error
iwn0: SCAN -> AUTH
iwn0: sending auth to c8:0e:14:ea:23:83 on channel 13 mode 11g
iwn0: fatal firmware error
firmware error log:
  error type  = "SYSASSERT" (0x0005)
  program counter = 0x0001DFB0
  source line = 0x01FB
  error data  = 0x01FB0009
  branch link = 0x0001DF940001DF94
  interrupt link  = 0x0916
  time= 3432666
driver status:
  tx ring  0: qid=0  cur=1   queued=1
  tx ring  1: qid=1  cur=0   queued=0
  tx ring  2: qid=2  cur=0   queued=0
  tx ring  3: qid=3  cur=0   queued=0
  tx ring  4: qid=4  cur=30  queued=0
  tx ring  5: qid=5  cur=0   queued=0
  tx ring  6: qid=6  cur=0   queued=0
  tx ring  7: qid=7  cur=0   queued=0
  tx ring  8: qid=8  cur=0   queued=0
  tx ring  9: qid=9  cur=0   queued=0
  tx ring 10: qid=10 cur=0   queued=0
  tx ring 11: qid=11 cur=0   queued=0
  tx ring 12: qid=12 cur=0   queued=0
  tx ring 13: qid=13 cur=0   queued=0
  tx ring 14: qid=14 cur=0   queued=0
  tx ring 15: qid=15 cur=0   queued=0
  tx ring 16: qid=16 cur=0   queued=0
  tx ring 17: qid=17 cur=0   queued=0
  tx ring 18: qid=18 cur=0   queued=0
  tx ring 19: qid=19 cur=0   queued=0
  rx ring: cur=49
  802.11 state 2
iwn0: AUTH -> INIT
iwn0: begin active scan
iwn0: INIT -> SCAN

Re: iwn0: fatal firmware error after sysupgrade to 7.1

[Christian Schulte]

Please see attached dmesg and pcidump. There has been a similar issue 
with the if_iwm.c driver. Maybe this one is related.


Best regards,
--
Christian
OpenBSD 7.1-stable (GENERIC.MP) #3: Sun Jun 12 10:08:50 CEST 2022
schu...@x500.schulte.it:/sys/arch/amd64/compile/GENERIC.MP
real mem = 4165603328 (3972MB)
avail mem = 4022091776 (3835MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xe0010 (80 entries)
bios0: vendor LENOVO version "6FET93WW (3.23 )" date 10/12/2012
bios0: LENOVO 22434DG
acpi0 at bios0: ACPI 3.0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT ECDT APIC MCFG HPET SLIC BOOT ASF! SSDT TCPA DMAR 
SSDT SSDT SSDT
acpi0: wakeup devices LID_(S3) SLPB(S3) UART(S3) IGBE(S4) EXP0(S4) EXP1(S4) 
EXP2(S4) EXP3(S4) EXP4(S4) PCI1(S4) USB0(S3) USB3(S3) USB5(S3) EHC0(S3) 
EHC1(S3) HDEF(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiec0 at acpi0
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM)2 Duo CPU P8400 @ 2.26GHz, 2261.32 MHz, 06-17-06
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,NXE,LONG,LAHF,PERF,SENSOR,MELTDOWN
cpu0: 3MB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 7 var ranges, 88 fixed ranges
cpu0: apic clock running at 266MHz
cpu0: mwait min=64, max=64, C-substates=0.2.2.2.2.1.3, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM)2 Duo CPU P8400 @ 2.26GHz, 2261.01 MHz, 06-17-06
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,NXE,LONG,LAHF,PERF,SENSOR,MELTDOWN
cpu1: 3MB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec0, version 20, 24 pins, remapped
acpimcfg0 at acpi0
acpimcfg0: addr 0xe000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (AGP_)
acpiprt2 at acpi0: bus 2 (EXP0)
acpiprt3 at acpi0: bus 3 (EXP1)
acpiprt4 at acpi0: bus -1 (EXP2)
acpiprt5 at acpi0: bus 5 (EXP3)
acpiprt6 at acpi0: bus 13 (EXP4)
acpiprt7 at acpi0: bus 21 (PCI1)
acpibtn0 at acpi0: LID_
acpibtn1 at acpi0: SLPB
acpipci0 at acpi0 PCI0: 0x 0x0011 0x0001
acpicmos0 at acpi0
com0 at acpi0 UART addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
tpm0 at acpi0 TPM_ 1.2 (TIS) addr 0xfed4/0x5000, device 0x10208086 rev 0x6
acpibat0 at acpi0: BAT0 not present
acpiac0 at acpi0: AC unit online
acpithinkpad0 at acpi0: version 1.0
"IBM0079" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpicpu0 at acpi0: !C3(250@17 mwait.3@0x20), !C2(500@1 mwait.1@0x10), C1(1000@1 
mwait.1), PSS
acpicpu1 at acpi0: !C3(250@17 mwait.3@0x20), !C2(500@1 mwait.1@0x10), C1(1000@1 
mwait.1), PSS
acpipwrres0 at acpi0: PUBS, resource for USB0, USB3, USB5, EHC0, EHC1
acpitz0 at acpi0: critical temperature is 127 degC
acpitz1 at acpi0: critical temperature is 100 degC
acpidock0 at acpi0: GDCK docked (15)
acpivideo0 at acpi0: VID_
acpivout0 at acpivideo0: LCD0
acpivideo1 at acpi0: VID_
cpu0: Enhanced SpeedStep 2261 MHz: speeds: 2267, 2266, 1600, 800 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel GM45 Host" rev 0x07
inteldrm0 at pci0 dev 2 function 0 "Intel GM45 Video" rev 0x07
drm0 at inteldrm0
intagp0 at inteldrm0
agp0 at intagp0: aperture at 0xd000, size 0x1000
inteldrm0: apic 1 int 16, GM45, gen 4
"Intel GM45 Video" rev 0x07 at pci0 dev 2 function 1 not configured
"Intel GM45 HECI" rev 0x07 at pci0 dev 3 function 0 not configured
puc0 at pci0 dev 3 function 3 "Intel GM45 KT" rev 0x07: ports: 16 com
com4 at puc0 port 0 apic 1 int 17: ns16550a, 16 byte fifo
com4: probed fifo depth: 0 bytes
em0 at pci0 dev 25 function 0 "Intel ICH9 IGP M AMT" rev 0x03: msi, address 
00:1c:25:95:55:a3
uhci0 at pci0 dev 26 function 0 "Intel 82801I USB" rev 0x03: apic 1 int 20
uhci1 at pci0 dev 26 function 1 "Intel 82801I USB" rev 0x03: apic 1 int 21
uhci2 at pci0 dev 26 function 2 "Intel 82801I USB" rev 0x03: apic 1 int 22
ehci0 at pci0 dev 26 function 7 "Intel 82801I USB" rev 0x03: apic 1 int 23
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 
addr 1
azalia0 at pci0 dev 27 function 0 "Intel 82801I HD Audio" rev 0x03: msi
azalia0: codecs: Conexant CX20561
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 "Intel 82801I PCIE" rev 0x03: msi
pci1 at ppb0 bus 2
ppb1 at pci0 dev 28 function 1 "Intel 82801I PCIE" rev 0x03: msi
pci2 at ppb1 bus 3
iwn0 at pci2 dev 0 function 0 "Intel WiFi Link 5300" rev 0x00: msi, MIMO 3T3R, 
MoW, address 00:21:6a:0e:02:58
ppb2 at pci0 dev 28 function 3 "Intel 82801I PCIE" rev 0x03: msi
pci3 at ppb2 bus 5
ppb3 at pci0 dev 28

Re: iwn0: fatal firmware error after sysupgrade to 7.1

[Christian Schulte]

On 12.06.22 13:22, Stefan Sperling wrote:

On Sun, Jun 12, 2022 at 01:16:00PM +0200, Stefan Sperling wrote:

On Sun, Jun 12, 2022 at 10:28:33AM +0200, Christian Schulte wrote:

Please see attached dmesg and pcidump. There has been a similar issue with
the if_iwm.c driver. Maybe this one is related.


These are seperate drivers so it is unlikely that these issues would
be related.

How many different access points have you tried to reproduce the
problem with so far?


Two. The router and a repeater bridging the WLAN of that router. There 
are 3 more repeaters I could test this with. Cannot be related to those. 
No issues with 7.0.




In any case we will need to figure out which command is failing for you.

The patch below adds additional debug output while 'ifconfig iwn0 debug'
is enabled and should display the command code sent to firmware. Hopefully
this can tell us which command firmware is complaining about.
Can you reproduce the failure with it again and send new debug output?


See attached dmesg and ls /etc/firmware. When running sysupgrade, it 
fetched the corresponding firmware upgrade from the 7.1 directory. I 
already tried to downgrade that to the 7.0 version, which should be 
equal. Did not solve the issue. Then ran fw_update again ant let it 
fetch the 7.1 version again. Just noticed the last 3 lines in the dmesg 
output. Those did not appear before. Does the listing of the 
/etc/firmware directory indicate something is wrong with the firmware 
image? The fsck during sysupgrade did not show any failures and I 
already reinstalled the firmware during testing. Thanks for helping me out.


Best regards,
--
Christian
total 17100
drwxr-xr-x   4 root  wheel   2.5K Jun 11 01:05 .
drwxr-xr-x  41 root  wheel   2.5K Jun 13 02:33 ..
-rw-r--r--   1 root  bin49.5K Jun 11 01:05 3c990
-rw-r--r--   1 root  bin 1.9K Jun 11 01:05 3c990-license
-rw-r--r--   1 root  bin 5.7K Jun 11 01:05 atu-at76c503-i3863-ext
-rw-r--r--   1 root  bin21.6K Jun 11 01:05 atu-at76c503-i3863-int
-rw-r--r--   1 root  bin15.0K Jun 11 01:05 atu-at76c503-rfmd-acc-ext
-rw-r--r--   1 root  bin21.8K Jun 11 01:05 atu-at76c503-rfmd-acc-int
-rw-r--r--   1 root  bin14.5K Jun 11 01:05 atu-at76c505-rfmd-ext
-rw-r--r--   1 root  bin20.1K Jun 11 01:05 atu-at76c505-rfmd-int
-rw-r--r--   1 root  bin 5.7K Jun 11 01:05 atu-intersil-ext
-rw-r--r--   1 root  bin21.7K Jun 11 01:05 atu-intersil-int
-rw-r--r--   1 root  bin 2.0K Jun 11 01:05 atu-license
-rw-r--r--   1 root  bin15.1K Jun 11 01:05 atu-rfmd-ext
-rw-r--r--   1 root  bin21.9K Jun 11 01:05 atu-rfmd-int
-rw-r--r--   1 root  bin15.2K Jun 11 01:05 atu-rfmd2958-ext
-rw-r--r--   1 root  bin20.9K Jun 11 01:05 atu-rfmd2958-int
-rw-r--r--   1 root  bin16.3K Jun 11 01:05 atu-rfmd2958smc-ext
-rw-r--r--   1 root  bin19.8K Jun 11 01:05 atu-rfmd2958smc-int
-rw-r--r--   1 root  bin71.5K Jun 11 01:05 bnx-b06
-rw-r--r--   1 root  bin85.0K Jun 11 01:05 bnx-b09
-rw-r--r--   1 root  bin 2.0K Jun 11 01:05 bnx-license
-rw-r--r--   1 root  bin 5.5K Jun 11 01:05 bnx-rv2p
-rw-r--r--   1 root  bin 5.9K Jun 11 01:05 bnx-xi-rv2p
-rw-r--r--   1 root  bin 6.4K Jun 11 01:05 bnx-xi90-rv2p
-rw-r--r--   1 root  bin54.0K Jun 11 01:05 cs4280
-rw-r--r--   1 root  bin 427B Jun 11 01:05 cs4280-license
-rw-r--r--   1 root  bin 408B Jun 11 01:05 fxp-d101a
-rw-r--r--   1 root  bin 408B Jun 11 01:05 fxp-d101b0
-rw-r--r--   1 root  bin 536B Jun 11 01:05 fxp-d101ma
-rw-r--r--   1 root  bin 536B Jun 11 01:05 fxp-d101s
-rw-r--r--   1 root  bin 536B Jun 11 01:05 fxp-d102
-rw-r--r--   1 root  bin 536B Jun 11 01:05 fxp-d102c
-rw-r--r--   1 root  bin 536B Jun 11 01:05 fxp-d102e
-rw-r--r--   1 root  bin 1.6K Jun 11 01:05 fxp-license
drwxr-xr-x   2 root  wheel   2.0K Jun 10 07:39 i915
drwxr-xr-x   2 root  wheel   6.0K Jun 10 07:39 intel
-rw-r--r--   1 root  bin 1.6K Jun  9  2021 intel-ucode-license
-rw-r--r--   1 root  bin 330K Mar  7 20:25 iwn-100
-rw-r--r--   1 root  bin 330K Mar  7 20:25 iwn-1000
-rw-r--r--   1 root  bin 674K Mar  7 20:25 iwn-105
-rw-r--r--   1 root  bin 685K Mar  7 20:25 iwn-135
-rw-r--r--   1 root  bin 680K Mar  7 20:25 iwn-2000
-rw-r--r--   1 root  bin 691K Mar  7 20:25 iwn-2030
-rw-r--r--   1 root  bin 184K Mar  7 20:25 iwn-4965
-rw-r--r--   1 root  bin 333K Mar  7 20:25 iwn-5000
-rw-r--r--   1 root  bin 330K Mar  7 20:25 iwn-5150
-rw-r--r--   1 root  bin 444K Mar  7 20:25 iwn-6000
-rw-r--r--   1 root  bin 661K Mar  7 20:25 iwn-6005
-rw-r--r--   1 root  bin 664K Mar  7 20:25 iwn-6030
-rw-r--r--   1 root  bin 459K Mar  7 20:25 iwn-6050
-rw-r--r--   1 root  bin 2.2K Mar  7 20:25 iwn-license
-rw-r--r--   1 root  bin 4.5K Jun 11 01:05 kue
-rw-r--r--   1 root  bin 1.8K Jun 11 01:05 kue-license
-rw-r--r--   1 root  bin 368K Jun 11 01:05 myx-eth_z8e
-rw-r--r--   1 root  bin 379K Jun 11 01:05 myx-ethp_z8e
-rw-

Re: iwn0: fatal firmware error after sysupgrade to 7.1

[Christian Schulte]

On 14.06.22 11:36, Stefan Sperling wrote:

I don't know what SYSASSERT 0x0005 is supposed to tell us. All
I can do is make guesses based on what changed between 7.0 and 7.1.

In addition to the previous change, this patch enables even more
debug output, and it delays enabling of 40MHz in the RXON command
until the association sequence has progressed to a later stage.
Does this make any difference?


Indeed. I am sending this email using the iwn0 interface which is up and 
running without any issues so far. That was cool. Thank you very much. I 
attached the dmesg with those debug messages. Regarding that sysassert, 
I can only throw guesses at it. Maybe the firmware is checking some pre 
condition, it relies on the driver to have checked, and just bails out? 
Without any documentation, we'll never know. Router is a Fritz!Box 7362 
SL and there are four Fritz WLAN Repeater 310 configured in a mesh. 
Intend was to make those smartphone users happy. Nothing fancy, so that 
something no one would ever believe is not supported by the router or 
repeaters has led to this not being noticed during testing. It has been 
the first time ever since nearly 2 decades, that I had an issue with 
OpenBSD after an upgrade. A driver backed by a firmware blob no one can 
debug. Thank you all for this amazing system! Seems I am going to 
unsubscribe from misc@ for the next 20 years to come. Wow!


Best regards,
--
Christian
syncing disks... done
rebooting...
OpenBSD 7.1-stable (GENERIC.MP) #5: Wed Jun 15 02:59:34 CEST 2022
schu...@x500.schulte.it:/sys/arch/amd64/compile/GENERIC.MP
real mem = 4165603328 (3972MB)
avail mem = 4022079488 (3835MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xe0010 (80 entries)
bios0: vendor LENOVO version "6FET93WW (3.23 )" date 10/12/2012
bios0: LENOVO 22434DG
acpi0 at bios0: ACPI 3.0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT ECDT APIC MCFG HPET SLIC BOOT ASF! SSDT TCPA DMAR 
SSDT SSDT SSDT
acpi0: wakeup devices LID_(S3) SLPB(S3) UART(S3) IGBE(S4) EXP0(S4) EXP1(S4) 
EXP2(S4) EXP3(S4) EXP4(S4) PCI1(S4) USB0(S3) USB3(S3) USB5(S3) EHC0(S3) 
EHC1(S3) HDEF(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiec0 at acpi0
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM)2 Duo CPU P8400 @ 2.26GHz, 2261.36 MHz, 06-17-06
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,NXE,LONG,LAHF,PERF,SENSOR,MELTDOWN
cpu0: 3MB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 7 var ranges, 88 fixed ranges
cpu0: apic clock running at 266MHz
cpu0: mwait min=64, max=64, C-substates=0.2.2.2.2.1.3, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM)2 Duo CPU P8400 @ 2.26GHz, 2261.01 MHz, 06-17-06
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,NXE,LONG,LAHF,PERF,SENSOR,MELTDOWN
cpu1: 3MB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec0, version 20, 24 pins, remapped
acpimcfg0 at acpi0
acpimcfg0: addr 0xe000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (AGP_)
acpiprt2 at acpi0: bus 2 (EXP0)
acpiprt3 at acpi0: bus 3 (EXP1)
acpiprt4 at acpi0: bus -1 (EXP2)
acpiprt5 at acpi0: bus 5 (EXP3)
acpiprt6 at acpi0: bus 13 (EXP4)
acpiprt7 at acpi0: bus 21 (PCI1)
acpibtn0 at acpi0: LID_
acpibtn1 at acpi0: SLPB
acpipci0 at acpi0 PCI0: 0x 0x0011 0x0001
acpicmos0 at acpi0
com0 at acpi0 UART addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
tpm0 at acpi0 TPM_ 1.2 (TIS) addr 0xfed4/0x5000, device 0x10208086 rev 0x6
acpibat0 at acpi0: BAT0 not present
acpiac0 at acpi0: AC unit online
acpithinkpad0 at acpi0: version 1.0
"IBM0079" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpicpu0 at acpi0: !C3(250@17 mwait.3@0x20), !C2(500@1 mwait.1@0x10), C1(1000@1 
mwait.1), PSS
acpicpu1 at acpi0: !C3(250@17 mwait.3@0x20), !C2(500@1 mwait.1@0x10), C1(1000@1 
mwait.1), PSS
acpipwrres0 at acpi0: PUBS, resource for USB0, USB3, USB5, EHC0, EHC1
acpitz0 at acpi0: critical temperature is 127 degC
acpitz1 at acpi0: critical temperature is 100 degC
acpidock0 at acpi0: GDCK docked (15)
acpivideo0 at acpi0: VID_
acpivout0 at acpivideo0: LCD0
acpivideo1 at acpi0: VID_
cpu0: Enhanced SpeedStep 2261 MHz: speeds: 2267, 2266, 1600, 800 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel GM45 Host" rev 0x07
inteldrm0 at pci0 dev 2 function 0 "Intel GM45 Video" rev 0x07
drm0 at inteldrm0
intagp0 at inteldrm0
agp0 at intagp0: aperture at 0xd000, size 0x1000
inteldrm0: apic 1 int 16, GM45, gen 4

smtpd warn: not enough disk space

[Christian Schulte]

Hello,

just stumbled upon this. In /usr/src/usr.sbin/smtpd/queue_fs.c there is 
a function fsqueue_check_space which checks for space based on 
percentages. I am not using any partitioning on my laptop. Not a mail 
server or anything special. One disk. / is that one disk. I know...


x500$ df -h
Filesystem SizeUsed   Avail Capacity  Mounted on
/dev/sd0a  105G   96.5G3.2G97%/

The check is working correctly. Capacity is at 97%. Still, there are 
3.2G free disk space and smtpd is refusing to send out mail.


Jul  5 02:06:17 x500 smtpd[63712]: warn: not enough disk space: 4% left
Jul  5 02:06:17 x500 smtpd[63712]: warn: temporarily rejecting messages

What is the reasoning to check for disk space based on percentages? I 
have never seen an application performing such kind of checks. If there 
is not enough space, write would fail anyway. Checking for available 
bytes before write would make sense. Checking a percentage value? I 
don't get it. Not being able to send mail, although 3.2G space is 
available makes no sense, IMHO.


Regards,
--
Christian

Re: smtpd warn: not enough disk space

[Christian Schulte]

On 05.07.24 13:50, Souji Thenria wrote:

On Fri Jul 5, 2024 at 4:19 AM BST, Christian Schulte wrote:

Hello,

Hi Christian,

What is the reasoning to check for disk space based on percentages? I 
have never seen an application performing such kind of checks. If 
there is not enough space, write would fail anyway. Checking for 
available bytes before write would make sense. Checking a percentage 
value? I don't get it. Not being able to send mail, although 3.2G 
space is available makes no sense, IMHO.


You are not the first one with this issue. However, it looks like it
went nowhere.
https://www.mail-archive.com/misc@opensmtpd.org/msg01007.html

It might be worth to restate the question on the OpenSMTPD mailing list.


Thank you. I should have used that mailing list, of course.

Regards,
--
Christian

Re: smtpd warn: not enough disk space

[Christian Schulte]

On 05.07.24 13:46, Jeremy Mates wrote:

On 2024-07-05 05:19:01 +0200, Christian Schulte wrote:

I have never seen an application performing such kind of checks.


Sendmail had a knob to refuse mail at a certain CPU load, on the
assumption that if a system was "too busy" it's in a bad state and
accepting mail would only make things worse. Also rogue(6) had CPU
checks, apparently they wanted to stop any rogue processes when the
system load got too high--maybe real work was being done? Speaking of
not checking the CPU, I have an amusing story of what a dual-processor
Linux box did at a CPU load of 5,000, something something sending the
wrong bytes to the wrong file descriptors, thus corrupting payment
transactions for a small internet retailer.

For application-specific filesystem checks I don't recall any off the
top of my head, but I tend to have monitoring send dire alerts when the
disk usage hits 90% (or less, if you want more breathing room to figure
out what is wrong, who to notify, them to act, etc) so would never have
seen anything that triggers at 5% free or whatever.


If there is not enough space, write would fail anyway.


Or the filesystem gets somewhat looped after being run at 100% full for
too long. Granted, that was Windows, and we did warn the user not to do
that, but they did, and then when a nice Windows admin tried to backup
the 500G disk, it filled up a 3T disk and wanted more. Who knows what
breaks when a stray dd(1) or package update or whatever eats too much of
the remaining space. Not breaking randomly is probably a design goal of
a SMTP server that tries to guarantee delivery? You could fiddle with
the percentage, or remove the code, but I'm going to guess that most
everyone else keeps their mail partitions not so full.


I just commented out the function call locally for my laptop. The issue 
had been discussed on the OpenSMTPD mailing list as well. As it seems, 
everyone seems to agree, that checking a fixed percentage value makes no 
sense. I do not disagree that checking/preserving some amount of free 
disk space makes sense. For example, I ran into an out of disk space 
situation on a server running postgresql filling up the disk 100% and 
then crashing. It's a personal VPS server I am not maintaining 
professionally. A database admin would have monitored the system and 
just enhanced storage when required. Bad thing for me was, that I could 
not vaccuum the database, because postgresql copies tables to new files 
to reclaim disk space afterwards. So the database was unusable for about 
12 hours, until I could manage to xz a backup and transfer it to another 
machine. Making postgresql aware of keeping some free disk space, it 
would need to preserve around 50%, so that vaccuum can always be 
performed. That would be just impractical. My objection is that 
something like this should not be coded into the application itself, but 
rather is a task of general system maintenance (e.g. setting up 
filesystem quotas and such). For what it's worth, it's just my private 
laptop...


Regards,
--
Christian

Re: smtpd warn: not enough disk space

[Christian Schulte]

On 06.07.24 04:08, Eric Pruitt wrote:

On Sat, Jul 06, 2024 at 01:49:05AM +0200, Christian Schulte wrote:

A database admin would have monitored the system and just enhanced
storage when required. Bad thing for me was, that I could not vaccuum
the database, because postgresql copies tables to new files to reclaim
disk space afterwards. So the database was unusable for about 12
hours, until I could manage to xz a backup and transfer it to another
machine. Making postgresql aware of keeping some free disk space, it
would need to preserve around 50%, so that vaccuum can always be
performed.


This is tangential to the original issue, but I've heard of some
administrators leaving around large files so they can be removed when
the disk unexpectedly fills up buying them more time to address the
underlying issue, and CockroachDB implements this natively:
<https://www.cockroachlabs.com/docs/stable/cockroach-debug-ballast>.

Eric


Those large files resided in /root/backup for me. Luckily I setup a 
cronjob doing some daily backups just to be save. If it would not have 
been possible to delete those, freeing some GB, I would have been really 
screwed. No way to get out of this situation without increasing storage. 
That would have been impossible with my provider as well. For the 
record: Never ever let postgresql fill up your storage 100%, when there 
is no way for you to increase storage. Just wondering how the postgresql 
port is configured. Really should setup quotas automatically when 
pkg_adding in a way, just to ensure, that no one ever runs into a 
situation, that there is no way out of a disk full situation.


Regards,
--
Christian

Re: smtpd warn: not enough disk space

[Christian Schulte]

On 07.07.24 03:51, Jeremy Evans wrote:
On Fri, Jul 5, 2024 at 9:16 PM Christian Schulte <mailto:schulte...@gmail.com>> wrote:


Just wondering how the postgresql
port is configured. Really should setup quotas automatically when
pkg_adding in a way, just to ensure, that no one ever runs into a
situation, that there is no way out of a disk full situation.


I'm not aware of any port that sets up quotas automatically, so I don't 
understand why you think this is an issue with the PostgreSQL port 
specifically.  Since you are wondering how the PostgreSQL port is 
configured: 
https://cvsweb.openbsd.org/ports/databases/postgresql/Makefile?rev=1.304&content-type=text/x-cvsweb-markup <https://cvsweb.openbsd.org/ports/databases/postgresql/Makefile?rev=1.304&content-type=text/x-cvsweb-markup>


Jeremy



I did not criticize the postgresql port in any way. I am just 
suggesting, that when you want to setup a postgresql server in a 
fire-and-forget way of things, it would be cool to restrict it from 
eating up all available storage. Just because only then will you notice 
how difficult it can get to get out of such a situation. That's all. 
Discussion started with an MTA blindly preserving 5% of space for 
temporary queue files, which are, well, temporary. Makes no sense. Queue 
will get emptied whatsoever automatically. Completely different scenario 
with a database system. The MTA can and will resolve such a situation 
automatically over a period of a few days. This does not hold true for a 
database system, which is not dealing with temporary data and just needs 
a way to ensure someone never runs into a non recoverable situation. No 
need to apply any changes to the postgresql port. If you know how nasty 
things can get, you can also just use GNU/Linux to shoot you into your 
own feet. So to say. There is a reason why we are here.


Regards,
--
Christian

Re: smtpd warn: not enough disk space

[Christian Schulte]

On 07.07.24 03:51, Jeremy Evans wrote:
On Fri, Jul 5, 2024 at 9:16 PM Christian Schulte <mailto:schulte...@gmail.com>> wrote:


Just wondering how the postgresql
port is configured. Really should setup quotas automatically when
pkg_adding in a way, just to ensure, that no one ever runs into a
situation, that there is no way out of a disk full situation.


I'm not aware of any port that sets up quotas automatically, so I don't 
understand why you think this is an issue with the PostgreSQL port 
specifically.  Since you are wondering how the PostgreSQL port is 
configured: 
https://cvsweb.openbsd.org/ports/databases/postgresql/Makefile?rev=1.304&content-type=text/x-cvsweb-markup <https://cvsweb.openbsd.org/ports/databases/postgresql/Makefile?rev=1.304&content-type=text/x-cvsweb-markup>


Jeremy



I am not wondering about anything. Just suggesting improvements. 
Currently I grabbed my old Atari 800XL and Atari Falcon 030 doing some 
cool things whit those machines using a PC for calculating data quicker, 
than had been possible in the 80s and 90s. Those 8 bit Ataris were 
really great machines. The ancestor Amiga as well. For example: Just 
remove the patches in this directory - well a lot of them - and see how 
those GNU folks have turned into complete idiots. I don't get it.


https://github.com/openbsd/ports/tree/master/devel/gettext/patches

They are introducing hotspots like hell and then try to speed those up 
by using insecure non sense. Those GNU folks have no soul. Just remove a 
bunch of those patches in the ports tree and see how stupid they were 
having not eliminated those hotspots in the first place but trying to 
make them as fast and insecure as possible.


Maybe time for me to write some 6502 on my Atari 800XL. There you do not 
need to cope with GNU idiots at all.


Regards,
--
Christian

Running OpenBSD on a VPS.

[Christian Schulte]

Hello misc@,

please see attached a dmesq of a Linux VPS server. I talked to the 
provider and got told that I can access the console usins VLC and can 
provide my own iso images to install whatever OS I like. Does OpenBSD 
support this kind of host system? I would like to avoid any kind of 
experiments, because I would need to migrate a machine, I had installed 
Debian Woody on a long time ago and then just dist-upgraded whenever 
needed over the years. I just learnt the wonders of systemd and a really 
nifty out of memory killer not giving my application any chance to 
detect, that it is running out of memory, although it could handle that 
and ... #$!"§ ... tl;dr. I understand I will need to setup a different 
system from scratch and replace various things (e.g. sendmail, 
milter-greylist, clamav-milter, spamass-milter, http, imap, etc.) with 
something else. I would really take the time to do it. Does OpenBSD 
support such a host? One fixed IPv4 and IPv6 address?


Regards,
--
Christian[0.00] Linux version 6.1.0-22-amd64 (debian-ker...@lists.debian.org) 
(gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 
SMP PREEMPT_DYNAMIC Debian 6.1.94-1 (2024-06-21)
[0.00] Command line: BOOT_IMAGE=/vmlinuz-6.1.0-22-amd64 
root=UUID=5efb3645-feed-412f-841d-4ddae853431f ro rootdelay=10 net.ifnames=0 
ixgbe.allow_unsupported_sfp=1 quiet
[0.00] BIOS-provided physical RAM map:
[0.00] BIOS-e820: [mem 0x-0x0009fbff] usable
[0.00] BIOS-e820: [mem 0x0009fc00-0x0009] reserved
[0.00] BIOS-e820: [mem 0x000f-0x000f] reserved
[0.00] BIOS-e820: [mem 0x0010-0xbffdafff] usable
[0.00] BIOS-e820: [mem 0xbffdb000-0xbfff] reserved
[0.00] BIOS-e820: [mem 0xfeffc000-0xfeff] reserved
[0.00] BIOS-e820: [mem 0xfffc-0x] reserved
[0.00] BIOS-e820: [mem 0x0001-0x00043fff] usable
[0.00] NX (Execute Disable) protection: active
[0.00] SMBIOS 2.8 present.
[0.00] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
[0.00] Hypervisor detected: KVM
[0.00] kvm-clock: Using msrs 4b564d01 and 4b564d00
[0.01] kvm-clock: using sched offset of 554415650994087 cycles
[0.05] clocksource: kvm-clock: mask: 0x max_cycles: 
0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[0.16] tsc: Detected 2794.748 MHz processor
[0.000861] e820: update [mem 0x-0x0fff] usable ==> reserved
[0.000864] e820: remove [mem 0x000a-0x000f] usable
[0.000869] last_pfn = 0x44 max_arch_pfn = 0x4
[0.000924] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
[0.000938] last_pfn = 0xbffdb max_arch_pfn = 0x4
[0.003801] found SMP MP-table at [mem 0x000f5a80-0x000f5a8f]
[0.003834] Using GB pages for direct mapping
[0.004071] RAMDISK: [mem 0x3451b000-0x36284fff]
[0.004094] ACPI: Early table checksum verification disabled
[0.004117] ACPI: RSDP 0x000F5880 14 (v00 BOCHS )
[0.004132] ACPI: RSDT 0xBFFE250D 34 (v01 BOCHS  BXPCRSDT 
0001 BXPC 0001)
[0.004143] ACPI: FACP 0xBFFE2399 74 (v01 BOCHS  BXPCFACP 
0001 BXPC 0001)
[0.004152] ACPI: DSDT 0xBFFE0040 002359 (v01 BOCHS  BXPCDSDT 
0001 BXPC 0001)
[0.004156] ACPI: FACS 0xBFFE 40
[0.004158] ACPI: APIC 0xBFFE240D A0 (v01 BOCHS  BXPCAPIC 
0001 BXPC 0001)
[0.004163] ACPI: HPET 0xBFFE24AD 38 (v01 BOCHS  BXPCHPET 
0001 BXPC 0001)
[0.004166] ACPI: WAET 0xBFFE24E5 28 (v01 BOCHS  BXPCWAET 
0001 BXPC 0001)
[0.004168] ACPI: Reserving FACP table memory at [mem 0xbffe2399-0xbffe240c]
[0.004169] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2398]
[0.004170] ACPI: Reserving FACS table memory at [mem 0xbffe-0xbffe003f]
[0.004171] ACPI: Reserving APIC table memory at [mem 0xbffe240d-0xbffe24ac]
[0.004172] ACPI: Reserving HPET table memory at [mem 0xbffe24ad-0xbffe24e4]
[0.004173] ACPI: Reserving WAET table memory at [mem 0xbffe24e5-0xbffe250c]
[0.004545] No NUMA configuration found
[0.004547] Faking a node at [mem 0x-0x00043fff]
[0.004556] NODE_DATA(0) allocated [mem 0x43ffd5000-0x43fff]
[0.004933] Zone ranges:
[0.004938]   DMA  [mem 0x1000-0x00ff]
[0.004940]   DMA32[mem 0x0100-0x]
[0.004942]   Normal   [mem 0x0001-0x00043fff]
[0.004943]   Device   empty
[0.004944] Movable zone start for each node
[0.004948] Early memory node ranges
[0.004948]   node   0: [mem 0x1000-0x0009efff]
[0.004

Re: Running OpenBSD on a VPS.

[Christian Schulte]

On 11.07.24 03:41, Geoff Steckel wrote:

On 7/10/24 20:40, Christian Schulte wrote:

Hello misc@,

 I understand I will need to setup a different system from scratch and 
replace various things (e.g. sendmail, milter-greylist, clamav-milter, 
spamass-milter, http, imap, etc.) with something else. I would really 
take the time to do it. Does OpenBSD support such a host? One fixed 
IPv4 and IPv6 address?


My web/mail server is on a VM and I literally have no idea what the host 
is!

It runs from os release (every 6 months) to os release.
IIRC it has 768Meg of (virtual) ram and 50GB (virtual) disk

The system comes with a core set of utilities and server programs
A mail and web server -might- need addons but they will work well out of 
the box.


I would *very* strongly recommend that you use the provided http and 
smtp servers
and any other ones that match your needs. The standard servers have been 
carefully audited

for security problems. The config files are -much- simpler than the
apache servers or legacy sendmail, etc. Many come already configured for 
a simple server or client.


Some of the milters are in the standard distribution or are in ports.

There are a lot of utilities and servers and... in ports. Thousands.
My imapd is from ports. It needs a tweak that I'll put in and submit 
Real Soon Now.

it does work quite well.

My setup has one IP4 address and a /64. I have a tunnel to my home network.

If you have VNC console access the half-yearly release update requires
"doas sysupgrade"
and afterwards
"doas package_add -u"
and you're done.
(doas is the sudo replacement - simple and easy to configure
IIRC it comes set up for group wheel or you remove a #)

The biggest downside is that the ports lag the "bleeding edge" by a year 
or a little more.
The core system maintainers are very conservative. The group is small 
and they concentrate
on the kernel and core utilities. There is a peripheral group of ports 
maintainers, etc.


a "ps a" shows about 20 daemons running. Many are split into 2 or more 
processes for
for security - one runs as root which configures and runs the others 
without privileges.


hth
Geoff Steckel
  good luck
  Geoff Steckel


It's like migrating the last 3 decades of your life to a new system. I 
knew that would hit me sometime in the future, when some commit made the 
a/c controller disappear back then. Fan controller stopped working after 
an update. And that was used to control air conditioning. Burn your 
basement, set your house on fire -> use linux.


<https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v6.10-rc7&id=c46c0e9188685c0276b4c0adf9fb7e903937e35b>

Running OpenBSD since then personally. Never had a chance to install it 
to a server, because the providers did not support it. Now they do. 
Making a list of what I need to do will take an awful lot of time. For 
example, I would not want to install an apache http server, just because 
I need DAV support for subversion, file uploads during automated release 
processes and so on. So I understand how much effort it would take to 
e.g. enhance the default httpd in OpenBSD base to support all of that. 
Maybe no one wants that httpd to support such kind of things. What do I 
know? And that's just one example. We are talking months fulltime here 
already. This will need an awful lot of planning...and an awful lot of 
asking questions here on how to replace this and that form here and 
there with something the BSD way. Never had a single issue with OpenBSD 
since I installed it back in 2009. Not a single one ever since.


Regards.
--
Christian

Re: smtpd warn: not enough disk space

[Christian Schulte]

On 09.07.24 11:18, Stuart Henderson wrote:

On 2024-07-09, Christian Schulte  wrote:



On 07.07.24 03:51, Jeremy Evans wrote:

On Fri, Jul 5, 2024 at 9:16 PM Christian Schulte mailto:schulte...@gmail.com>> wrote:

 Just wondering how the postgresql
 port is configured. Really should setup quotas automatically when
 pkg_adding in a way, just to ensure, that no one ever runs into a
 situation, that there is no way out of a disk full situation.


The port can't sanely do that, because it doesn't know how the admin
has configured their system.

Also, openbsd doesn't enable filesystem quotas by default.


I did not criticize the postgresql port in any way. I am just
suggesting, that when you want to setup a postgresql server in a
fire-and-forget way of things, it would be cool to restrict it from
eating up all available storage.


That is simple, use a separate filesystem for /var/postgresql.



Indeed. This is what my provider provided me with. Watch out for the 
mount point of /. This would not have been a problem, if postgresql 
would be able to reclaim diskspace without requiring disk space to do 
so. This is what you get, for not taking care of yourself and blindly 
relying on others to provide you with sane defaults. Was it really my 
fault? Maybe. A simple warning message during install of postgresql 
about what issue might I run into would have been - hmm - nice.


sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs 
(rw,nosuid,relatime,size=8168948k,nr_inodes=2042237,mode=755,inode64)
devpts on /dev/pts type devpts 
(rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs 
(rw,nosuid,nodev,noexec,relatime,size=1637608k,mode=755,inode64)
/dev/sda2 on / type ext4 (rw,relatime,errors=remount-ro)
securityfs on /sys/kernel/security type securityfs 
(rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run/lock type tmpfs 
(rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 
(rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs 
(rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2495)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl 
(rw,nosuid,nodev,noexec,relatime)
ramfs on /run/credentials/systemd-sysusers.service type ramfs 
(ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-sysctl.service type ramfs 
(ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup-dev.service type ramfs 
(ro,nosuid,nodev,noexec,relatime,mode=700)
/dev/sda1 on /boot type ext4 (rw,noatime)
ramfs on /run/credentials/systemd-tmpfiles-setup.service type ramfs 
(ro,nosuid,nodev,noexec,relatime,mode=700)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc 
(rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/user/1000 type tmpfs 
(rw,nosuid,nodev,relatime,size=1637604k,nr_inodes=409401,mode=700,uid=1000,gid=1000,inode64)

Re: smtpd warn: not enough disk space

[Christian Schulte]

On 09.07.24 11:16, Stuart Henderson wrote:

On 2024-07-09, Christian Schulte  wrote:

For example: Just
remove the patches in this directory - well a lot of them - and see how
those GNU folks have turned into complete idiots. I don't get it.

https://github.com/openbsd/ports/tree/master/devel/gettext/patches


A lot of those patches are to avoid triggering warnings from ld when
linking other programs which use the gettext library due to the
api warnings openbsd has for some libc functions.




Of course they do. And there is a reason they do. This goes back to me 
running i386 instead of amd64 due to RAM constraints and using the gnome 
desktop environment, which makes heavy use of gettext. gettext authors 
have just sped up those functions, because they noticed themselves, that 
they are called way to often. Then those patches removed those speed ups 
uncovering those design flaws. Of course they noticed this themselves. 
They never rethought design decisions. Install i386 with a gnome desktop 
environment. So slow. Rebuild gettext wihtout some of those patches 
removing those speed ups. There you go. It's so noticeable and it's not 
those patches to blame. That's what I meant with "avoid hotspots" rather 
than trying to make them run faster. That's theire philosophy. And that 
is just wrong.

Re: Running OpenBSD on a VPS.

[Christian Schulte]

On 11.07.24 12:11, Kirill A. Korinsky wrote:

On Thu, 11 Jul 2024 03:10:43 +0100,
Christian Schulte  wrote:


Running OpenBSD since then personally. Never had a chance to install it
to a server, because the providers did not support it. Now they do.


Not all of them. Special in case like Hetzner, online.net and similar one.

But they allow to load the server from rescue mode with some linux, what
opens a kind of backdoor where you run QEMU to install OpenBSD :)

You may achive some kind of semi-automatic installation with answer files,
but QEMU uses only tftp-server-name which support was removed at 7.0, so,
semi-automatic installation via QEMU works for OpenBSD up to 6.9.



Does not make much sense discussing this here. Even if OpenBSD would 
encrypt the vmm layer as well, there is no way to setup OpenBSD with 
such a provider in a way not risking someone to be able to break 
security. Moved away from very stressful Alfahosting to Contabo. 
Alfahosting once thought it would be cool to just assign a new IP 
address and things like that. Contabo at least offers to setup a VPS 
with custom iso images providing VLC console access and such. From a 
security point of view, this makes no sense at all discussing this here. 
There is no security with those kind of setups and we all know it. I am 
just glad I can run OpenBSD there.


Regards,
--
Christian

Re: Running OpenBSD on a VPS.

[Christian Schulte]

On 13.07.24 11:03, Janne Johansson wrote:

address and things like that. Contabo at least offers to setup a VPS
with custom iso images providing VLC console access and such. From a


I think you mean a VNC console, not the road-cone media player.
I could bear the mistake once, but now it looks like a pattern, hence
the nitpicking about this detail.



VNC, of course. Not the media player. Thanks for pointing that out.

Re: Running OpenBSD on a VPS.

[Christian Schulte]

On 13.07.24 07:52, Crystal Kolipe wrote:

On Sat, Jul 13, 2024 at 02:16:12AM +0200, Christian Schulte wrote:

There is no security with those kind of setups and we
all know it. I am just glad I can run OpenBSD there.


But if you want to run internet-facing servers without exposing access to them
to the VPS hosting provider, you can still make use of inexpensive VPS
services running OpenBSD to tunnel connections back to a server on a home
broadband connection, (which might not have a static IP, might lack IPv6, or
might not allow inbound connections).

As long as you control the keys and certs on the machine which is physically
under your control, and you are using appropriate algorithms for
authentication, then access to the upstream VPS by the provider, (or anyone
else), can't be used to man-in-the-middle your connections, (without being
detected).


That's an interesting idea. Using some cheap VPS server just to get a 
static IP address but do not host any data there. Thanks a lot.

Re: Optimization Advice for High Resource Utilization PostgreSQL Query on OpenBSD

[Christian Schulte]

On 30.07.24 19:29, Kihaguru Gathura wrote:
> Hi Claudio,
> 
> Yes, I did run 'Explain Analyze' on the query to diagnose the
> performance issues. Based on the analysis, I created indexes on the
> relevant columns and ran a VACUUM on the tables. This resulted in an
> improvement, reducing CPU utilization from 40% to 30%.

Out of curiosity. How many records (table rows) are you talking about?
Maybe partitioning is the way to go, if you are dealing with huge
amounts of data. Indexing a couple of billions of records still makes
selecting expensive and you maybe want to partition things?

-- 
Christian

Re: Lastest snapshot - all application got a speed increase

[Christian Schulte]

On 07.08.24 13:54, Mihai Popescu wrote:
> On Wed, Aug 7, 2024 at 2:35 PM Janne Johansson  wrote:
>>
>>> What is this kernel lock everybody talks about. I mean what is locked?
>>> Some actions must be done and devs call lock before and after it is
>>> done, they call unlock?
>>> What is kernel lock doing exactly, it prevents other procedures to run?
>>
>> It is quite a large topic, but the 30 second intro is
>> https://en.wikipedia.org/wiki/Giant_lock
>>
>> Lots of more detailed information is available with a bit of googling.
>>
>> --
>> May the most significant bit of your life be positive.
> 
> I was on wikipedia, i did my gogling. There is too much spread and not
> very much related to OpenBSD.
> But it is a large topic i think. And i am not up to understand all.
> 
> Thank you.
> 

That's a historic issue as well. Systems were not designed for more than
one CPU back in the days. There are very few kernels and systems which
had been designed from the ground up for supporting parallel tasks. From
the top of my head:




Issue with making those single CPU systems support parallelism is to
carefully review the data structures. What once was global, really was
either global or per CPU. Finding out about this and how to adopt to
those new structures is a challenge, everyone has another view to. If
you compare FreeBSD to DragonFlyBSD, or to Linux, OpenBSD, whatever,
everyone gives it another try. We'll see. Even CPU interfaces have
changed over time, so better not hurry with it. It really boils down to
carefully reviewing global state and relations. For example: Is it one
scheduler per N CPUs, or really should that be N schedulers per N CPUs
and things like that.


Regards,
-- 
Christian

How to add support to OpenSMTPD to distinguish between MTA and MSA operation mode?

[Christian Schulte]

Hi misc@,

I just started to read OpenSMTPD sources. Regarding the latest
discussions on tech@, there maybe seems to be the need to instruct
OpenSMTPD listeners to behave differently when acting as MTA or MSA.
Reading man smtpd.conf(5)[1] there is an option to add a tag to the
"listen on [socket]" directive. Those tags are currently used in "match"
directives. Maybe this could be extended to add well known tags to a
listener to control operation modes like MTA or MSA. Both of this is
documented in corresponding RFCs. Is there any interest for me to take a
closer look any maybe come up with some patches tech@? I am really just
starting to read OpenSMTPD sources. If there already is a way to
instruct OpenSMTPD listeners to behave differently regarding to - for
example - handling message ids based on in either MTA or MSA mode,
please let me know. I would like to avoid spending time into this, if
there is no interest but I think there really should be a way to
configure listeners to specific operation modes like MTA or MSA.

[1] 

-- 
Christian

Re: How to add support to OpenSMTPD to distinguish between MTA and MSA operation mode?

[Christian Schulte]

On 01.09.24 15:39, Kirill A. Korinsky wrote:
> On Sun, 01 Sep 2024 13:12:19 +0200,
> Christian Schulte  wrote:
>>
>> I just started to read OpenSMTPD sources. Regarding the latest
>> discussions on tech@, there maybe seems to be the need to instruct
>> OpenSMTPD listeners to behave differently when acting as MTA or MSA.
>> Reading man smtpd.conf(5)[1] there is an option to add a tag to the
>> "listen on [socket]" directive. Those tags are currently used in "match"
>> directives. Maybe this could be extended to add well known tags to a
>> listener to control operation modes like MTA or MSA. Both of this is
>> documented in corresponding RFCs. Is there any interest for me to take a
>> closer look any maybe come up with some patches tech@? I am really just
>> starting to read OpenSMTPD sources. If there already is a way to
>> instruct OpenSMTPD listeners to behave differently regarding to - for
>> example - handling message ids based on in either MTA or MSA mode,
>> please let me know. I would like to avoid spending time into this, if
>> there is no interest but I think there really should be a way to
>> configure listeners to specific operation modes like MTA or MSA.
>>
> 
> As far as I know and had discovered by reading sources the specified
> behaviour for the MSA like adding Message-Id is triggered only when listener
> is used submission port.
> 
> So, simple use submission port and that's it.
> 
> P.S. I think that m...@opensmtpd.org is the right misc@ for this email.
> 

Thinking out loud now. If OpenSMTPD does something based on the port
configured, I would rate that utterly workaround. I am thinking about
adding support for something like this.


listen on ... mode mta
listen on ... mode mta-accurate
listen on ... mode msa
listen on ... mode msa-accurate

Maybe someone wants to s/accurate/pedantic/g. But that's a language thing.

mode mta

Default mode, when no mode is given in the configuration file. Without
any logic about ports. Just transport the message. Even if it contains a
message id .

mode mta-accurate

Would refuse to accept any mails without a (valid) message id. Permanent
failure. "No message id or message id invalid" right now lacking a
definition of invalid but that is documented in various RFCs.

mode msa

Check validity of the message id. If not present, add a message id. If
not valid, overwrite it silently with something sane.

mode msa-accurate

Check validity of the message id. If not present, add a message id. If
not valid, permanent failure.

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/12/24 18:33, Philip Guenther wrote:
> On Sat, Oct 12, 2024 at 3:58 AM Christian Schulte  wrote:
>>
>> On 10/11/24 15:05, Claudio Jeker wrote:
>>> On Fri, Oct 11, 2024 at 03:00:23PM +0200, Christian Schulte wrote:
>>>> On 10/11/24 13:57, Stuart Henderson wrote:
>>>>> On 2024-10-09, obs...@loopw.com  wrote:
>>>>>>> In a second server I have upgraded from 7.5 i386 to 7.6 i386 but server 
>>>>>>> sees only 4GB of RAM
>>>>>>
>>>>>>> Is anybody with similar experiences? Any ideas how to fix RAM?
>>>>>>
>>>>>> run 64bit OpenBSD
>>>>>>
>>>>>> 32bit address space is limited to a max of 4GB**, and some of that is 
>>>>>> eaten up by PCI devices, etc.
>>>>>> ** PAE can work around this, but I’m not sure if OpenBSD supports PAE at 
>>>>>> all (and theres other issues/caveats with using PAE of course, including 
>>>>>> speed and security)
>>>>>
>>>>> OpenBSD does support PAE on i386, but not for increasing address space,
>>>>> just for W^X. See /sys/arch/i386/i386/pmapae.c r1.28.
>>>>>
>>>>>
>>>>
>>>> Hmm. Why not give up on i386 and make that i686 instead (Pentium Pro)?
>>>> What I mean by this. Rename the current i386 to i686 by compiling it for
>>>> i686 and update the pmap to support more than 4GB physical RAM (PAE).
>>>> Supporting i386 is like supporting 68030.
>>>
>>> That's such a trivial task you should be able to finish it over the
>>> weekend. We all wait for your diff on Monday.
>>
>> Is this irony, or would someone really wait for a diff from me and use
>> it? Take i386. Compile it with something -march=i686 or pentiumpro by
>> default. That's it. Add support for the various PAE MMU options. You get
>> a 32 bit OS supporting more than 4GB of physical RAM.
> 
> Your list two things to do, both of which are either already done or
> trivial, and fail to mention either where the real work would be
> needed (adapting UVM for sizeof(paddr_t) > sizeof(vaddr_t)) or, more
> importantly, what the long term costs of maintaining that support when
> i386 would be the only arch using it.

Touching machine independent interfaces/implementations for this, of
course, would not be that ideal. No one can predict the future. Maybe
there will come a time when current 64 bit CPUs will start doing
something similar to PAE before a new 128 bit era or such. That
sizeof(paddr_t) == sizeof(vaddr_t) invariant maybe sometime in the far
future need to be rethought of anyway. Who knows?

> Your obsolete boxes do not merit our paying the costs into the far future.

Fair enough. No useless efforts from me on this then. The original
author of this thread got told to just give up on i386. I got told to
give up on i386. What boxes is i386 used on not obsolete because not
supporting amd64? You could even pull the plug from i386, no? Who is
using it and why? Buy a new device supporting amd64 and be done with it.

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/11/24 15:05, Claudio Jeker wrote:
> On Fri, Oct 11, 2024 at 03:00:23PM +0200, Christian Schulte wrote:
>> On 10/11/24 13:57, Stuart Henderson wrote:
>>> On 2024-10-09, obs...@loopw.com  wrote:
>>>>> In a second server I have upgraded from 7.5 i386 to 7.6 i386 but server 
>>>>> sees only 4GB of RAM
>>>>
>>>>> Is anybody with similar experiences? Any ideas how to fix RAM?
>>>>
>>>> run 64bit OpenBSD
>>>>
>>>> 32bit address space is limited to a max of 4GB**, and some of that is 
>>>> eaten up by PCI devices, etc.
>>>> ** PAE can work around this, but I’m not sure if OpenBSD supports PAE at 
>>>> all (and theres other issues/caveats with using PAE of course, including 
>>>> speed and security)
>>>
>>> OpenBSD does support PAE on i386, but not for increasing address space,
>>> just for W^X. See /sys/arch/i386/i386/pmapae.c r1.28.
>>>
>>>
>>
>> Hmm. Why not give up on i386 and make that i686 instead (Pentium Pro)?
>> What I mean by this. Rename the current i386 to i686 by compiling it for
>> i686 and update the pmap to support more than 4GB physical RAM (PAE).
>> Supporting i386 is like supporting 68030.
> 
> That's such a trivial task you should be able to finish it over the
> weekend. We all wait for your diff on Monday.
> 

Is this irony, or would someone really wait for a diff from me and use
it? Take i386. Compile it with something -march=i686 or pentiumpro by
default. That's it. Add support for the various PAE MMU options. You get
a 32 bit OS supporting more than 4GB of physical RAM. I have so many
older laptops/servers/whatever lying around I could (re)start using with
this. My current daily is a Lenovo x240 with 8GB of RAM running amd64
and this thing is swapping like mad. Throw a 32 bit OS at it supporting
those 8GB of RAM and go for it. Why would anyone throw away such a
machine, just because a 64 bit OS hits its boundaries, when a 32 bit OS
would not?

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/14/24 10:33, Stuart Henderson wrote:
> On 2024-10-12, Christian Schulte  wrote:
>> Take i386. Compile it with something -march=i686 or pentiumpro by
>> default. That's it. Add support for the various PAE MMU options.
> 
> "That's it". "Add support for". Do you really think it's a thing simple
> enough to sum up in a few words?

That "add support for the various PAE MMU options" is not just a few
words, of course. Waste of time? Maybe. If 4GB is not enough upgrade the
hardware and run amd64.

> 
> Last time steps in this direction were attempted, i386 was subtly broken
> on AMD CPUs for months.
> 
>>   My current daily is a Lenovo x240 with 8GB of RAM running amd64
>> and this thing is swapping like mad. Throw a 32 bit OS at it supporting
>> those 8GB of RAM and go for it. Why would anyone throw away such a
>> machine, just because a 64 bit OS hits its boundaries, when a 32 bit OS
>> would not?
> 
> And then ASLR would be seriously limited, because of the low amount of
> address space per process. And it's hard to predict how usable it would
> actually be, especially on an OS that uses PIE widely, due to the lower
> number of registers.
>

Running i386 on a CPU supporting amd64 makes no sense, I admit. Last
time running i386 is more than a decade ago. Step one for me currently is:

1. Am I the only one experiencing this 8GB of RAM is not enough for an
amd64 laptop just because Firefox with a few open tabs and Thunderbird
running in parallel will make it swap? The answer seems to be yes. Quite
confused right now. I upgraded RAM from 4GB to

spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM PC3-12800 SO-DIMM

two weeks ago. That's why the subject caught my attention. Still swaps
but cannot add more than 8GB to that machine. What now? Have some fun
with KiCAD? No - buy a new laptop. How on earth can 8GB physical RAM not
be enough for browsing the web and doing email? I must be doing
something seriously wrong.

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/15/24 12:45, Claudio Jeker wrote:
> On Tue, Oct 15, 2024 at 12:28:20PM +0200, Christian Schulte wrote:
>> On 10/15/24 12:09, Stuart Henderson wrote:
>>> On 2024-10-15, Zé Loff  wrote:
>>>> On Tue, Oct 15, 2024 at 10:14:42AM +0200, Christian Schulte wrote:
>>>>> ulimit -d `ulimit -aH | grep data | awk '{print $2}'`
>>>>> ulimit -n `ulimit -aH | grep nofiles | awk '{print $2}'`
>>>
>>> ulimit -d `ulimit -dH` etc... but then there's no point setting a
>>> separate hard limit in login.conf.
>>
>> Of course. I am the only user on that system and the only limits I want
>> "my" xsession to be in effect on that system are the hard limits setup
>> by the kernel. Those make the system swap for no apparent reasons. So.
>> Why is this thing swapping?
> 
> Because you are out of memory (most probably the usual amd64 problem of
> running out of dma reachable memory and the pagedaemon going berserk about
> that). You have plenty of ram just in the wrong spot.
> 

According to the readings of top(1) or vmstat(8) I am not hitting any
physical RAM limits. Still. The system starts swapping and I am yet to
find out why it does. Maybe it just cannot fulfill requests for larger
chunks of memory but does not "tell" an application about it and just
commits itself to swapping? Makes no sense to me reading output of
top(1) or vmstat(8) displaying that the system has swapped out more than
half a GB to disk when nearly half of the RAM available to the system
(8GB) is not even wired up. The system reports nearly 4GB of physical
RAM available for allocation together with more than half of a GB
swapped out to disk. Makes no sense.

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/15/24 09:51, Christian Schulte wrote:
> On 10/14/24 15:49, Stuart Henderson wrote:
>> On 2024-10-14, Christian Schulte  wrote:
>>> On 10/14/24 10:33, Stuart Henderson wrote:
>>>> On 2024-10-12, Christian Schulte  wrote:
>>>>> Take i386. Compile it with something -march=i686 or pentiumpro by
>>>>> default. That's it. Add support for the various PAE MMU options.
>>>>
>>>> "That's it". "Add support for". Do you really think it's a thing simple
>>>> enough to sum up in a few words?
>>>
>>> That "add support for the various PAE MMU options" is not just a few
>>> words, of course. Waste of time? Maybe. If 4GB is not enough upgrade the
>>> hardware and run amd64.
>>>
>>>>
>>>> Last time steps in this direction were attempted, i386 was subtly broken
>>>> on AMD CPUs for months.
>>>>
>>>>>   My current daily is a Lenovo x240 with 8GB of RAM running amd64
>>>>> and this thing is swapping like mad. Throw a 32 bit OS at it supporting
>>>>> those 8GB of RAM and go for it. Why would anyone throw away such a
>>>>> machine, just because a 64 bit OS hits its boundaries, when a 32 bit OS
>>>>> would not?
>>>>
>>>> And then ASLR would be seriously limited, because of the low amount of
>>>> address space per process. And it's hard to predict how usable it would
>>>> actually be, especially on an OS that uses PIE widely, due to the lower
>>>> number of registers.
>>>>
>>>
>>> Running i386 on a CPU supporting amd64 makes no sense, I admit.
>>
>> i386 snapshot packages would take a _lot_ longer to build if I had to do
>> that on hardware which does not support amd64...
>>
>>> Last
>>> time running i386 is more than a decade ago. Step one for me currently is:
>>>
>>> 1. Am I the only one experiencing this 8GB of RAM is not enough for an
>>> amd64 laptop just because Firefox with a few open tabs and Thunderbird
>>> running in parallel will make it swap? The answer seems to be yes. Quite
>>> confused right now. I upgraded RAM from 4GB to
>>>
>>> spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM PC3-12800 SO-DIMM
>>>
>>> two weeks ago. That's why the subject caught my attention. Still swaps
>>> but cannot add more than 8GB to that machine. What now? Have some fun
>>> with KiCAD? No - buy a new laptop. How on earth can 8GB physical RAM not
>>> be enough for browsing the web and doing email? I must be doing
>>> something seriously wrong.
>>
>> Running Firefox and Thunderbird simultaneously is probably asking
>> a lot from 8GB.
>>
>> I wouldn't consider less than 16GB for a laptop now. For that,
>> Thinkpad-wise you'll need at least X250 (if you can find one where
>> the machine hasn't been destroyed with extreme prejudice due to the
>> absolutely terrible clunkpad) or more likely X260, if not one of the
>> newer ones - X240 is single slot and I don't think Intel released a
>> MRC for the generation of CPUs used in X240 that supports 16GB DIMMs.
>> (https://en.wikipedia.org/wiki/Memory_Reference_Code)
>>
> 
> I'd first like to find out why this system is swapping at all. I managed
> to issue top(1) this time, when the system was locked up due to
> swapping. Here are the readings I could capture - issuing top(1) took
> nearly a minute to come up:
> 
> load averages:  0.04,  0.35,  0.54
> 
>   x500.schulte.it 09:44:01
> 70 processes: 68 idle, 2 on processor
> 
> up 4 days 01:17:15
> CPU0 states:  9.0% user,  0.0% nice,  3.2% sys,  0.2% spin,  1.0% intr,
> 86.6% idle
> CPU2 states:  6.9% user,  0.0% nice,  3.5% sys,  0.2% spin,  0.0% intr,
> 89.4% idle
> Memory: Real: 2867M/4129M act/tot Free: 3483M Cache: 277M Swap: 718M/4230M
> 
> That's more than half a GB of swapped out memory, although there are
> nearly 4GB of memory reported as free. So why does it start swapping,
> locking up the whole system from time to time, although there is nearly
> 4GB of free unwired RAM available?
> 

Maybe I did something wrong when still running i386. I always copied my
home directory over to new systems and moved the hard disk a couple of
times from one laptop to the other. This is, what I found in my .xession
file. Last time I touched that file is more than a decade

Re: Memory upgrade

[Christian Schulte]

On 10/15/24 15:09, Claudio Jeker wrote:
> On Tue, Oct 15, 2024 at 02:35:03PM +0200, Christian Schulte wrote:
>> On 10/15/24 12:45, Claudio Jeker wrote:
>>> On Tue, Oct 15, 2024 at 12:28:20PM +0200, Christian Schulte wrote:
>>>> On 10/15/24 12:09, Stuart Henderson wrote:
>>>>> On 2024-10-15, Zé Loff  wrote:
>>>>>> On Tue, Oct 15, 2024 at 10:14:42AM +0200, Christian Schulte wrote:
>>>>>>> ulimit -d `ulimit -aH | grep data | awk '{print $2}'`
>>>>>>> ulimit -n `ulimit -aH | grep nofiles | awk '{print $2}'`
>>>>>
>>>>> ulimit -d `ulimit -dH` etc... but then there's no point setting a
>>>>> separate hard limit in login.conf.
>>>>
>>>> Of course. I am the only user on that system and the only limits I want
>>>> "my" xsession to be in effect on that system are the hard limits setup
>>>> by the kernel. Those make the system swap for no apparent reasons. So.
>>>> Why is this thing swapping?
>>>
>>> Because you are out of memory (most probably the usual amd64 problem of
>>> running out of dma reachable memory and the pagedaemon going berserk about
>>> that). You have plenty of ram just in the wrong spot.
>>>
>>
>> According to the readings of top(1) or vmstat(8) I am not hitting any
>> physical RAM limits. Still. The system starts swapping and I am yet to
>> find out why it does. Maybe it just cannot fulfill requests for larger
>> chunks of memory but does not "tell" an application about it and just
>> commits itself to swapping? Makes no sense to me reading output of
>> top(1) or vmstat(8) displaying that the system has swapped out more than
>> half a GB to disk when nearly half of the RAM available to the system
>> (8GB) is not even wired up. The system reports nearly 4GB of physical
>> RAM available for allocation together with more than half of a GB
>> swapped out to disk. Makes no sense.
>>
> 
> Please read again. You are out of memory below 4GB (dma reachable physical
> memory). The pagedaemon does a very poor job in that case and this is what
> you see. It is a known issue and a fix will eventually emerge.
> 
> If the problem was trivial it would have been fixed already.

I am not around here for working on things a chimpanzee could be trained
to do.

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/14/24 15:49, Stuart Henderson wrote:
> On 2024-10-14, Christian Schulte  wrote:
>> On 10/14/24 10:33, Stuart Henderson wrote:
>>> On 2024-10-12, Christian Schulte  wrote:
>>>> Take i386. Compile it with something -march=i686 or pentiumpro by
>>>> default. That's it. Add support for the various PAE MMU options.
>>>
>>> "That's it". "Add support for". Do you really think it's a thing simple
>>> enough to sum up in a few words?
>>
>> That "add support for the various PAE MMU options" is not just a few
>> words, of course. Waste of time? Maybe. If 4GB is not enough upgrade the
>> hardware and run amd64.
>>
>>>
>>> Last time steps in this direction were attempted, i386 was subtly broken
>>> on AMD CPUs for months.
>>>
>>>>   My current daily is a Lenovo x240 with 8GB of RAM running amd64
>>>> and this thing is swapping like mad. Throw a 32 bit OS at it supporting
>>>> those 8GB of RAM and go for it. Why would anyone throw away such a
>>>> machine, just because a 64 bit OS hits its boundaries, when a 32 bit OS
>>>> would not?
>>>
>>> And then ASLR would be seriously limited, because of the low amount of
>>> address space per process. And it's hard to predict how usable it would
>>> actually be, especially on an OS that uses PIE widely, due to the lower
>>> number of registers.
>>>
>>
>> Running i386 on a CPU supporting amd64 makes no sense, I admit.
> 
> i386 snapshot packages would take a _lot_ longer to build if I had to do
> that on hardware which does not support amd64...
> 
>> Last
>> time running i386 is more than a decade ago. Step one for me currently is:
>>
>> 1. Am I the only one experiencing this 8GB of RAM is not enough for an
>> amd64 laptop just because Firefox with a few open tabs and Thunderbird
>> running in parallel will make it swap? The answer seems to be yes. Quite
>> confused right now. I upgraded RAM from 4GB to
>>
>> spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM PC3-12800 SO-DIMM
>>
>> two weeks ago. That's why the subject caught my attention. Still swaps
>> but cannot add more than 8GB to that machine. What now? Have some fun
>> with KiCAD? No - buy a new laptop. How on earth can 8GB physical RAM not
>> be enough for browsing the web and doing email? I must be doing
>> something seriously wrong.
> 
> Running Firefox and Thunderbird simultaneously is probably asking
> a lot from 8GB.
> 
> I wouldn't consider less than 16GB for a laptop now. For that,
> Thinkpad-wise you'll need at least X250 (if you can find one where
> the machine hasn't been destroyed with extreme prejudice due to the
> absolutely terrible clunkpad) or more likely X260, if not one of the
> newer ones - X240 is single slot and I don't think Intel released a
> MRC for the generation of CPUs used in X240 that supports 16GB DIMMs.
> (https://en.wikipedia.org/wiki/Memory_Reference_Code)
> 

I'd first like to find out why this system is swapping at all. I managed
to issue top(1) this time, when the system was locked up due to
swapping. Here are the readings I could capture - issuing top(1) took
nearly a minute to come up:

load averages:  0.04,  0.35,  0.54

  x500.schulte.it 09:44:01
70 processes: 68 idle, 2 on processor

up 4 days 01:17:15
CPU0 states:  9.0% user,  0.0% nice,  3.2% sys,  0.2% spin,  1.0% intr,
86.6% idle
CPU2 states:  6.9% user,  0.0% nice,  3.5% sys,  0.2% spin,  0.0% intr,
89.4% idle
Memory: Real: 2867M/4129M act/tot Free: 3483M Cache: 277M Swap: 718M/4230M

That's more than half a GB of swapped out memory, although there are
nearly 4GB of memory reported as free. So why does it start swapping,
locking up the whole system from time to time, although there is nearly
4GB of free unwired RAM available?

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/15/24 12:09, Stuart Henderson wrote:
> On 2024-10-15, Zé Loff  wrote:
>> On Tue, Oct 15, 2024 at 10:14:42AM +0200, Christian Schulte wrote:
>>> ulimit -d `ulimit -aH | grep data | awk '{print $2}'`
>>> ulimit -n `ulimit -aH | grep nofiles | awk '{print $2}'`
> 
> ulimit -d `ulimit -dH` etc... but then there's no point setting a
> separate hard limit in login.conf.

Of course. I am the only user on that system and the only limits I want
"my" xsession to be in effect on that system are the hard limits setup
by the kernel. Those make the system swap for no apparent reasons. So.
Why is this thing swapping?

> 
>>> data(kbytes) 134217728
>>
>> That's 128 GB.
> ...
>> I have no idea what stating "you can use 128GB of memory on this 8GB RAM
>> + 4GB swap machine" does to the system's memory management, but I
>> wouldn't be surprised if weird things happen.

Same for me. This is the default hard limit on that system without me
having touched anything.

> 
> IIRC some things do use ulimit -d as a hint to how much memory they can
> allocate.
> 
>>> Still. this does not explain why the system is swapping, although half
>>> of the physical RAM is not even wired.
> 
> just read /sys/uvm ;)

I am reading that already. That thing should not swap. It does. Some
application may well not have been that nice to the memory allocator.
The Java VM seems to have this gotten right. Fingers crossed.

> 
> (btw if you're not on -current, you may want uvm_pdaemon.c r1.116 if you are
> low on memory).
> 

Will take a look at it. Thank you.

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/10/24 00:07, Zbigniew Kossowski wrote:
> I have 2 HP Proliant miniserver and upgraded RAM from 1x4GB to 2x8GB (by
> YT user).
> 
> First server: 
> BIOS sees 16GB, dmesg sees 16GB spdmem0 and 1 but only 4GB of RAM
> 
> # dmesg |grep mem
> real mem  = 3622907904 (3455MB)
> avail mem = 3539423232 (3375MB)
> spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM PC3-12800 with thermal sensor
> spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM PC3-12800 with thermal sensor
> 
> I have upgraded from 7.4 i386 to 7.5 amd64 due to the fact that server
> has AMD Turion processor.
> This made a job.
> 
> # dmesg |grep mem
> real mem = 17028415488 (16239MB)
> avail mem = 16491167744 (15727MB)
> spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM PC3-12800
> spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM PC3-12800
> 
> In a second server I have upgraded from 7.5 i386 to 7.6 i386 but server
> sees only 4GB of RAM

I think it is normal for i386 to have a maximum limit of 4GB. I do not
know about any real i386 CPUs which could address more than those 4GB
physically. They had a 32bit address bus anyways. Addressing more than
what the address bus is capable of would mean some kind of bank
switching to be performed including driver support. I may be wrong about
it. As long as you are running i386, 4GB is the maximum limit. No?

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/11/24 13:57, Stuart Henderson wrote:
> On 2024-10-09, obs...@loopw.com  wrote:
>>> In a second server I have upgraded from 7.5 i386 to 7.6 i386 but server 
>>> sees only 4GB of RAM
>>
>>> Is anybody with similar experiences? Any ideas how to fix RAM?
>>
>> run 64bit OpenBSD
>>
>> 32bit address space is limited to a max of 4GB**, and some of that is eaten 
>> up by PCI devices, etc.
>> ** PAE can work around this, but I’m not sure if OpenBSD supports PAE at all 
>> (and theres other issues/caveats with using PAE of course, including speed 
>> and security)
> 
> OpenBSD does support PAE on i386, but not for increasing address space,
> just for W^X. See /sys/arch/i386/i386/pmapae.c r1.28.
> 
> 

Hmm. Why not give up on i386 and make that i686 instead (Pentium Pro)?
What I mean by this. Rename the current i386 to i686 by compiling it for
i686 and update the pmap to support more than 4GB physical RAM (PAE).
Supporting i386 is like supporting 68030.

-- 
Christian

Start the kernel under a debugger

[Christian Schulte]

Hi @misc,

could someone point me to some documentation on how to do the following:
Set a breakpoint on e.g. main() in sys/kern/init_main.c or even earlier
in locore.S and start the kernel under a debugger to be able to step
through the early initialization stages. Searching the web yields quite
some kind of tutorials. Mainly mentioning kgdb but that got removed in
6.2. Could be qemu. I do not really care. Read ddb(4) already but that
is a bit too late, when the kernel already is running. Of course I would
like to have source level debugging available to see the actual sources
being executed. Last time I did something like this was on a Atari
Falcon 030 using FreeMINT. So there is a lot I have to catch up with.
Any pointers highly appreciated. Thank you.

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/16/24 22:37, Thomas Frohwein wrote:
> On Tue, 15 Oct 2024 16:08:03 +0200
> Christian Schulte  wrote:
> 
>> On 10/15/24 15:09, Claudio Jeker wrote:
>>> On Tue, Oct 15, 2024 at 02:35:03PM +0200, Christian Schulte wrote:  
>>>> On 10/15/24 12:45, Claudio Jeker wrote:  
>>>>> On Tue, Oct 15, 2024 at 12:28:20PM +0200, Christian Schulte
>>>>> wrote:  
>>>>>> On 10/15/24 12:09, Stuart Henderson wrote:  
>>>>>>> On 2024-10-15, Zé Loff  wrote:  
>>>>>>>> On Tue, Oct 15, 2024 at 10:14:42AM +0200, Christian Schulte
>>>>>>>> wrote:  
>>>>>>>>> ulimit -d `ulimit -aH | grep data | awk '{print $2}'`
>>>>>>>>> ulimit -n `ulimit -aH | grep nofiles | awk '{print $2}'`  
>>>>>>>
>>>>>>> ulimit -d `ulimit -dH` etc... but then there's no point setting
>>>>>>> a separate hard limit in login.conf.  
>>>>>>
>>>>>> Of course. I am the only user on that system and the only limits
>>>>>> I want "my" xsession to be in effect on that system are the hard
>>>>>> limits setup by the kernel. Those make the system swap for no
>>>>>> apparent reasons. So. Why is this thing swapping?  
>>>>>
>>>>> Because you are out of memory (most probably the usual amd64
>>>>> problem of running out of dma reachable memory and the pagedaemon
>>>>> going berserk about that). You have plenty of ram just in the
>>>>> wrong spot. 
>>>>
>>>> According to the readings of top(1) or vmstat(8) I am not hitting
>>>> any physical RAM limits. Still. The system starts swapping and I
>>>> am yet to find out why it does. Maybe it just cannot fulfill
>>>> requests for larger chunks of memory but does not "tell" an
>>>> application about it and just commits itself to swapping? Makes no
>>>> sense to me reading output of top(1) or vmstat(8) displaying that
>>>> the system has swapped out more than half a GB to disk when nearly
>>>> half of the RAM available to the system (8GB) is not even wired
>>>> up. The system reports nearly 4GB of physical RAM available for
>>>> allocation together with more than half of a GB swapped out to
>>>> disk. Makes no sense. 
>>>
>>> Please read again. You are out of memory below 4GB (dma reachable
>>> physical memory). The pagedaemon does a very poor job in that case
>>> and this is what you see. It is a known issue and a fix will
>>> eventually emerge.
>>>
>>> If the problem was trivial it would have been fixed already.  
>>
>> I am not around here for working on things a chimpanzee could be
>> trained to do.
>>
> 
> You are overstepping and have been for a while. If you want any help,
> better watch your tone.
> 

Sorry. Next time I am going to reach out on any lists here, I will save
the mail as a draft, take a sleep and read it again before sending.
Seems a lot of devs around here know each other personally. That's of
course a very big advantage. That line above from me really is an
insider joke I did not even notice not well known. It should have made
you laugh. It did not. My fault.

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/15/24 20:29, Kirill A. Korinsky wrote:
> On Tue, 15 Oct 2024 12:28:20 +0200,
> Christian Schulte  wrote:
>>
>> On 10/15/24 12:09, Stuart Henderson wrote:
>>> On 2024-10-15, Zé Loff  wrote:
>>>> On Tue, Oct 15, 2024 at 10:14:42AM +0200, Christian Schulte wrote:
>>>>> ulimit -d `ulimit -aH | grep data | awk '{print $2}'`
>>>>> ulimit -n `ulimit -aH | grep nofiles | awk '{print $2}'`
>>>
>>> ulimit -d `ulimit -dH` etc... but then there's no point setting a
>>> separate hard limit in login.conf.
>>
>> Of course. I am the only user on that system and the only limits I want
>> "my" xsession to be in effect on that system are the hard limits setup
>> by the kernel. Those make the system swap for no apparent reasons. So.
>> Why is this thing swapping?
>>
>>>
>>>>> data(kbytes) 134217728
>>>>
>>>> That's 128 GB.
>>> ...
>>>> I have no idea what stating "you can use 128GB of memory on this 8GB RAM
>>>> + 4GB swap machine" does to the system's memory management, but I
>>>> wouldn't be surprised if weird things happen.
>>
>> Same for me. This is the default hard limit on that system without me
>> having touched anything.
>>
> 
> If I assume that your user is member of staff and you use default
> login.conf, when the following settings applies:
> 
> staff:\
> :datasize-cur=1536M:\
> :datasize-max=infinity:\
> ...
> 
> here, infinity means that you're using up to MAXDSIZ bytes.
> 
> I've checked vmparam.h and it is indeed 128Gb for amd64, but for i386 such
> value is way lower: 3Gb.
> 
> Have I miss something?
> 

No. That's what seems to went wrong when going from i386 to amd64. The
3GB hard limit of i386 was in the range of available physical memory
(4GB) without swap. The 128GB on amd64 do not. Changing this to what
"memory" reads, makes no difference, though.

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/15/24 18:38, Stefan Sperling wrote:
> On Tue, Oct 15, 2024 at 04:08:03PM +0200, Christian Schulte wrote:
>> On 10/15/24 15:09, Claudio Jeker wrote:
>>> If the problem was trivial it would have been fixed already.
>>
>> I am not around here for working on things a chimpanzee could be trained
>> to do.
> 
> Ouch, you stepped on a big pile of poo right there Christian.
> 
> If you want people to take anything you say and report and ask
> for help with seriously, do not say things like that at their face.
> 
> You have no idea how much effort claudio and mpi and others have
> already put into this DMA resource issue. It's been going on at
> least since the Toulouse hackathon in May 2023. Probably earlier,
> I recall it coming up at the Bruges hackathon in Summer 2022.

This was not meant offensive in any way. I just wanted to express that
if something is not trivial, this does not mean it's not worth working
on. Just a misunderstanding on my side. I understood this to mean, it's
not trivial, so please go away asking for things not trivial. So to say.
That's the internet. No gestures, no mimics. Just words. Exactly. I have
no idea how much effort anyone has put into anything.

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/17/24 09:40, Stuart Henderson wrote:
> On 2024-10-16, Christian Schulte  wrote:
>>
>> No. That's what seems to went wrong when going from i386 to amd64.
>> The 3GB hard limit of i386 was in the range of available physical
>> memory (4GB) without swap. The 128GB on amd64 do not. Changing this
>> to what "memory" reads, makes no difference, though.
> 
> A 32-bit architecture allows for 2^32 bytes, or 4GB, of address space
> in a process. Memory addresses are stored in 32-bit variables - a
> single process can't use memory beyond this. Even on OS which use PAE
> to allow for larger _overall_ memory, and regardless of whether that
> memory is RAM or swap space on disk, the maximum that a single 32-bit
> process can use is still 4GB.
>

Could really need some guidance on this. Trying to make the mails as
small as possible.

We are talking about this commit, right?

<https://github.com/openbsd/src/commit/b426ab7bc6c256bfb7af9a9f082a20534b39a28b>

Reading all of this takes a lot of time. So I "switched" to a brute
force approach. Make it panic and drop to the debugger. This diff:


Index: sys/arch/amd64/amd64/machdep.c
===
RCS file: /cvs/src/sys/arch/amd64/amd64/machdep.c,v
retrieving revision 1.297
diff -u -p -u -r1.297 machdep.c
--- sys/arch/amd64/amd64/machdep.c  21 Sep 2024 19:06:07 -  1.297
+++ sys/arch/amd64/amd64/machdep.c  17 Oct 2024 10:55:05 -
@@ -214,7 +214,9 @@ struct vm_map *phys_map = NULL;
 
 /* UVM constraint ranges. */
 struct uvm_constraint_range  isa_constraint = { 0x0, 0x00ffUL };
-struct uvm_constraint_range  dma_constraint = { 0x0, 0xUL };
+// struct uvm_constraint_range  dma_constraint = { 0x0, 0xUL };
+struct uvm_constraint_range  dma_constraint = { 0x0, (paddr_t)-1 };
+
 struct uvm_constraint_range *uvm_md_constraints[] = {
 &isa_constraint,
 &dma_constraint,

I expected the system to immediately panic and drop to the debugger but
it does not. I then tried to make it swap again. I could open various
tabs in firefox, run thunderbird in parallel and it does not swap as
before. I then tried to make it swap by opening up various other
applications. Libreoffice. Still does not swap. I then started Gimp,
selected all images in an images directory and opened them all to make
it swap. It then swapped, of course. But it is now using much more
physical ram before starting to swap. Feels like a different system.
Never could open that many applications in parallel before. Video,
audio, USB mouse, keyboard is working. Here is an output of pcidump -v:

Domain /dev/pci0:
 0:0:0: Intel Core 4G Host
0x: Vendor ID: 8086, Product ID: 0a04
0x0004: Command: 0006, Status: 2090
0x0008: Class: 06 Bridge, Subclass: 00 Host,
Interface: 00, Revision: 0b
0x000c: BIST: 00, Header Type: 00, Latency Timer: 00,
Cache Line Size: 00
0x0010: BAR empty ()
0x0014: BAR empty ()
0x0018: BAR empty ()
0x001c: BAR empty ()
0x0020: BAR empty ()
0x0024: BAR empty ()
0x0028: Cardbus CIS: 
0x002c: Subsystem Vendor ID: 17aa Product ID: 2214
0x0030: Expansion ROM Base Address: 
0x0038: 
0x003c: Interrupt Pin: 00 Line: 00 Min Gnt: 00 Max Lat: 00
0x00e0: Capability 0x09: Vendor Specific
 0:2:0: Intel HD Graphics
0x: Vendor ID: 8086, Product ID: 0a16
0x0004: Command: 0007, Status: 0090
0x0008: Class: 03 Display, Subclass: 00 VGA,
Interface: 00, Revision: 0b
0x000c: BIST: 00, Header Type: 00, Latency Timer: 00,
Cache Line Size: 00
0x0010: BAR mem 64bit addr: 0xf000/0x0040
0x0018: BAR mem prefetchable 64bit addr: 0xe000/0x1000
0x0020: BAR io addr: 0x3000/0x0040
0x0024: BAR empty ()
0x0028: Cardbus CIS: 
0x002c: Subsystem Vendor ID: 17aa Product ID: 2214
0x0030: Expansion ROM Base Address: 
0x0038: 
0x003c: Interrupt Pin: 01 Line: 0b Min Gnt: 00 Max Lat: 00
0x0090: Capability 0x05: Message Signalled Interrupts (MSI)
Enabled: yes; 1 vectors (1 enabled)
0x00d0: Capability 0x01: Power Management
State: D0
0x00a4: Capability 0x13: PCI Advanced Features
 0:3:0: Intel Core 4G HD Audio
0x: Vendor ID: 8086, Product ID: 0a0c
0x0004: Command: 0006, Status: 0010
0x0008: Class: 04 Multimedia, Subclass: 03 HD Audio,
Interface: 00, Revision: 0b
0x000c: BIST: 00, Header Type: 00, Latency Timer: 00,
Cache Line Size: 10
0x0010: BAR mem 64bit addr: 0xf063/0x400

Re: Minimum supported chipsets of amd64

[Christian Schulte]

On 10/28/24 22:53, Anon Loli wrote:
> On Mon, Oct 28, 2024 at 05:35:47PM +0100, Christian Schulte wrote:
>> On 10/24/24 03:01, Mike Larkin wrote:
>>>
>>> Every one of us who has worked in this area, at this level, has read those
>>> 800+ page documents. Sometimes they are many thousands of pages (eg the 
>>> latest
>>> Intel SDM or latest ACPI spec).
>>>
>>> Tell us what you are doing and what you want to know and maybe we can point
>>> you to the right docs, but there is no short-cutting reading the reference
>>> manuals.
>>>
>>
>> I would really like to understand why this architecture stood the test
>> of time. Just because it boots in 8 bit CPU mode from the 70ties not
>> even capable of beating a 6502? Just because developers were not
>> continuously forced to throw away all knowledge and could build upon it?
>> Seems to be the reason. Intel tried to throw away legacy burdens and got
>> set straight by AMD. I am currently approaching page 4000 of
>> documentation. Shaking heads. Unbelievable. What I am lacking so far is
>> a current PCI bus specification. This seems to not be available to non
>> members who I am certainly not. Coming from a hardware background,
>> documents like this
>>
>> <https://www.intel.sg/content/dam/doc/datasheet/io-controller-hub-10-family-datasheet.pdf>
>>
>> clearly were a waste of time, at least when your goal is not to produce
>> mainboards. Well. Normally you would program devices directly. It even
>> contains write-once-by-firmware registers. It will take some time for me
>> to understand the reasoning behind this. Not questioning there are no
>> reasons for doing it that way. I am just trying to make me stop hating
>> that architecture. I am still failing at this task but I would like to
>> overcome this. At least it has linear address space. Oh. What a wonder.
>> Every 68k had this decades ago. Oh sorry. Your comments are very helpful
>> to me so far so thank you. Because the machine independent parts in the
>> kernel really are abstractions of formerly machine dependent parts,
>> understanding the worst case of those - namely x86 and amd64 - will help
>> me understand those. I am still in the process of reading x86/amd64
>> documentation even if it make me shake my head every so often.
>>
>> Regards,
>> -- 
>> Christian
> 
> I just wrote a whole big-ass e-mail about how hardware has been shit for
> decades now.
> I do not feel like rewriting all of it right now it was a genius e-mail.
> 
> I fucking hate when my e-mail client goes bananas because it's terminal based.
> Fuck escape sequences and stupid retarded Unix.
> When do escape sequences actually work as intended? When?
> 
> Anyways suckless.org rocks, and should be implied to hardware.
> 
> Open Source is Insufficient to Solve Trust Problems in Hardware
> https://youtube.com/watch?v=Hzb37RyagCQ
> 
> How do you know the hardware in front of you actually conforms to the hardware
> design you might or might not have?
> You can't, it's not like software, at least you can't with existing hardware,
> watch the video.
> 
> Mud towers build on mud foundation are still mud and will collapse under mud.
> 
> This was more-less the important stuff
> Fuck I hate re-writing emails fuck me!
> 

Fuck. Ass. Genius. You maybe want to watch the Youtube Channels of Ben
Eater [1] or James Sharman [2] for a starting point talking about
hardware and how to build a CPU from scratch using bread boards. Fuck.
Ass. Genius. Then start reading about what microelectronics is about or
even get a degree in microelectronics. Fuck. Ass. Genius.

[1] <https://www.youtube.com/@BenEater/playlists>
[2] <https://www.youtube.com/@weirdboyjim/playlists>

-- 
Christian

Re: long-running future of OpenBSD

[Christian Schulte]

On 10/21/24 15:26, notpeter87 notpeter87 wrote:
> if anything would happen to Theo de Raadt, he is 56 years old according
> to wikipedia,
> 
> what would happen to OpenBSD?

If anything would happen to Linus Torvalds, he is 55 years old according
to wikipedia.

What would happen to Linux?

If anything would happen to Richard Stallman, he is 71 years old
according to wikipedia.

What would happen to GNU?

You would not ask those questions, would you?

-- 
Christian

Re: Minimum supported chipsets of amd64

[Christian Schulte]

On 10/29/24 04:49, Philip Guenther wrote:
> On Mon, Oct 28, 2024 at 11:43 AM Christian Schulte  wrote:
> ...
>> I would really like to understand why this architecture stood the test
>> of time. Just because it boots in 8 bit CPU mode from the 70ties not
>> even capable of beating a 6502?
> <...>
> 
> Building a base of users with computers capable of running a more
> powerful ISA/OS/whatever (while it still supports their existing
> applications), so that application writers believe that there will be
> sufficient user base with that capability to use software written to
> use that power, which drives more people to get those more capable
> computers, has been a huge driver of not just the evolution of x86 but
> of the computer industry as a whole.  x86 made those steps easy for a
> line of ISA evolution; Apple went a different direction and put the
> backwards/forwards compat into their build tools so you could build on
> one arch/OS-version and move to a different one and did that so well
> that they managed to move user bases from m68k to powerpc to x86 to
> arm64 with compat across each transition.  You _do_ understand that
> the set of people who can rebuild all the software they directly use
> is *tiny* and the subset of those for whom doing so is a net positive
> use of the limited time they have between birth is death is
> insubstantial, yes?
> 
> For Linus's thoughts on the amd64 transition, consider
> https://yarchive.net/comp/amd64.html

He is more or less speaking about transitioning from 32bit to 64bit in
general there. Of course for someone having e.g. a Windows installation
on a hard disk, being able to just replace a e.g. broken mainboard with
a new one and being able to just boot the existing installation with
that without having to re-install everything may be of great value. Is
that really required and justifies backwards compatibility for more than
4 decades? You could run e.g. DOS on a smartphone these days, for
example. No need for the CPU to support all those obsolete modes. What
do I know? It is, what it is. So I just need to deal with it.

> 
> 
> If the above doesn't have you going "I see how others could value
> something that I found inscrutable" then I don't think I can help you
> any further in that understanding.
> 
> 
> Philip Guenther

Maybe I am just failing the definition of "general purpose" those not so
general purpose registers and instructions are called. Who knows? Maybe
it's just brilliant to need to know what instructions can be used with
what registers and what addressing modes. I read everything from mbr to
init_x86_64(paddr_t first_avail) without committing suicide so far.

-- 
Christian

Re: Configuring IPv6 addresses using dhcp6leased(8)

[Christian Schulte]

On 11/2/24 16:49, Peter Hessler wrote:
> What does the full output of 'slaacctl show interface iwm0', 'ifconfig iwm0',
> and  'netstat -rnf inet6' say?

Disabling the DHCPv6 server in the fritz box seems to do the job. Every device
now gets a working IPv6 configuration using SLAAC. Is there anything special
I need to add to pf.conf(5) to make IPv6 work? Currently IPv6 only works when
pf(4) is disabled. Following script also contains the contents of the current
pf.conf(5) file. Neither the FAQ nor the manpages contain anything special for
IPv6 and I do not see the reason the following pf.conf(5) file will not behave
the same for IPv6 as for IPv4. Any hints highly appreciated.


Script started on Tue Nov  5 09:48:16 2024
x500# pfctl -d
pfctl: pf not enabled
x500# slaacctl show interface iwm0
iwm0:
 index:   2 running: yes temporary: yes
lladdr: e8:b1:fc:51:73:7c
 inet6: fe80::eab1:fcff:fe51:737c%iwm0
Router Advertisement from fe80::6b4:feff:fe18:9cab%iwm0
received: 2024-11-05 09:48:25; 2s ago
Cur Hop Limit: 255, M: 0, O: 0, Router Lifetime:  1800s
Default Router Preference: High
Reachable Time: 0ms, Retrans Timer: 0ms
MTU: 1492 bytes
prefix: fd20:3eea:92dd::/64
On-link: 1, Autonomous address-configuration: 1
vltime:   7200, pltime:   3600
prefix: 2001:16b8:8170:200::/64
On-link: 1, Autonomous address-configuration: 1
vltime:   7200, pltime:   3600
rdns: 2001:16b8:8170:200:6b4:feff:fe18:9cab, lifetime: 1200
rdns: fd20:3eea:92dd:0:6b4:feff:fe18:9cab, lifetime: 1200
Address proposals
id:5, state: PROPOSAL_CONFIGURED, temporary: y
vltime:   7200, pltime:   3600, timeout:   3586s
updated: 2024-11-05 09:48:25; 2s ago
2001:16b8:8170:200:b1a3:64d1:e5b4:1ebb, 2001:16b8:8170:200::/64
id:4, state: PROPOSAL_CONFIGURED, temporary: n
vltime:   7200, pltime:   3600, timeout:   3586s
updated: 2024-11-05 09:48:25; 2s ago
2001:16b8:8170:200:c5e3:8aee:2f6e:407c, 2001:16b8:8170:200::/64
id:3, state: PROPOSAL_CONFIGURED, temporary: y
vltime:   7200, pltime:   3600, timeout:   3586s
updated: 2024-11-05 09:48:25; 2s ago
fd20:3eea:92dd:0:f82:3437:a92d:f03e, fd20:3eea:92dd::/64
id:2, state: PROPOSAL_CONFIGURED, temporary: n
vltime:   7200, pltime:   3600, timeout:   3586s
updated: 2024-11-05 09:48:25; 2s ago
fd20:3eea:92dd:0:5e46:169e:4173:a36e, fd20:3eea:92dd::/64
Default router proposals
id:1, state: PROPOSAL_CONFIGURED
router: fe80::6b4:feff:fe18:9cab%iwm0
router lifetime:   1800
Preference: High
updated: 2024-11-05 09:48:25; 2s ago, timeout:   1786s
rDNS proposals
id:6, state: PROPOSAL_CONFIGURED
router: fe80::6b4:feff:fe18:9cab%iwm0
rdns lifetime:   1200
rdns:
2001:16b8:8170:200:6b4:feff:fe18:9cab
fd20:3eea:92dd:0:6b4:feff:fe18:9cab
updated: 2024-11-05 09:48:25; 2s ago, timeout:   1186s
x500# ifconfig iwm0
iwm0: 
flags=a48843
 mtu 1492
lladdr e8:b1:fc:51:73:7c
index 2 priority 4 llprio 3
groups: wlan egress
media: IEEE802.11 autoselect (HT-MCS15 mode 11n)
status: active
ieee80211: nwid FLSTR81WHG6DG chan 11 bssid 04:b4:fe:18:9c:ad 82% 
wpakey wpaprotos wpa2 wpaakms psk wpaciphers ccmp wpagroupcipher ccmp
inet6 fe80::eab1:fcff:fe51:737c%iwm0 prefixlen 64 scopeid 0x2
inet 10.0.0.115 netmask 0x broadcast 10.0.255.255
inet6 fd20:3eea:92dd:0:5e46:169e:4173:a36e prefixlen 64 autoconf pltime 
3593 vltime 7193
inet6 fd20:3eea:92dd:0:175c:a176:d0d4:f714 prefixlen 64 autoconf 
temporary pltime 3384 vltime 6984
inet6 2001:16b8:8170:200:c5e3:8aee:2f6e:407c prefixlen 64 autoconf 
pltime 3593 vltime 7193
inet6 2001:16b8:8170:200:4a3c:e602:d10b:e61e prefixlen 64 autoconf 
temporary pltime 3384 vltime 6984
inet6 fd20:3eea:92dd:0:f82:3437:a92d:f03e prefixlen 64 autoconf 
temporary pltime 3593 vltime 7193
inet6 2001:16b8:8170:200:b1a3:64d1:e5b4:1ebb prefixlen 64 autoconf 
temporary pltime 3593 vltime 7193
x500# cat /etc/pf.conf
#   $OpenBSD: pf.conf,v 1.55 2017/12/03 20:40:04 sthen Exp $
#
# See pf.conf(5) and /etc/examples/pf.conf

set skip on lo

block return# block stateless traffic
pass# establish keep-state

# By default, do not permit remo

Re: Minimum supported chipsets of amd64

[Christian Schulte]

On 10/29/24 04:49, Philip Guenther wrote:
> On Mon, Oct 28, 2024 at 11:43 AM Christian Schulte  wrote:
> ...
>> I would really like to understand why this architecture stood the test
>> of time. Just because it boots in 8 bit CPU mode from the 70ties not
>> even capable of beating a 6502?
> <...>
> 
> Building a base of users with computers capable of running a more
> powerful ISA/OS/whatever (while it still supports their existing
> applications), so that application writers believe that there will be
> sufficient user base with that capability to use software written to
> use that power, which drives more people to get those more capable
> computers, has been a huge driver of not just the evolution of x86 but
> of the computer industry as a whole.  x86 made those steps easy for a
> line of ISA evolution; Apple went a different direction and put the
> backwards/forwards compat into their build tools so you could build on
> one arch/OS-version and move to a different one and did that so well
> that they managed to move user bases from m68k to powerpc to x86 to
> arm64 with compat across each transition.  You _do_ understand that
> the set of people who can rebuild all the software they directly use
> is *tiny* and the subset of those for whom doing so is a net positive
> use of the limited time they have between birth is death is
> insubstantial, yes?

This justifies this tiny group of people to enforce everyone to use
technology outdated for more than 4 decades? Just because their
knowledge would become useless and outdated? Deal with it. Learning
about that architecture is exactly about that kind of wasting time you
are talking about. There is no value in forcing anyone to learn about
how PCs in the late 70s or early 80s used to boot the CPU in what
insanity mode whatsoever. Get over it. Want to run DOS or CP/M. Even
your smartphone is capable of doing so. Advocating i368 or AMD64 as
being the most brilliant invention mankind ever has produced is just
insane. It really is full of design flaws. Even an Intel engineer would
agree to this privately.

-- 
Christian

Configuring IPv6 addresses using dhcp6leased(8)

[Christian Schulte]

Hello misc@,

x500$ cat /etc/hostname.iwm0
nwid "FLSTR81WHG6DG" wpa wpakey "xyz"
inet autoconf
inet6 autoconf

Does "inet6 autoconf" with dhcp6leased(8) already work the same way
"inet autoconf" with dhcpleased(8) works so far? "inet autoconf" makes
my laptop obtain an IPv4 address with dhcpleased(8) from the fritz box.
"inet6 autoconf" does not. Is dhcp6leased(8) already supporting this
kind of client configuration?

Regards,
-- 
Christian

Re: Configuring IPv6 addresses using dhcp6leased(8)

[Christian Schulte]

On 11/2/24 16:49, Peter Hessler wrote:
> 
> What does the full output of 'slaacctl show interface iwm0', 'ifconfig iwm0',
> and  'netstat -rnf inet6' say?


Script started on Sat Nov  2 17:52:32 2024
x500$ slaacctl show interface iwm0
iwm0:
 index:   2 running: yes temporary: yes
lladdr: e8:b1:fc:51:73:7c
 inet6: fe80::eab1:fcff:fe51:737c%iwm0
Router Advertisement from fe80::6b4:feff:fe18:9cab%iwm0
received: 2024-11-01 20:14:47; 77879s ago
Cur Hop Limit: 255, M: 1, O: 1, Router Lifetime:  1800s
Default Router Preference: Medium
Reachable Time: 0ms, Retrans Timer: 0ms
MTU: 1492 bytes
prefix: fd20:3eea:92dd::/64
On-link: 1, Autonomous address-configuration: 1
vltime:   7200, pltime:   3600
prefix: 2001:16b8:814d:7f00::/64
On-link: 1, Autonomous address-configuration: 1
vltime:   7200, pltime:   3600
rdns: 2001:16b8:814d:7f00:6b4:feff:fe18:9cab, lifetime: 1200
rdns: fd20:3eea:92dd:0:6b4:feff:fe18:9cab, lifetime: 1200
x500$ ifconfig iwm0
iwm0: 
flags=a48843
 mtu 1492
lladdr e8:b1:fc:51:73:7c
index 2 priority 4 llprio 3
groups: wlan egress
media: IEEE802.11 autoselect (HT-MCS15 mode 11n)
status: active
ieee80211: nwid FLSTR81WHG6DG chan 11 bssid 04:b4:fe:18:9c:ad 80% 
wpakey wpaprotos wpa2 wpaakms psk wpaciphers ccmp wpagroupcipher ccmp
inet6 fe80::eab1:fcff:fe51:737c%iwm0 prefixlen 64 scopeid 0x2
inet 10.0.0.115 netmask 0x broadcast 10.0.255.255
x500$ netstat -rnf inet6
Routing tables

Internet6:
Destination Gateway 
Flags   Refs  Use   Mtu  Prio Iface
::/96   ::1 
UGRS   00 32768 8 lo0  
::1 ::1 
UHhl  10   20 32768 1 lo0  
:::0.0.0.0/96   ::1 
UGRS   00 32768 8 lo0  
2002::/24   ::1 
UGRS   00 32768 8 lo0  
2002:7f00::/24  ::1 
UGRS   00 32768 8 lo0  
2002:e000::/20  ::1 
UGRS   00 32768 8 lo0  
2002:ff00::/24  ::1 
UGRS   00 32768 8 lo0  
fe80::/10   ::1 
UGRS   0  145 32768 8 lo0  
fec0::/10   ::1 
UGRS   00 32768 8 lo0  
fe80::%iwm0/64  fe80::eab1:fcff:fe51:737c%iwm0  
UCn11 - 8 iwm0 
fe80::6b4:feff:fe18:9cab%iwm0   link#2  
UHLc   0   10 - 7 iwm0 
fe80::eab1:fcff:fe51:737c%iwm0  e8:b1:fc:51:73:7c   
UHLl   00 - 1 iwm0 
fe80::1%lo0 fe80::1%lo0 
UHl00 32768 1 lo0  
ff01::/16   ::1 
UGRS   01 32768 8 lo0  
ff01::%iwm0/32  fe80::eab1:fcff:fe51:737c%iwm0  
Um 04 - 4 iwm0 
ff01::%lo0/32   fe80::1%lo0 
Um 01 32768 4 lo0  
ff02::/16   ::1 
UGRS   01 32768 8 lo0  
ff02::%iwm0/32  fe80::eab1:fcff:fe51:737c%iwm0  
Um 0   15 - 4 iwm0 
ff02::%lo0/32   fe80::1%lo0 
Um 01 32768 4 lo0  
x500$ ^D

Script done on Sat Nov  2 17:53:13 2024

Re: Configuring IPv6 addresses using dhcp6leased(8)

[Christian Schulte]

On 11/2/24 15:32, Brian Conway wrote:
> On Sat, Nov 2, 2024, at 7:09 AM, Christian Schulte wrote:
>> Hello misc@,
>>
>> x500$ cat /etc/hostname.iwm0
>> nwid "FLSTR81WHG6DG" wpa wpakey "xyz"
>> inet autoconf
>> inet6 autoconf
>>
>> Does "inet6 autoconf" with dhcp6leased(8) already work the same way
>> "inet autoconf" with dhcpleased(8) works so far? "inet autoconf"
>> makes
> 
> No, it is used for DHCPv6-PD. See the man pages for
> dhcp6leased[.conf].
> 
> 'inet6 autoconf' is handled by slaacd.

Thank you. So there must be something else I am doing wrong. Why can't
I ping6 IPv6 addresses? Other devices like smartphones or tablets get
different addresses from the fritz box than the OpenBSD laptop and IPv6
is working flawlessly with them. The fritz box seems to only support
DHCPv6 for handing out IPv6 addresses.

Script started on Sat Nov  2 16:17:57 2024
x500$ slaacctl show interface
iwm0:
 index:   2 running: yes temporary: yes
lladdr: e8:b1:fc:51:73:7c
 inet6: fe80::eab1:fcff:fe51:737c%iwm0
Router Advertisement from fe80::6b4:feff:fe18:9cab%iwm0
received: 2024-11-01 20:14:47; 72199s ago
Cur Hop Limit: 255, M: 1, O: 1, Router Lifetime:  1800s
Default Router Preference: Medium
Reachable Time: 0ms, Retrans Timer: 0ms
MTU: 1492 bytes
prefix: fd20:3eea:92dd::/64
On-link: 1, Autonomous address-configuration: 1
vltime:   7200, pltime:   3600
prefix: 2001:16b8:814d:7f00::/64
On-link: 1, Autonomous address-configuration: 1
vltime:   7200, pltime:   3600
rdns: 2001:16b8:814d:7f00:6b4:feff:fe18:9cab, lifetime: 1200
rdns: fd20:3eea:92dd:0:6b4:feff:fe18:9cab, lifetime: 1200
x500$ dig www.openbsd.org 

; <<>> dig 9.10.8-P1 <<>> www.openbsd.org 
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17979
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.openbsd.org.   IN  

;; ANSWER SECTION:
www.openbsd.org.28713   IN  2620:3d:c000:178::80

;; Query time: 1 msec
;; SERVER: 10.0.0.1#53(10.0.0.1)
;; WHEN: Sat Nov 02 16:18:34 CET 2024
;; MSG SIZE  rcvd: 72

x500$ ping6 2620:3d:c000:178::80
PING 2620:3d:c000:178::80 (2620:3d:c000:178::80): 56 data bytes
ping6: sendmsg: No route to host
ping: wrote 2620:3d:c000:178::80 64 chars, ret=-1
ping6: sendmsg: No route to host
ping: wrote 2620:3d:c000:178::80 64 chars, ret=-1
ping6: sendmsg: No route to host
ping: wrote 2620:3d:c000:178::80 64 chars, ret=-1
ping6: sendmsg: No route to host
ping: wrote 2620:3d:c000:178::80 64 chars, ret=-1

--- 2620:3d:c000:178::80 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
x500$ ^D

Script done on Sat Nov  2 16:18:49 2024

-- 
Christian

Minimum supported chipsets of amd64

[Christian Schulte]

Hi misc@,

 mentions this:

"Supported hardware:

Processors

All versions of the AMD Athlon 64 processors and their clones are
supported."

What are the minimum requirements with respect to chipsets?

I see that e.g. QEMU defaults to



This goes back to Pentium II/Pentium Pro. There is not a single real
machine around with such a chipset supporting anything 64 bit. In
addition sys/arch/amd64/include/vmparam.h has this:


#define MAXDSIZ ((paddr_t)128*1024*1024*1024)

That's 128GB.

Could you please point me to the first chipsets supporting those amounts
of memory physically. My understanding is that the so called memory
controller hubs got integrated into the CPUs. So what is the first CPU
supporting those 128GB physically? I am having a hard time getting my
hands on the PC architecture and I would really like to understand why
amd64 supports processor chipset combinations which do not exist in real
life. Currently only reading documentation provided by Intel. Athlon 64
seems to be on the same timeline than Pentium 4. At that time there
where no motherboards around supporting more than 8GB physical memory,
no? For example, I am having difficulties understanding things like

isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12

in the dmesg. Reading the various chipset documentations there, of
course, is no ISA bus anywhere since a very long time and things just
have been produced for compatibility with existing OSes. Enforcing some
ISA constraints on hardware which does not need any of those constraints
is hard to understand for someone never having followed hardware
development since the early 90s. I could really need some pointers on
what documents to read, given they are around 800 pages long. Thank you.

-- 
Christian

Re: Minimum supported chipsets of amd64

[Christian Schulte]

On 10/24/24 03:01, Mike Larkin wrote:
> 
> Every one of us who has worked in this area, at this level, has read those
> 800+ page documents. Sometimes they are many thousands of pages (eg the latest
> Intel SDM or latest ACPI spec).
> 
> Tell us what you are doing and what you want to know and maybe we can point
> you to the right docs, but there is no short-cutting reading the reference
> manuals.
>

I would really like to understand why this architecture stood the test
of time. Just because it boots in 8 bit CPU mode from the 70ties not
even capable of beating a 6502? Just because developers were not
continuously forced to throw away all knowledge and could build upon it?
Seems to be the reason. Intel tried to throw away legacy burdens and got
set straight by AMD. I am currently approaching page 4000 of
documentation. Shaking heads. Unbelievable. What I am lacking so far is
a current PCI bus specification. This seems to not be available to non
members who I am certainly not. Coming from a hardware background,
documents like this



clearly were a waste of time, at least when your goal is not to produce
mainboards. Well. Normally you would program devices directly. It even
contains write-once-by-firmware registers. It will take some time for me
to understand the reasoning behind this. Not questioning there are no
reasons for doing it that way. I am just trying to make me stop hating
that architecture. I am still failing at this task but I would like to
overcome this. At least it has linear address space. Oh. What a wonder.
Every 68k had this decades ago. Oh sorry. Your comments are very helpful
to me so far so thank you. Because the machine independent parts in the
kernel really are abstractions of formerly machine dependent parts,
understanding the worst case of those - namely x86 and amd64 - will help
me understand those. I am still in the process of reading x86/amd64
documentation even if it make me shake my head every so often.

Regards,
-- 
Christian

Re: Minimum supported chipsets of amd64

[Christian Schulte]

On 10/30/24 23:21, Anon Loli wrote:
> On Tue, Oct 29, 2024 at 12:26:54PM +0100, Christian Schulte wrote:
>> On 10/28/24 22:53, Anon Loli wrote:
>>> On Mon, Oct 28, 2024 at 05:35:47PM +0100, Christian Schulte wrote:
>>>> On 10/24/24 03:01, Mike Larkin wrote:
>>>>>
>>>>> Every one of us who has worked in this area, at this level, has read those
>>>>> 800+ page documents. Sometimes they are many thousands of pages (eg the 
>>>>> latest
>>>>> Intel SDM or latest ACPI spec).
>>>>>
>>>>> Tell us what you are doing and what you want to know and maybe we can 
>>>>> point
>>>>> you to the right docs, but there is no short-cutting reading the reference
>>>>> manuals.
>>>>>
>>>>
>>>> I would really like to understand why this architecture stood the test
>>>> of time. Just because it boots in 8 bit CPU mode from the 70ties not
>>>> even capable of beating a 6502? Just because developers were not
>>>> continuously forced to throw away all knowledge and could build upon it?
>>>> Seems to be the reason. Intel tried to throw away legacy burdens and got
>>>> set straight by AMD. I am currently approaching page 4000 of
>>>> documentation. Shaking heads. Unbelievable. What I am lacking so far is
>>>> a current PCI bus specification. This seems to not be available to non
>>>> members who I am certainly not. Coming from a hardware background,
>>>> documents like this
>>>>
>>>> <https://www.intel.sg/content/dam/doc/datasheet/io-controller-hub-10-family-datasheet.pdf>
>>>>
>>>> clearly were a waste of time, at least when your goal is not to produce
>>>> mainboards. Well. Normally you would program devices directly. It even
>>>> contains write-once-by-firmware registers. It will take some time for me
>>>> to understand the reasoning behind this. Not questioning there are no
>>>> reasons for doing it that way. I am just trying to make me stop hating
>>>> that architecture. I am still failing at this task but I would like to
>>>> overcome this. At least it has linear address space. Oh. What a wonder.
>>>> Every 68k had this decades ago. Oh sorry. Your comments are very helpful
>>>> to me so far so thank you. Because the machine independent parts in the
>>>> kernel really are abstractions of formerly machine dependent parts,
>>>> understanding the worst case of those - namely x86 and amd64 - will help
>>>> me understand those. I am still in the process of reading x86/amd64
>>>> documentation even if it make me shake my head every so often.
>>>>
>>>> Regards,
>>>> -- 
>>>> Christian
>>>
>>> I just wrote a whole big-ass e-mail about how hardware has been shit for
>>> decades now.
>>> I do not feel like rewriting all of it right now it was a genius e-mail.
>>>
>>> I fucking hate when my e-mail client goes bananas because it's terminal 
>>> based.
>>> Fuck escape sequences and stupid retarded Unix.
>>> When do escape sequences actually work as intended? When?
>>>
>>> Anyways suckless.org rocks, and should be implied to hardware.
>>>
>>> Open Source is Insufficient to Solve Trust Problems in Hardware
>>> https://youtube.com/watch?v=Hzb37RyagCQ
>>>
>>> How do you know the hardware in front of you actually conforms to the 
>>> hardware
>>> design you might or might not have?
>>> You can't, it's not like software, at least you can't with existing 
>>> hardware,
>>> watch the video.
>>>
>>> Mud towers build on mud foundation are still mud and will collapse under 
>>> mud.
>>>
>>> This was more-less the important stuff
>>> Fuck I hate re-writing emails fuck me!
>>>
>>
>> Fuck. Ass. Genius. You maybe want to watch the Youtube Channels of Ben
>> Eater [1] or James Sharman [2] for a starting point talking about
>> hardware and how to build a CPU from scratch using bread boards. Fuck.
>> Ass. Genius. Then start reading about what microelectronics is about or
>> even get a degree in microelectronics. Fuck. Ass. Genius.
>>
>> [1] <https://www.youtube.com/@BenEater/playlists>
>> [2] <https://www.youtube.com/@weirdboyjim/playlists>
>>
>> -- 
>> Christian
> 
> Since I am autistic, I have trouble understanding when exactly someone is
> making fun of me or what tone they are using, especially trough text.

According to the messages you sent here this far, I pretty much doubt
you are suffering autism.

> For the sake of friendliness I shall assume that you said all of that in good
> spirit and were not condescending.

Everyone on these lists here will be friendly. You appear really rude. I
just used kind of the same words you are using. If this makes you feel
uncomfortable or even irritates you, what do you think those same words
make others feel about you?

> Fuck stupid retarded Unix.

Very nice posting this here. Thank you.

> Since I extremely-extremely love problem solving - creating and debugging
> hardware ought to be fun!

Then just brew your own circuits in your kitchen with a density of 5nm
and tell us how many silicon atoms you could fit on your designs.

-- 
Christian

Building -current incrementally.

[Christian Schulte]

So I installed 7.5-stable on a VPS and that kernel panicked during installation
several times. I then grabbed a 7.6-current image and installed the system
using that. That kernel does not panic so I am running -current on that system
since then. Checked out /usr/src, build the system from source using
make build. Did cvs -q update -dP -A every now and then and just ran
make followed by doas make install from time to time in the various directories.
Accidentally ran make in /usr/src and got an unexpected Permission denied
error. I would have expected to be able to run make in /usr/src without any
permission issues. Is this a bug in the build system? There must be a way to
build base incrementally without having to rebuild everything.
$cd /usr/src && make currently stops with a Permission denied error I would not
have expected to happen.


build: atu-intersil-int: Permission denied
*** Error 1 in sys/dev/microcode/atmel (Makefile:22 'atu-intersil-int')
*** Error 2 in sys/dev/microcode (:48 'all': @for entry in afb 
atmel bnx cirruslogic fxp kue neomagic myx ral rsu rtwn rum  s...)
*** Error 2 in sys (:48 'all': @for entry in dev/microcode  
arch/alpha arch/amd64 arch/arm64 arch/armv7  arch/hppa arch/i386 ...)
*** Error 2 in /usr/src (:48 'all': @for entry in lib include 
bin libexec sbin usr.bin usr.sbin share games gnu sys; do  set ...)

Re: Building -current incrementally.

[Christian Schulte]

On 9/26/24 07:15, Otto Moerbeek wrote:
> On Thu, Sep 26, 2024 at 06:38:00AM +0200, Christian Schulte wrote:
> 
>> So I installed 7.5-stable on a VPS and that kernel panicked during 
>> installation
>> several times. I then grabbed a 7.6-current image and installed the system
>> using that. That kernel does not panic so I am running -current on that 
>> system
>> since then. Checked out /usr/src, build the system from source using
>> make build. Did cvs -q update -dP -A every now and then and just ran
>> make followed by doas make install from time to time in the various 
>> directories.
>> Accidentally ran make in /usr/src and got an unexpected Permission denied
>> error. I would have expected to be able to run make in /usr/src without any
>> permission issues. Is this a bug in the build system? There must be a way to
>> build base incrementally without having to rebuild everything.
>> $cd /usr/src && make currently stops with a Permission denied error I would 
>> not
>> have expected to happen.
>>
>>
>> build: atu-intersil-int: Permission denied
>> *** Error 1 in sys/dev/microcode/atmel (Makefile:22 'atu-intersil-int')
>> *** Error 2 in sys/dev/microcode (:48 'all': @for entry in 
>> afb atmel bnx cirruslogic fxp kue neomagic myx ral rsu rtwn rum  s...)
>> *** Error 2 in sys (:48 'all': @for entry in dev/microcode  
>> arch/alpha arch/amd64 arch/arm64 arch/armv7  arch/hppa arch/i386 ...)
>> *** Error 2 in /usr/src (:48 'all': @for entry in lib include 
>> bin libexec sbin usr.bin usr.sbin share games gnu sys; do  set ...)
>>
> 
> 1. Note that "make build" builds kernels and install images but does
> not install them.
> 
> 2. Did you follow the steps in the FAQ and release(5):
> https://www.openbsd.org/faq/faq5.html? And no, it is not *always*
> possible to build incrementally. Sometimes a clean build is needed.

Of course I followed those steps. That's what makes me think something
is not working the way it is intended. After installing a recent
snapshot, I checked out /usr/src and did a make build. I thought this
leaves me with a pre built tree I can just build incrementally. Indeed
this is working. So, after a cvs -q update -dP -A I just cd'ed into the
directories containing files with changes and did make followed by doas
make install. Also in /sys/arch/amd64/compile/GENERIC.MP. This has been
working as expected. I then accidentally ran make directly in /usr/src
which is failing due to some Permission denied I do not understand. I
can still run make install in /usr/src without issues.


> 
> 3. You might want to consider not building from source, but install a
> snapshot by running sysupgrade -s.

I am keen on knowing how those snapshots are build. Do they really wipe
out everything and then do a fresh build - lasting nearly 24h here for
me. I doubt it.

> 
>   -Otto

-- 
Christian

Re: Building -current incrementally.

[Christian Schulte]

On 9/26/24 10:47, Philip Guenther wrote:
> On Thu, Sep 26, 2024 at 12:30 AM Christian Schulte  wrote:
>>
>> On 9/26/24 07:15, Otto Moerbeek wrote:
>>> On Thu, Sep 26, 2024 at 06:38:00AM +0200, Christian Schulte wrote:
> ...
>>>> Accidentally ran make in /usr/src and got an unexpected Permission denied
>>>> error. I would have expected to be able to run make in /usr/src without any
>>>> permission issues. Is this a bug in the build system?
> 
> The build system makes some assumptions about permissions and starting
> state.  From what you describe, your setup violated those assumptions
> in some way, probably by having /usr/obj contain files from a previous
> build or release process.
> 
> 
>>>>  There must be a way to build base incrementally without having to rebuild 
>>>> everything.
> 
> Does there exist a documented, maintained, and tested way to "build
> base incrementally"?  No.
> 
> Would it be *possible* for someone(s) to spend the time to develop,
> document, and maintain going forward a way to build base
> incrementally?  It's software, anything is possible with enough
> resources
> 
> Will the project do that or distribute such a thing provided by
> others?  Short answer: no.  Longer answer: doing so would absolutely
> require understanding the project's development process and the
> consequences of that process and how the change would affect the
> incentives.  I think it is *extremely* unlikely to happen, being a
> gigantic effort that would create new, more complex failure modes that
> would be a net negative for the project.  Given all the things to
> spend our time on, it seems like an unwise choice.

For a newcomer to the codebase working on the build system would be an
ideal task to begin with. After that you know the dependencies and such.
Would you say that if someone would take the time to work an that, it
would not do any good because that would need to be maintained over time
and no one would care? Currently I only had to make clean in
sys/dev/microcode and after that running make followed by make install
in /usr/src starts to work. Running make after make install, lots of
things get relinked/rebuild so that you end up in a make && doas make
install endless loop. So that makes me think something gets installed by
make install although it did not change and thus triggers a
relink/rebuild the next make. Looking at that and finding out about all
that when no one is interested in the results clearly would be a useless
effort.

> 
> 
> ...
>> I am keen on knowing how those snapshots are build. Do they really wipe
>> out everything and then do a fresh build - lasting nearly 24h here for
>> me. I doubt it.
> 
> Almost all (99+%) snapshot builds are by the book, starting with
> clearing /usr/obj/ and rebuilding the symlinks.  On modern amd64
> hardware that takes a few hours for base + release; for landisk it
> takes days.
> 
> The exceptions are the weird ABI breaks where parts of the system have
> to be built and installed in a non-standard order to cross over the
> change.  For example, when I drove the 64bit time_t transition I wrote
> up a careful sequence of build steps to have a kernel that supported
> old+new, then get in a new libc and other affected libs, then enough
> utilities that used the new libs so that you could then switch back to
> the normal "make build" sequence and have it complete.  I did those
> and created snaps for everyone to use for some archs and other people
> did them for the remaining archs, but immediately after we all went
> back to "make clean && make build".  Computer time is cheaper than our
> personal time.

I followed that process manually without snapshots back then. Started
using -stable some time after. Not because of base, but because of
packages being broken for a couple of days every now and then in
-current, which is perfectly fine when following -current, but not, if
you need e.g. LibreOffice and cannot wait 3 days until it starts working
again. I really tried setting up a bulk build machine with dpb but that
machine was old and much too slow to provide any advantage.

> 
> tl;dr: no really, snaps are full builds; if you take short cuts and it
> breaks you get to keep both pieces but you'll just get told to follow
> the normal process to get back and not waste people's time.
>

Maybe I just should have used -current snapshots and be done with it.
Don't know.

-- 
Christian

Re: Building -current incrementally.

[Christian Schulte]

On 9/26/24 10:43, Stuart Henderson wrote:
> On 2024-09-26, Christian Schulte  wrote:
>> I am keen on knowing how those snapshots are build. Do they really wipe
>> out everything and then do a fresh build - lasting nearly 24h here for
>> me. I doubt it.
> 
> That's how I do base+x "make build" and "make release" when I do them.
> 
> 24h is a long time. Are you using very slow machines or did you just not
> use -j?
>

time doas make build

chown root:wheel /tmp/_etcdir.ajLXZ6LHx1/var/sysmerge/etc.tgz
chmod 644 /tmp/_etcdir.ajLXZ6LHx1/var/sysmerge/etc.tgz
45014.21 real 32976.78 user 10507.23 sys

Nearly 13h. Seems the host the guest is running on currently has I/O issues. A
linux guest on the same host also currently has issues. Lots of log messages
like these.

kernel:[ 460.053622] watchdog: BUG: soft lockup - CPU#5 stuck for 75s! 
[kworker/5:0:42]

Linux needs a reboot after those messages, OpenBSD just successfully ran a
make build.

make -j does not make a lot of a difference. The disk I/O currently is just
very slow. Has nothing to do with OpenBSD.

OpenBSD 7.6-current (GENERIC.MP) #12: Thu Sep 26 20:25:45 CEST 2024
schu...@0x02.schulte.it:/sys/arch/amd64/compile/GENERIC.MP
real mem = 6425518080 (6127MB)
avail mem = 6207578112 (5920MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5a10 (10 entries)
bios0: vendor SeaBIOS version "rel-1.16.1-0-g3208b098f51a-prebuilt.qemu.org" 
date 04/01/2014
bios0: QEMU Standard PC (i440FX + PIIX, 1996)
acpi0 at bios0: ACPI 1.0
acpi0: sleep states S3 S4 S5
acpi0: tables DSDT FACP APIC SSDT HPET WAET
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD EPYC 9224 24-Core Processor, 2496.57 MHz, 19-11-01
cpu0: cpuid 1 
edx=178bfbff
 
ecx=f7fa3203
cpu0: cpuid 6 eax=4
cpu0: cpuid 7.0 
ebx=f1bf07ab
 ecx=415f4e edx=ac10
cpu0: cpuid d.1 eax=f
cpu0: cpuid 8001 edx=2fd3fbff 
ecx=8003f3
cpu0: cpuid 8008 ebx=300d205
cpu0: 64KB 64b/line 2-way D-cache, 64KB 64b/line 2-way I-cache
cpu0: 512KB 64b/line 16-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 1000MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD EPYC 9224 24-Core Processor, 2496.59 MHz, 19-11-01
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: AMD EPYC 9224 24-Core Processor, 2496.52 MHz, 19-11-01
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: AMD EPYC 9224 24-Core Processor, 2496.63 MHz, 19-11-01
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins
acpihpet0 at acpi0: 1 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
"ACPI0006" at acpi0 not configured
acpipci0 at acpi0 PCI0
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"QEMU0002" at acpi0 not configured
com0 at acpi0 COM1 addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
acpicmos0 at acpi0
"ACPI0010" at acpi0 not configured
"QEMUVGID" at acpi0 not configured
acpicpu0 at acpi0: C1(@1 halt!)
acpicpu1 at acpi0: C1(@1 halt!)
acpicpu2 at acpi0: C1(@1 halt!)
acpicpu3 at acpi0: C1(@1 halt!)
pvbus0 at mainbus0: KVM
pvclock0 at pvbus0
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel 0 
wired to compatibility, channel 1 wired to compatibility
pciide0: channel 0 disabled (no drives)
pciide0: channel 1 disabled (no drives)
uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 0 int 11
piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 0 int 9
iic0 at piixpm0
vga1 at pci0 dev 2 function 0 "Bochs VGA" rev 0x02
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Memory Balloon" rev 0x00
viomb0 at virtio0
virtio0: apic 0 int 11
virtio1 at pci0 dev 5 function 0 "Qumranet Virtio SCSI" rev 0x00
vioscsi0 at virtio1: qsize 256
scsibus1 at vioscsi0: 255 targets
sd0 at scsibus1 targ 0 lun 0: 
sd0: 614400MB, 512 bytes/sector, 1258291200 sectors, thin
virtio1: msix per-VQ
virtio2 at pci0 dev 18 function 0 "Qumranet Virtio Network" rev 0x00
vio0 at virtio2, address 00:50:56:55:0f:b1
virtio2: msix per-VQ
ppb0 at pci0 dev 30 function 0 "Red Hat Qemu PCI-PCI" rev 0x00
pci1 at ppb0 bus 1
ppb1 at pci0 dev 31 function 0 "Red Hat Qemu PCI-PCI" rev 0x00
pci2 at ppb1 bus 

Re: Building -current incrementally.

[Christian Schulte]

On 9/27/24 11:21, Zé Loff wrote:
> 
> On Fri, Sep 27, 2024 at 09:39:07AM +0200, Christian Schulte wrote:
>> On 9/26/24 10:43, Stuart Henderson wrote:
>>> On 2024-09-26, Christian Schulte  wrote:
>>>> I am keen on knowing how those snapshots are build. Do they really wipe
>>>> out everything and then do a fresh build - lasting nearly 24h here for
>>>> me. I doubt it.
>>>
>>> That's how I do base+x "make build" and "make release" when I do them.
>>>
>>> 24h is a long time. Are you using very slow machines or did you just not
>>> use -j?
>>>
>>
>> time doas make build
>>
>> chown root:wheel /tmp/_etcdir.ajLXZ6LHx1/var/sysmerge/etc.tgz
>> chmod 644 /tmp/_etcdir.ajLXZ6LHx1/var/sysmerge/etc.tgz
>> 45014.21 real 32976.78 user 10507.23 sys
>>
>> Nearly 13h. Seems the host the guest is running on currently has I/O issues. 
>> A
>> linux guest on the same host also currently has issues. Lots of log messages
>> like these.
>>
>> kernel:[ 460.053622] watchdog: BUG: soft lockup - CPU#5 stuck for 75s! 
>> [kworker/5:0:42]
>>
>> Linux needs a reboot after those messages, OpenBSD just successfully ran a
>> make build.
>>
>> make -j does not make a lot of a difference. The disk I/O currently is just
>> very slow. Has nothing to do with OpenBSD.
>>
>> OpenBSD 7.6-current (GENERIC.MP) #12: Thu Sep 26 20:25:45 CEST 2024
>> schu...@0x02.schulte.it:/sys/arch/amd64/compile/GENERIC.MP
>> real mem = 6425518080 (6127MB)
>> avail mem = 6207578112 (5920MB)
>> random: good seed from bootblocks
>> mpath0 at root
>> scsibus0 at mpath0: 256 targets
>> mainbus0 at root
>> bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5a10 (10 entries)
>> bios0: vendor SeaBIOS version "rel-1.16.1-0-g3208b098f51a-prebuilt.qemu.org" 
>> date 04/01/2014
>> bios0: QEMU Standard PC (i440FX + PIIX, 1996)
>  
> 
> You can't expect the build time in a QEMU guest to compare with the one
> from a "bare metal" machine.  Of course it its slower.  And I guess the
> I/O performance will be highly dependent on whatever else the QEMU host
> is running, and on how it prioritizes things.
> 

I did not even expect time(1) to show accurate readings on that host
after make build.

-- 
Christian

Re: pf.conf(5): How to implement sendmail's connection/rate control features with pf?

[Christian Schulte]

On 9/25/24 14:31, Peter N. M. Hansteen wrote:
> On Wed, Sep 25, 2024 at 02:26:18PM +0200, Peter N. M. Hansteen wrote:
>> Another related set of examples and explanations can be found in the blog 
>> post
> 
> I sense a complete URL would have been beneficial here, as in 
> 
> https://nxdomain.no/~peter/forcing_the_password_gropers_through_a_smaller_hole.html
> 
>> forcing_the_password_gropers_through_a_smaller_hole.html (or with nicer 
>> formatting
>> in exchange for G's trackers 
>> https://bsdly.blogspot.com/2017/04/forcing-password-gropers-through.html).
> 
> All the best,
> Peter
> 

Thank you so much. That's exactly what I was looking for.

-- 
Christian

pf.conf(5): How to implement sendmail's connection/rate control features with pf?

[Christian Schulte]

Hello @misc,

I am currently searching for a way to implement sendmail's connection control
features using pf. In sendmail I am using:

dnl # Define connection throttling and window length
define(`confCONNECTION_RATE_THROTTLE', `15')dnl
define(`confCONNECTION_RATE_WINDOW_SIZE',`10m')dnl

dnl # Stop connections that overflow our concurrent and time connection rates
FEATURE(`conncontrol', `nodelay', `terminate')dnl
FEATURE(`ratecontrol', `nodelay', `terminate')dnl

How is this - or something similar - done using pf?

I read about max-pkt-rate, set delay, queueing, state modulation but still fail
to get the full picture.

Following is the pf.conf I am currently using I would like to extend to get
those features. Thanks.

#   $OpenBSD: pf.conf,v 1.55 2017/12/03 20:40:04 sthen Exp $
#
# See pf.conf(5) and /etc/examples/pf.conf

set skip on lo

block return# block stateless traffic
pass# establish keep-state

# By default, do not permit remote connections to X11
block return in on ! lo0 proto tcp to port 6000:6010

# Port build user does not need network
block return out log proto {tcp udp} user _pbuild

# rules for spamd(8)
table  persist
#table  persist file "/etc/mail/nospamd"
pass in on egress inet proto tcp from any to any port smtp \
divert-to 127.0.0.1 port spamd
#pass in on egress proto tcp from  to any port smtp
pass in log on egress proto tcp from  to any port smtp
pass out log on egress proto tcp to any port smtp

Re: Memory upgrade

[Christian Schulte]

On 10/17/24 17:03, Stuart Henderson wrote:
> On 2024-10-17, Christian Schulte  wrote:
>> On 10/17/24 09:40, Stuart Henderson wrote:
>>> On 2024-10-16, Christian Schulte  wrote:
>>>>
>>>> No. That's what seems to went wrong when going from i386 to amd64.
>>>> The 3GB hard limit of i386 was in the range of available physical
>>>> memory (4GB) without swap. The 128GB on amd64 do not. Changing this
>>>> to what "memory" reads, makes no difference, though.
>>>
>>> A 32-bit architecture allows for 2^32 bytes, or 4GB, of address space
>>> in a process. Memory addresses are stored in 32-bit variables - a
>>> single process can't use memory beyond this. Even on OS which use PAE
>>> to allow for larger _overall_ memory, and regardless of whether that
>>> memory is RAM or swap space on disk, the maximum that a single 32-bit
>>> process can use is still 4GB.
>>>
>>
>> Could really need some guidance on this. Trying to make the mails as
>> small as possible.
>>
>> We are talking about this commit, right?
>>
>> <https://github.com/openbsd/src/commit/b426ab7bc6c256bfb7af9a9f082a20534b39a28b>
> 
> That's nothing to do with the paragraph of mine quoted above.
> 
>> Reading all of this takes a lot of time. So I "switched" to a brute
>> force approach. Make it panic and drop to the debugger. This diff:
>>
>>
>> Index: sys/arch/amd64/amd64/machdep.c
>> ===
>> RCS file: /cvs/src/sys/arch/amd64/amd64/machdep.c,v
>> retrieving revision 1.297
>> diff -u -p -u -r1.297 machdep.c
>> --- sys/arch/amd64/amd64/machdep.c   21 Sep 2024 19:06:07 -  1.297
>> +++ sys/arch/amd64/amd64/machdep.c   17 Oct 2024 10:55:05 -
>> @@ -214,7 +214,9 @@ struct vm_map *phys_map = NULL;
>>  
>>  /* UVM constraint ranges. */
>>  struct uvm_constraint_range  isa_constraint = { 0x0, 0x00ffUL };
>> -struct uvm_constraint_range  dma_constraint = { 0x0, 0xUL };
>> +// struct uvm_constraint_range  dma_constraint = { 0x0, 0xUL };
>> +struct uvm_constraint_range  dma_constraint = { 0x0, (paddr_t)-1 };
>> +
>>  struct uvm_constraint_range *uvm_md_constraints[] = {
>>  &isa_constraint,
>>  &dma_constraint,
> 
> Here lie dragons. I don't understand this area particularly well but
> think you are definitely in "you get to keep the broken fragments
> of your filesystem if things go badly wrong" territory. I recommend
> backups.
> 

I just don't get the point for trying to support 32bit hardware on a
64bit system. If the hardware does not support 64bit but is limited to
32bit, use i386. If the hardware supports 64bit, just don't limit it to
32bit. Does not make sense to me making a 64bit OS support 32bit
devices. Of course, there is an upgrade path. amd64 just should fail to
boot on any hardware not 100% 64bit capable. For such hardware use i386.
Either the hardware is fully 64bit ready, or it is not. In my opinion
amd64 should really give up on anything 32bit and should really just
refuse to boot non 100% 64bit capable systems at all.

-- 
Christian

Re: Memory upgrade

[Christian Schulte]

On 10/17/24 18:10, Christian Schulte wrote:
> On 10/17/24 17:03, Stuart Henderson wrote:
>> On 2024-10-17, Christian Schulte  wrote:
>>> On 10/17/24 09:40, Stuart Henderson wrote:
>>>> On 2024-10-16, Christian Schulte  wrote:
>>>>>
>>>>> No. That's what seems to went wrong when going from i386 to amd64.
>>>>> The 3GB hard limit of i386 was in the range of available physical
>>>>> memory (4GB) without swap. The 128GB on amd64 do not. Changing this
>>>>> to what "memory" reads, makes no difference, though.
>>>>
>>>> A 32-bit architecture allows for 2^32 bytes, or 4GB, of address space
>>>> in a process. Memory addresses are stored in 32-bit variables - a
>>>> single process can't use memory beyond this. Even on OS which use PAE
>>>> to allow for larger _overall_ memory, and regardless of whether that
>>>> memory is RAM or swap space on disk, the maximum that a single 32-bit
>>>> process can use is still 4GB.
>>>>
>>>
>>> Could really need some guidance on this. Trying to make the mails as
>>> small as possible.
>>>
>>> We are talking about this commit, right?
>>>
>>> <https://github.com/openbsd/src/commit/b426ab7bc6c256bfb7af9a9f082a20534b39a28b>
>>
>> That's nothing to do with the paragraph of mine quoted above.
>>
>>> Reading all of this takes a lot of time. So I "switched" to a brute
>>> force approach. Make it panic and drop to the debugger. This diff:
>>>
>>>
>>> Index: sys/arch/amd64/amd64/machdep.c
>>> ===
>>> RCS file: /cvs/src/sys/arch/amd64/amd64/machdep.c,v
>>> retrieving revision 1.297
>>> diff -u -p -u -r1.297 machdep.c
>>> --- sys/arch/amd64/amd64/machdep.c  21 Sep 2024 19:06:07 -  1.297
>>> +++ sys/arch/amd64/amd64/machdep.c  17 Oct 2024 10:55:05 -
>>> @@ -214,7 +214,9 @@ struct vm_map *phys_map = NULL;
>>>  
>>>  /* UVM constraint ranges. */
>>>  struct uvm_constraint_range  isa_constraint = { 0x0, 0x00ffUL };
>>> -struct uvm_constraint_range  dma_constraint = { 0x0, 0xUL };
>>> +// struct uvm_constraint_range  dma_constraint = { 0x0, 0xUL };
>>> +struct uvm_constraint_range  dma_constraint = { 0x0, (paddr_t)-1 };
>>> +
>>>  struct uvm_constraint_range *uvm_md_constraints[] = {
>>>  &isa_constraint,
>>>  &dma_constraint,
>>
>> Here lie dragons. I don't understand this area particularly well but
>> think you are definitely in "you get to keep the broken fragments
>> of your filesystem if things go badly wrong" territory. I recommend
>> backups.

Ok. Just tell me what I need to do to make a kernel with that diff
applied panic and drop to the debugger. A make build in parallel with
Gimp opened up a ton of images to make the system swap still does not
drop me to the debugger.

-- 
Christian

Re: malloc_trim(3) alternative in OpenBSD?

[Christian Schulte]

On 2/7/25 04:56, Daniel Wilkins wrote:
> On Fri, Feb 07, 2025 at 03:37:31AM +0100, Christian Schulte wrote:
>> Hi @misc,
>>
>> does OpenBSD provide something equivalent to Linux' malloc_trim(3)[1]? I
>> am yet to port an application from Linux to OpenBSD and that application
>> is making use of malloc_trim(3). I removed the calls to malloc_trim(3)
>> --
>> Christian
>>
> Obvious question but: did you try just running it on OpenBSD?

No. Not yet. Will just give it a try. Thanks.

-- 
Christian

libjwt available in ports?

[Christian Schulte]

Hi @misc,

is this library [1] available in ports? I could not find it and the API
is very volatile. It's in current Debian stable but the API there
already is outdated as well. Would it make sense to port it to OpenBSD?
How to deal with developers providing such libraries breaking API/ABI
with every new major release every odd month? The application I am
building on Debian stable against that library would stop compiling the
next dist upgrade. Quite insane for such a trivial task. I can built
that library on on OpenBSD myself, of course.

[1] 

-- 
Christian

malloc_trim(3) alternative in OpenBSD?

[Christian Schulte]

Hi @misc,

does OpenBSD provide something equivalent to Linux' malloc_trim(3)[1]? I
am yet to port an application from Linux to OpenBSD and that application
is making use of malloc_trim(3). I removed the calls to malloc_trim(3)
from that application while still running on Linux and it seems it
really needs to perform those calls there in order to release memory to
the OS after having run for a couple of weeks without restart. That
application is using pthreads(7)[2] a lot but this is not the issue.
Even if it would fork/execve, those processes also would not get
terminated up until the whole application is terminated.

[1] 
[2] 

-- 
Christian

Need some advice on C semantics.

[Christian Schulte]

Hi @misc,

is there something specific for OpenBSD like style(9) but for semantics?
I understand that style(9) is all about syntax. As a long term Java
developer having lost all interest in Java, I am searching for something
like PMD, Checkstyle etc. for C and rules OpenBSD developers tend to
adhere to. Recently read some C comments like

/*
 * This is designed to be small, not fast.
 */

and things like that. In Java, we always had some CI server checking
various design guidelines like

A method should have only one return statement.

and things like this. In C this is very different due to e.g. lack of
exceptions and so. I am still failing to find semantic design guidelines
or best practices documentation for C which I believe OpenBSD developers
adhere to but never documented them somewhere. Could you please point me
to some documentation regarding this? Thank you.

-- 
Christian

Re: Need some advice on C semantics.

[Christian Schulte]

On 12/12/24 12:13, Kirill A. Korinsky wrote:
> On Thu, 12 Dec 2024 11:54:29 +0100,
> Christian Schulte  wrote:
>>
>> is there something specific for OpenBSD like style(9) but for semantics?
>> I understand that style(9) is all about syntax. As a long term Java
>> developer having lost all interest in Java, I am searching for something
>> like PMD, Checkstyle etc. for C and rules OpenBSD developers tend to
>> adhere to. Recently read some C comments like
>>
> 
> Probably you're looking for clang-format which can be installed as part of
> devel/clang-tools-extra.
> 

I am not asking about syntax. I am asking about semantics. Documentation
about best practices used in the OpenBSD source tree I am certain exist
but I am failing to find any documentation about.

-- 
Christian

Re: Need some advice on C semantics.

[Christian Schulte]

On 12/12/24 16:38, Janne Johansson wrote:
>> and things like that. In Java, we always had some CI server checking
>> various design guidelines like
>>
>> A method should have only one return statement.
>>
>> and things like this. In C this is very different due to e.g. lack of
>> exceptions and so. I am still failing to find semantic design guidelines
>> or best practices documentation for C which I believe OpenBSD developers
>> adhere to but never documented them somewhere. Could you please point me
>> to some documentation regarding this? Thank you.
> 
> I think there was some codified "rules" in the old C-lint but as
> compilers grew better and less code (at least the native obsd code at
> that) showed issues that the linters could find, it was eventually
> removed. Stuff like "don't use gets(), sprintf(), strcpy()" is a rule
> that is enforced by the linker complaining as soon as someone uses
> such unsafe interfaces, combined with at least the kernel being built
> with -Wall -Werror to make sure the user is notified as soon as known
> bad constructs are added.
> 
> Other "rules" like "put large local vars before smaller data types in
> functions or structs" is seen if you read a lot of OpenBSD code, but
> is probably more of an optimization to spill less dummy bytes than a
> fixed rule, though for 64 bit arches, not having to step over single
> bytes to access a 64bit entity in a packed struct may or may not be a
> problem for strict arches, but it is so easy to avoid by having larger
> first and smaller later that it is just how it is done.
> 
> I don't think there will be a good complete set of semantic rules,
> somewhat due to lots of code being of different authors and ages, but
> also because if there had to be rules, it would more be "learn the
> environment you code in" than something like "don't use goto". The
> latter is easy to state, and easy to "test for" using scripts and
> programs, but the former has far more value in making your code
> better.
> 

I am quite tired now and will need some sleep. I will try to come up
with an example the next day. One using pointer syntax and a while loop
and one using array syntax and a for loop. The first will make the
compiler produce something similar to what I would have written in
assembly directly. The latter will magically make the compiler start
generating SIMD instructions. I may be very wrong about this and things
just happened out of luck, but I'll give it a try the next day and
report back. Those linker warnings about using questionable functions
are of great help already, but this is not what I am heading after. Lets
see if I can come up with an example tomorrow.

-- 
Christian

Re: Need some advice on C semantics.

[Christian Schulte]

On 12/13/24 12:26, Maxim wrote:
> Christian Schulte, 2024-12-12 11:54 +0100:
>> is there something specific for OpenBSD like style(9) but for semantics?
> 
> I believe such document doesn't exist. As it's been suggested to you,
> reading and learning for the codebase comes closest.
> 
> However, you probably will be interested in general documentation on
> "C semantics". Look out for things like infamous UB (undefined
> behaviour), implicit type and arithmetic conversions (pitfalls of mixing
> signed and unsigned integers is one example), secure coding, etc.
> 
> A few pointers to get you started:
> -   C89/C99/C11/C17/C23 standards
> (copies of final drafts are available online for free)
> -   SEI CERT C Coding Standards
> -   MISRA C
> -   "Expert C programming" by Peter van der Linden
> -   "How to C in 2016" by Matt Stancliff
> 
> Beware some advice and guidelines found in these documents will be very
> opinionated.

Thank you very much. That's indeed what I was looking for. Those
undefined behaviour pitfalls. This just does not exist in the Java
Virtual Machine specification - or - when something was not quite clear,
the next specification version just clarified things and the
specification got updated accordingly. Just like when the meaning of the
volatile keyword got reworked/changed. Stopped following Java evolution
after the acquision by Oracle due to the way fundamental Java principles
got broken. Sad story. 27 years of Java experience to throw away now.
Just glad there is C.

-- 
Christian

Re: Need some advice on C semantics.

[Christian Schulte]

On 12/18/24 06:46, Geoff Steckel wrote:
> On 12/17/24 7:17 PM, Christian Schulte wrote:
>> Thank you very much. That's indeed what I was looking for. Those
>> undefined behaviour pitfalls. This just does not exist in the Java
>> Virtual Machine specification - or - when something was not quite clear,
>> the next specification version just clarified things and the
>> specification got updated accordingly. Just like when the meaning of the
>> volatile keyword got reworked/changed. Stopped following Java evolution
>> after the acquision by Oracle due to the way fundamental Java principles
>> got broken. Sad story. 27 years of Java experience to throw away now.
>> Just glad there is C.
>>
> You've been given very good advice.

Acknowledged.

> C is very old and has changed a lot since it was invented.

It's all still either von Neumann or Harvard. C is just a portable
abstraction to writing machine code directly and it does a great job
about it.

> It still manipulates addressable bits on a particular hardware platform.
> Discipline beyond that is entirely up to you.

This one I cannot comment due to lack of experience with C.

> 
> Suggestions for less stressful C:
> Learn the common idioms which protect against the simple
> fatal errors - overruns, null pointers, off-by etc. immediately.
> Pointer scope control is vital.
> Code sprawl conceals many problems.
> Old paraphrase: "Show me your code and I know nothing
> Show me your data structures and I know exactly what you can do"
> Knuth? Wirth? I forget.

That's exactly what James Gosling et. al. did a very good job about
abstracting all of this by inventing Java (the language and the machine
specification). That's pretty much why a lot of businesses adopt Java to
replace centralized COBOL mainframes in favor of decentralized Java
cluster nodes. Oracle, IBM and such know they will be able to make money
out of it for decades to come, of course. They just shouldn't have
crippled it.

> 
> good luck,
> geoff steckel
> 

May the forth be we with you. Thanks. Nice one.

-- 
Christian

Re: SMTPD start failed - table-proc: unexpected EOF during handshake

[Christian Schulte]

On 11/21/24 11:04, Danny König wrote:
> Hi,
>  
> after upgrading from 7.5 to 7.6 smtpd won't start anymore. Nothing else
> has changed. Packages has been updated (pkg_add -u).
>  
> Any ideas on how to fix this?
> 
> # smtpd -d
> info: OpenSMTPD 7.6.0 starting
> credentials[98842]: warn: table-api: imsg_get: Result too large
> lookup: table-proc: unexpected EOF during handshake
> smtpd: process lka socket closed
>  
> Thanks.
>  
> Regards,
> Danny
>  

This mail has been answered on the @misc OpenSMTPD mailing list.



It may not be available in the archive yet.

Regarding the "How can I fix this asap? This system is in production."

If you would have read the upgrade guide before upgrading a system in
production, you could have saved yourself a lot of time without needing
to post to two different mailing lists.

-- 
Christian

How to parallelization

[Christian Schulte]

Hello everybody,

as a long time (decades) Java developer now converting to bare metal C,
I would like to ask about how to do parallelization properly. The poor
man's approach the Java APIs implement pretty much resemble what every
one else is doing. Take the number of processors available in a system
and size everything to this. This just does not make sense, because
there are a lot of other processes or even kernel threads competing in
parallel. Is there some standard API (e.g. POSIX) allowing an
application to reserve a certain amount of processors to a specific
application? What I mean is - you can run make -j8 on a machine with 16
CPUs. That does not mean that this make really can use those 8 CPUs
exclusively as requested. Same for the Java VM. Starts multiple
background threads. Needs to share all CPUs with the OS and various
other processes. Still just uses the number of CPUs available to size
things. I am searching for some kind of system API allowing an
application to reserve a certain amount of CPUs exclusively - not shared
with any other application - maybe not even the OS. Does this make sense?

-- 
Christian

Re: How to parallelization

[Christian Schulte]

On 12/23/24 17:03, Jan Stary wrote:
> On Dec 23 14:55:30, c...@schulte.it wrote:
>> things. I am searching for some kind of system API allowing an
>> application to reserve a certain amount of CPUs exclusively - not shared
>> with any other application - maybe not even the OS.
> 
> If that existed, every application would start with
> "reserve everything for me".
> 

I doubt it. I am heading for an API an application can use to "tell" its
environment about how much "compute" it needs to perform its tasks
optimally. For example:

make -j8
Error: Failed to provide 8 independent processors.

I am not Nostradamus. Just considering it will not take that long until
there will be some kind of 1024+ CPUs available to the OS to manage
without any single application benefiting from it. Even today
applications are still just badly designed and will slow down themselves
due to "give me more than I will ever need and slow everything down -
even me."

-- 
Christian

Re: How to parallelization

[Christian Schulte]

On 12/23/24 17:37, Geoff Steckel wrote:
> On 12/23/24 11:20 AM, Gábor LENCSE wrote:
>> Under Linux, one can use the isolcpus kernel command line
>> parameter to exclude certain cores from the scheduler.
>> I use the DPDK rte_eal_remote_launch() function to start a thread on
>> an isolated CPU core.
>>
>> Is there anything similar under OpenBSD?
>>
> Is there any reason why multiple processes with shared memory segments
> won't work as well?
> In my experience there's little if any performance difference.
> Unless the application is truly SIMD, the additional security
> and ease of debugging pay off quickly.
> 
> In my experience, OpenBSD does a good job spreading compute
> bound processes over all available CPUs.
> 

Not criticizing OpenBSD in any way. Let me try to explain a common use
case. There is a data source capable of providing X bytes per second at
max. The application needs to be setup in a way it can receive those X
bytes per second without spin locking or waiting for data. If it would
be "polling" too fast, it would slow down the whole system waiting for
data. If it would be "polling" too slow, it would not be able to process
those bytes fast enough. Those bytes need to be processed. So there is a
receiving process which needs to be able to consume exactly those X
bytes per second. That consumer also needs to be defined in a way it can
process those bytes in parallel as fast as possible. Sizing the consumer
too small, the producer will start spin locking or such and cannot keep
up with the data rate it needs to process, because the consumer does not
process the data fast enough. Sizing the consumer too big, the consumer
will start spin locking or such waiting for the producer to provide more
data. I am searching for an API to make the application adhere to those
situations automatically. Data rate on the receiving part decreases,
consumer part does not need to use Y processes in parallel all spin
locking waiting for more data. Data rate on the receiving part
increases, consumer needs to increase compute to not slow down the
receiver. Does this make things more clear?

-- 
Christian


-- 
Christian

Re: How to parallelization

[Christian Schulte]

On 12/23/24 19:43, Christian Schulte wrote:
> On 12/23/24 17:37, Geoff Steckel wrote:
>> On 12/23/24 11:20 AM, Gábor LENCSE wrote:
>>> Under Linux, one can use the isolcpus kernel command line
>>> parameter to exclude certain cores from the scheduler.
>>> I use the DPDK rte_eal_remote_launch() function to start a thread on
>>> an isolated CPU core.
>>>
>>> Is there anything similar under OpenBSD?
>>>
>> Is there any reason why multiple processes with shared memory segments
>> won't work as well?
>> In my experience there's little if any performance difference.
>> Unless the application is truly SIMD, the additional security
>> and ease of debugging pay off quickly.
>>
>> In my experience, OpenBSD does a good job spreading compute
>> bound processes over all available CPUs.
>>
> 
> Not criticizing OpenBSD in any way. Let me try to explain a common use
> case. There is a data source capable of providing X bytes per second at
> max. The application needs to be setup in a way it can receive those X
> bytes per second without spin locking or waiting for data. If it would
> be "polling" too fast, it would slow down the whole system waiting for
> data. If it would be "polling" too slow, it would not be able to process
> those bytes fast enough. Those bytes need to be processed. So there is a
> receiving process which needs to be able to consume exactly those X
> bytes per second. That consumer also needs to be defined in a way it can
> process those bytes in parallel as fast as possible. Sizing the consumer
> too small, the producer will start spin locking or such and cannot keep
> up with the data rate it needs to process, because the consumer does not
> process the data fast enough. Sizing the consumer too big, the consumer
> will start spin locking or such waiting for the producer to provide more
> data. I am searching for an API to make the application adhere to those
> situations automatically. Data rate on the receiving part decreases,
> consumer part does not need to use Y processes in parallel all spin
> locking waiting for more data. Data rate on the receiving part
> increases, consumer needs to increase compute to not slow down the
> receiver. Does this make things more clear?
> 

Hmm. I am using this:

<https://mongoose.ws/>

to handle a websocket. Every websocket message received needs to be
processed in parallel. So there is a queue the receiving thread is
enqueuing messages to and there are multiple threads dequeuing messages
from that queue. I'd like to use some kind of API to size the capacity
of the queue and the number of threads dequeuing automatically based on
the data rate of the websocket. Very few bytes per second, small queue
capacity and a single thread processing the websocket data will do
without any locking. Lots of bytes per second - burst - enlarge queue
capacity and start enough threads to dequeue things in time. I am very
sure I am failing to describe the issue I am having. Just make
everything "big enough" to be able to handle those bursts slows things
down in every other situation. Maybe the APIs I am using just do not
provide the features I am heading for.

-- 
Christian

Re: How to parallelization

[Christian Schulte]

On 12/23/24 22:23, Geoff Steckel wrote:
> On 12/23/24 1:43 PM, Christian Schulte wrote:
>> Not criticizing OpenBSD in any way. Let me try to explain a common use
>> case. There is a data source capable of providing X bytes per second at
>> max. The application needs to be setup in a way it can receive those X
>> bytes per second without spin locking or waiting for data. If it would
>> be "polling" too fast, it would slow down the whole system waiting for
>> data. If it would be "polling" too slow, it would not be able to process
>> those bytes fast enough. Those bytes need to be processed. So there is a
>> receiving process which needs to be able to consume exactly those X
>> bytes per second. That consumer also needs to be defined in a way it can
>> process those bytes in parallel as fast as possible. Sizing the consumer
>> too small, the producer will start spin locking or such and cannot keep
>> up with the data rate it needs to process, because the consumer does not
>> process the data fast enough. Sizing the consumer too big, the consumer
>> will start spin locking or such waiting for the producer to provide more
>> data. I am searching for an API to make the application adhere to those
>> situations automatically. Data rate on the receiving part decreases,
>> consumer part does not need to use Y processes in parallel all spin
>> locking waiting for more data. Data rate on the receiving part
>> increases, consumer needs to increase compute to not slow down the
>> receiver. Does this make things more clear?
>>
> Thank you for your explanation.
> 
> I'm assuming that there multiple logical streams which are
> extracted from the incoming packet streams and distributed
> to multiple consumers.

Exactly. Read samples from a stream. Group them in a way they can be
processed in parallel and that number is very dynamic. For example: Get
1000 samples and all of them can be processed in parallel vs. get 1000
samples and only 10 of them can be processed in parallel. Manage the
number of processors dynamically.

> Is it true that you are attempting to perfectly assign and utilize
> all CPUs for packet & application processing?
> If packet ordering is to be preserved it's not clear that
> perfect allocation is possible.
> 
>>>> I am searching for an API to make the application adhere to those
>>>> situations automatically
> 
> The only way I can see to achieve 100% perfect usage this is:
>   buffer the incoming packet stream deeply
>   inspect each packet to measure application resources needed
>   summarize those results over some time period

6 years manually calibrating parameters and the next catastrophe will
prove you wrong. Either someone needs to monitor the system 24/7 and be
able to tweak parameters immediately, or the application must do this
automatically and be done with it.

>   systemwide, measure resource capability and current utilization
>   determine systemwide resource allocation using some algorithm
>   adjust systemwide application resources
>   forward packets to each application

The only API I found is clock_getttime(CLOCK_MONOTONIC) and doing the
rest manually myself. I am not sure if I'd better be off using
CLOCK_THREAD_CPUTIME_ID because I am not sure if pthreads mutex
lock/unlock and condition wait/broadcast will be accounted for
correctly. Control flow becomes something like this:

t0 = start time
do the work - no syscalls here just number crunching on memory
t1 = end time

So add two syscalls for getting the time values into branches not doing
any syscalls intentionally. Calculate corresponding rate values
manually. Manage the number of worker threads, queue size, network
buffer size and what not based on this. Works for me on OpenBSD. Does
not work for me on Linux, because there it is not easy to keep things in
well known bounds. Seems there are no well known bounds. Make that the
sole application on a dedicated machine where nothing can get in between.

> 
> I worked on a network appliance which did complex resource
> allocation while forwarding packets. It wasn't simple.

Not simple indeed. Thank's for your thoughts on this.

-- 
Christian