Re: ntpd segfaults on start
On Sat, Sep 07, 2019 at 12:53:19AM -0700, Harlan Stenn wrote: > Cy, > > On 9/6/2019 4:56 PM, Cy Schubert wrote: > > ... > > > > For those who enable ASLR, a better workaround is, to add this to your > > ntp.conf: > > > > rlimit memlock 64 > > > > Until a more precise default is determined. > > Should I change the default value for FreeBSD-12 to be 64 for now? > > I can get this change in place for the upcoming ntp-4.2.8p14 release, > and we can change it later if needed. ASLR is disabled by default, so if anybody tweak a system config, she should know better to tweak ntpd as well. I am fine with changing the defaults for ntpd, but I think that more useful would be to update the documentation (but where to put it ?). ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ntpd segfaults on start
In message <20190907075619.gg2...@kib.kiev.ua>, Konstantin Belousov writes: > On Sat, Sep 07, 2019 at 12:53:19AM -0700, Harlan Stenn wrote: > > Cy, > > > > On 9/6/2019 4:56 PM, Cy Schubert wrote: > > > ... > > > > > > For those who enable ASLR, a better workaround is, to add this to your > > > ntp.conf: > > > > > > rlimit memlock 64 > > > > > > Until a more precise default is determined. > > > > Should I change the default value for FreeBSD-12 to be 64 for now? > > > > I can get this change in place for the upcoming ntp-4.2.8p14 release, > > and we can change it later if needed. > > ASLR is disabled by default, so if anybody tweak a system config, she > should know better to tweak ntpd as well. I am fine with changing the > defaults for ntpd, but I think that more useful would be to update > the documentation (but where to put it ?). I agree. We should update the documentation for now. 64 MB was my first successful test but I suspect we can get it lower, like 47 MB. For now we can update the documentation to say that if a person enables ASLR they must add this to ntp.conf. I'll find the best number instead of the current sledgehammer. Where to put it? I've added it to the ASLR wiki (https://wiki.freebsd.org/AS LR) for now. An ASLR page should go into the handbook documenting how to use up ASLR and gotchas like this and mitigations. 64 MB is safe for now. I will do more testing. I think it can go below 47. My sandbox has been running ntpd all night with 47 so far. I will try lower. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
panic: VOP_UNSET_TEXT returned 22: on r351627
I got the following panic this AM during a poudriere run.
r351627 is the revision I'm at.
Core *IS* available.
Ideas?
Unread portion of the kernel message buffer:
VNASSERT failed
0xf809e6335960: tag tmpfs, type VREG
usecount 1, writecount 0, refcount 2
flags (VI_ACTIVE)
v_object 0xf81f37227000 ref 2 pages 1063 cleanbuf 0 dirtybuf 0
lock type tmpfs: SHARED (count 1)
tag VT_TMPFS, tmpfs_node 0xf803214f83a0, flags 0x0, links 1
mode 0755, owner 65534, group 0, size 4352808, status 0x0
panic: VOP_UNSET_TEXT returned 22
cpuid = 22
time = 1567862254
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
0xfe01bfd618b0
vpanic() at vpanic+0x19d/frame 0xfe01bfd61900
panic() at panic+0x43/frame 0xfe01bfd61960
vm_map_entry_set_vnode_text() at vm_map_entry_set_vnode_text+0x275/frame
0xfe01bfd619b0
vm_map_process_deferred() at vm_map_process_deferred+0x70/frame
0xfe01bfd619d0
vm_map_remove() at vm_map_remove+0xc6/frame 0xfe01bfd61a00
vmspace_exit() at vmspace_exit+0xd8/frame 0xfe01bfd61a40
exit1() at exit1+0x57d/frame 0xfe01bfd61ab0
sys_sys_exit() at sys_sys_exit+0xd/frame 0xfe01bfd61ac0
amd64_syscall() at amd64_syscall+0x29f/frame 0xfe01bfd61bf0
fast_syscall_common() at fast_syscall_common+0x101/frame
0xfe01bfd61bf0
--- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8008326aa, rsp =
0x7fffe1b8, rbp = 0x7fffe1d0 ---
Uptime: 7d15h33m31s
Dumping 23246 out of 131027
MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
__curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
55 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct
pcpu,
(kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
#1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:392
#2 0x804bcf60 in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:479
#3 0x804bd3d9 in vpanic (fmt=, ap=out>)
at /usr/src/sys/kern/kern_shutdown.c:905
#4 0x804bd113 in panic (fmt=)
at /usr/src/sys/kern/kern_shutdown.c:832
#5 0x807644e5 in vm_map_entry_set_vnode_text (entry=out>,
add=) at /usr/src/sys/vm/vm_map.c:557
#6 0x807645a0 in vm_map_process_deferred ()
at /usr/src/sys/vm/vm_map.c:593
#7 0x8076a1b6 in _vm_map_unlock (map=,
file=, line=3653) at /usr/src/sys/vm/vm_map.c:607
#8 vm_map_remove (map=, start=4096, end=140737488355328)
at /usr/src/sys/vm/vm_map.c:3653
#9 0x80764118 in vmspace_dofree (vm=)
at /usr/src/sys/vm/vm_map.c:335
#10 vmspace_exit (td=0xf8016632c000) at /usr/src/sys/vm/vm_map.c:416
#11 0x8047d27d in exit1 (td=0xf8016632c000, rval=out>,
signo=0) at /usr/src/sys/kern/kern_exit.c:416
#12 0x8047ccfd in sys_sys_exit (td=, uap=out>)
at /usr/src/sys/kern/kern_exit.c:195
#13 0x807f13df in syscallenter (td=0xf8016632c000)
at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:144
#14 amd64_syscall (td=0xf8016632c000, traced=0)
at /usr/src/sys/amd64/amd64/trap.c:1180
#15
#16 0x0008008326aa in ?? ()
Backtrace stopped: Cannot access memory at address 0x7fffe1b8
(kgdb)
--
Larry Rosenman http://people.freebsd.org/~ler
Phone: +1 214-642-9640 E-Mail: l...@freebsd.org
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
signature.asc
Description: OpenPGP digital signature
Re: panic: VOP_UNSET_TEXT returned 22: on r351627
On Sat, Sep 07, 2019 at 08:49:10AM -0500, Larry Rosenman wrote:
> I got the following panic this AM during a poudriere run.
>
> r351627 is the revision I'm at.
>
> Core *IS* available.
>
> Ideas?
It highly suspect that is should be fixed by
https://reviews.freebsd.org/D21560.
Slightly amused that your report comes a day after the Andrew' one.
>
>
>
> Unread portion of the kernel message buffer:
> VNASSERT failed
> 0xf809e6335960: tag tmpfs, type VREG
> usecount 1, writecount 0, refcount 2
> flags (VI_ACTIVE)
> v_object 0xf81f37227000 ref 2 pages 1063 cleanbuf 0 dirtybuf 0
> lock type tmpfs: SHARED (count 1)
> tag VT_TMPFS, tmpfs_node 0xf803214f83a0, flags 0x0, links 1
> mode 0755, owner 65534, group 0, size 4352808, status 0x0
>
> panic: VOP_UNSET_TEXT returned 22
> cpuid = 22
> time = 1567862254
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> 0xfe01bfd618b0
> vpanic() at vpanic+0x19d/frame 0xfe01bfd61900
> panic() at panic+0x43/frame 0xfe01bfd61960
> vm_map_entry_set_vnode_text() at vm_map_entry_set_vnode_text+0x275/frame
> 0xfe01bfd619b0
> vm_map_process_deferred() at vm_map_process_deferred+0x70/frame
> 0xfe01bfd619d0
> vm_map_remove() at vm_map_remove+0xc6/frame 0xfe01bfd61a00
> vmspace_exit() at vmspace_exit+0xd8/frame 0xfe01bfd61a40
> exit1() at exit1+0x57d/frame 0xfe01bfd61ab0
> sys_sys_exit() at sys_sys_exit+0xd/frame 0xfe01bfd61ac0
> amd64_syscall() at amd64_syscall+0x29f/frame 0xfe01bfd61bf0
> fast_syscall_common() at fast_syscall_common+0x101/frame
> 0xfe01bfd61bf0
> --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x8008326aa, rsp =
> 0x7fffe1b8, rbp = 0x7fffe1d0 ---
> Uptime: 7d15h33m31s
> Dumping 23246 out of 131027
> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
>
> __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
> 55__asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct
> pcpu,
> (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
> #1 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:392
> #2 0x804bcf60 in kern_reboot (howto=260)
> at /usr/src/sys/kern/kern_shutdown.c:479
> #3 0x804bd3d9 in vpanic (fmt=, ap= out>)
> at /usr/src/sys/kern/kern_shutdown.c:905
> #4 0x804bd113 in panic (fmt=)
> at /usr/src/sys/kern/kern_shutdown.c:832
> #5 0x807644e5 in vm_map_entry_set_vnode_text (entry= out>,
> add=) at /usr/src/sys/vm/vm_map.c:557
> #6 0x807645a0 in vm_map_process_deferred ()
> at /usr/src/sys/vm/vm_map.c:593
> #7 0x8076a1b6 in _vm_map_unlock (map=,
> file=, line=3653) at /usr/src/sys/vm/vm_map.c:607
> #8 vm_map_remove (map=, start=4096, end=140737488355328)
> at /usr/src/sys/vm/vm_map.c:3653
> #9 0x80764118 in vmspace_dofree (vm=)
> at /usr/src/sys/vm/vm_map.c:335
> #10 vmspace_exit (td=0xf8016632c000) at /usr/src/sys/vm/vm_map.c:416
> #11 0x8047d27d in exit1 (td=0xf8016632c000, rval= out>,
> signo=0) at /usr/src/sys/kern/kern_exit.c:416
> #12 0x8047ccfd in sys_sys_exit (td=, uap= out>)
> at /usr/src/sys/kern/kern_exit.c:195
> #13 0x807f13df in syscallenter (td=0xf8016632c000)
> at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:144
> #14 amd64_syscall (td=0xf8016632c000, traced=0)
> at /usr/src/sys/amd64/amd64/trap.c:1180
> #15
> #16 0x0008008326aa in ?? ()
> Backtrace stopped: Cannot access memory at address 0x7fffe1b8
> (kgdb)
>
> --
> Larry Rosenman http://people.freebsd.org/~ler
> Phone: +1 214-642-9640 E-Mail: l...@freebsd.org
> US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ntpd segfaults on start
On Sat, Sep 07, 2019 at 06:09:16AM -0700, Cy Schubert wrote: > In message <20190907075619.gg2...@kib.kiev.ua>, Konstantin Belousov writes: > > On Sat, Sep 07, 2019 at 12:53:19AM -0700, Harlan Stenn wrote: > > > Cy, > > > > > > On 9/6/2019 4:56 PM, Cy Schubert wrote: > > > > ... > > > > > > > > For those who enable ASLR, a better workaround is, to add this to your > > > > ntp.conf: > > > > > > > > rlimit memlock 64 > > > > > > > > Until a more precise default is determined. > > > > > > Should I change the default value for FreeBSD-12 to be 64 for now? > > > > > > I can get this change in place for the upcoming ntp-4.2.8p14 release, > > > and we can change it later if needed. > > > > ASLR is disabled by default, so if anybody tweak a system config, she > > should know better to tweak ntpd as well. I am fine with changing the > > defaults for ntpd, but I think that more useful would be to update > > the documentation (but where to put it ?). > > I agree. We should update the documentation for now. 64 MB was my first > successful test but I suspect we can get it lower, like 47 MB. For now we > can update the documentation to say that if a person enables ASLR they must > add this to ntp.conf. I'll find the best number instead of the current > sledgehammer. > > Where to put it? I've added it to the ASLR wiki (https://wiki.freebsd.org/AS > LR) for now. An ASLR page should go into the handbook documenting how to > use up ASLR and gotchas like this and mitigations. May be in security(7). There are actually two workarounds, with enabled ASLR. One is the rlimit, another one is to disable stack base randomization by gap. > > 64 MB is safe for now. I will do more testing. I think it can go below 47. > My sandbox has been running ntpd all night with 47 so far. I will try lower. > > > -- > Cheers, > Cy Schubert > FreeBSD UNIX: Web: http://www.FreeBSD.org > > The need of the many outweighs the greed of the few. > ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ntpd segfaults on start
In message <20190907153226.gi2...@kib.kiev.ua>, Konstantin Belousov writes: > On Sat, Sep 07, 2019 at 06:09:16AM -0700, Cy Schubert wrote: > > In message <20190907075619.gg2...@kib.kiev.ua>, Konstantin Belousov writes: > > > On Sat, Sep 07, 2019 at 12:53:19AM -0700, Harlan Stenn wrote: > > > > Cy, > > > > > > > > On 9/6/2019 4:56 PM, Cy Schubert wrote: > > > > > ... > > > > > > > > > > For those who enable ASLR, a better workaround is, to add this to you > r > > > > > ntp.conf: > > > > > > > > > > rlimit memlock 64 > > > > > > > > > > Until a more precise default is determined. > > > > > > > > Should I change the default value for FreeBSD-12 to be 64 for now? > > > > > > > > I can get this change in place for the upcoming ntp-4.2.8p14 release, > > > > and we can change it later if needed. > > > > > > ASLR is disabled by default, so if anybody tweak a system config, she > > > should know better to tweak ntpd as well. I am fine with changing the > > > defaults for ntpd, but I think that more useful would be to update > > > the documentation (but where to put it ?). > > > > I agree. We should update the documentation for now. 64 MB was my first > > successful test but I suspect we can get it lower, like 47 MB. For now we > > can update the documentation to say that if a person enables ASLR they must > > > add this to ntp.conf. I'll find the best number instead of the current > > sledgehammer. > > > > Where to put it? I've added it to the ASLR wiki (https://wiki.freebsd.org/A > S > > LR) for now. An ASLR page should go into the handbook documenting how to > > use up ASLR and gotchas like this and mitigations. > May be in security(7). Maybe. > > There are actually two workarounds, with enabled ASLR. One is the rlimit, > another one is to disable stack base randomization by gap. The latter works but I'm not enamoured with it. I suppose we can list the workarounds and let the user pick the one they want to use. I've been able to set the memlock rlimit as low as 20 MB. The issue is letting it default to 0 which allows ntp to mlockall() anything it wants. ntpd on my sandbox is currently using 18 MB. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ntpd segfaults on start
On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote: > I've been able to set the memlock rlimit as low as 20 MB. The issue is > letting it default to 0 which allows ntp to mlockall() anything it wants. > ntpd on my sandbox is currently using 18 MB. Default stack size on amd64 is 512M, and default stack gap percentage is 3%. This means that the gap can be as large as ~17MB. If 3MB is enough for the stack of the main thread of ntpd, then fine. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ntpd segfaults on start
In message <20190907161749.gj2...@kib.kiev.ua>, Konstantin Belousov writes: > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote: > > I've been able to set the memlock rlimit as low as 20 MB. The issue is > > letting it default to 0 which allows ntp to mlockall() anything it wants. > > ntpd on my sandbox is currently using 18 MB. > > Default stack size on amd64 is 512M, and default stack gap percentage is > 3%. This means that the gap can be as large as ~17MB. If 3MB is enough > for the stack of the main thread of ntpd, then fine. The default stack is 200K, which is also tuneable in ntp.conf. rlimit [memlock Nmegabytes | stacksize N4kPages filenum Nfiledescriptors] memlock Nmegabytes Specify the number of megabytes of memory that should be allocated and locked. Probably only available under Linux, this option may be useful when dropping root (the -i option). The default is 32 megabytes on non-Linux machines, and -1 under Linux. -1 means "do not lock the process into memory". 0 means "lock whatever memory the process wants into memory". stacksize N4kPages Specifies the maximum size of the process stack on systems with the mlockall() function. Defaults to 50 4k pages (200 4k pages in OpenBSD). filenum Nfiledescriptors Specifies the maximum number of file descriptors ntpd may have open at once. Defaults to the system default. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
