Re: HEADS UP: caution required with updates using custom kernels
On Fri, Jun 24, 2016 at 10:50:34PM +, Brooks Davis wrote: > pipe(2) had an unnecessarily odd calling convention (ignoring the > argument the user thought they were passing and returning the two file > descriptors via the two return registers). This required machine > dependent assembly for every target and special handling in tracing > tools (ktrace, dtrace, etc). On 64-bit platforms, pipe(2)'s > implementation is the only reason the two-register return model needs to > exist at all (on 32-bit platforms it allows off_t to be returned from > lseek). getpid() is another instance. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: 11.0 -r301815 "kyua test -k /usr/tests/Kyuafile" on rpi2 [armv7-a/cortex-a7]: broken (24) and failing (59) lists
On Fri, Jun 24, 2016 at 9:50 PM, Mark Millard wrote: > On 2016-Jun-24, at 2:50 PM, Alan Somers wrote: > >> As of r302180., the usr.sbin/rpcbind, sys/acl, and sys/sys/bitstring >> tests should all be fixed. I opened PR 210329 for the >> usr.bin/lastcomm test. I haven't investigated the others. ... > This time the totals were 15 broken (down from 24) and 41 failed (down > from 59). > > My results this time were. . . Hi Mark, Please file bugs for all of the individual component failures, e.g. lib/msun, and CC me on the bugs. Thanks, -Ngie ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Samba 4.3 and 4.4 crashes on FreeBSD 11-ALPHA4
On Sat, Jun 25, 2016 at 01:03:13AM +0300, Guy Yur wrote:
> libtdb.so.1`tdb_runtime_check_for_robust_mutexes + 1475 at mutex.c:957
> ...
The pointer to tdb_runtime_check_for_robust_mutexes() appeared to be
most useful, thanks.
The two patches below should fix samba use of robustness. First,
kernel erronously reset robust lists locations on fork. Second, the
pthread_mutex_trylock() for owned errorcheck mutex must return EDEADLK
and not EBUSY. Try that.
diff --git a/lib/libthr/thread/thr_mutex.c b/lib/libthr/thread/thr_mutex.c
index 5a99605..da71c70 100644
--- a/lib/libthr/thread/thr_mutex.c
+++ b/lib/libthr/thread/thr_mutex.c
@@ -850,9 +871,12 @@ mutex_self_trylock(struct pthread_mutex *m)
switch (PMUTEX_TYPE(m->m_flags)) {
case PTHREAD_MUTEX_ERRORCHECK:
- case PTHREAD_MUTEX_NORMAL:
case PTHREAD_MUTEX_ADAPTIVE_NP:
- ret = EBUSY;
+ ret = EDEADLK;
+ break;
+
+ case PTHREAD_MUTEX_NORMAL:
+ ret = EBUSY;
break;
case PTHREAD_MUTEX_RECURSIVE:
diff --git a/sys/sys/proc.h b/sys/sys/proc.h
index 6d03062..6162a16 100644
--- a/sys/sys/proc.h
+++ b/sys/sys/proc.h
@@ -282,9 +282,6 @@ struct thread {
int td_no_sleeping; /* (k) Sleeping disabled count. */
int td_dom_rr_idx; /* (k) RR Numa domain selection. */
void*td_su; /* (k) FFS SU private */
- uintptr_t td_rb_list; /* (k) Robust list head. */
- uintptr_t td_rbp_list;/* (k) Robust priv list head. */
- uintptr_t td_rb_inact;/* (k) Current in-action mutex loc. */
#definetd_endzero td_sigmask
/* Copied during fork1() or create_thread(). */
@@ -298,6 +295,9 @@ struct thread {
u_char td_base_user_pri; /* (t) Base user pri */
u_int td_dbg_sc_code; /* (c) Syscall code to debugger. */
u_int td_dbg_sc_narg; /* (c) Syscall arg count to debugger.*/
+ uintptr_t td_rb_list; /* (k) Robust list head. */
+ uintptr_t td_rbp_list;/* (k) Robust priv list head. */
+ uintptr_t td_rb_inact;/* (k) Current in-action mutex loc. */
#definetd_endcopy td_pcb
/*
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Samba 4.3 and 4.4 crashes on FreeBSD 11-ALPHA4
On Sat, Jun 25, 2016 at 11:49:39AM +0300, Konstantin Belousov wrote:
> On Sat, Jun 25, 2016 at 01:03:13AM +0300, Guy Yur wrote:
> > libtdb.so.1`tdb_runtime_check_for_robust_mutexes + 1475 at mutex.c:957
> > ...
>
> The pointer to tdb_runtime_check_for_robust_mutexes() appeared to be
> most useful, thanks.
>
> The two patches below should fix samba use of robustness. First,
> kernel erronously reset robust lists locations on fork. Second, the
> pthread_mutex_trylock() for owned errorcheck mutex must return EDEADLK
> and not EBUSY. Try that.
Correction, there was a reason why I initially put the rb list pointers
into zeroed region. It still needs to be zeroed on new thread creation.
Updated patch.
diff --git a/lib/libthr/thread/thr_mutex.c b/lib/libthr/thread/thr_mutex.c
index 5a99605..da71c70 100644
--- a/lib/libthr/thread/thr_mutex.c
+++ b/lib/libthr/thread/thr_mutex.c
@@ -850,9 +871,12 @@ mutex_self_trylock(struct pthread_mutex *m)
switch (PMUTEX_TYPE(m->m_flags)) {
case PTHREAD_MUTEX_ERRORCHECK:
- case PTHREAD_MUTEX_NORMAL:
case PTHREAD_MUTEX_ADAPTIVE_NP:
- ret = EBUSY;
+ ret = EDEADLK;
+ break;
+
+ case PTHREAD_MUTEX_NORMAL:
+ ret = EBUSY;
break;
case PTHREAD_MUTEX_RECURSIVE:
diff --git a/sys/kern/kern_thr.c b/sys/kern/kern_thr.c
index 10ccdab..293574c 100644
--- a/sys/kern/kern_thr.c
+++ b/sys/kern/kern_thr.c
@@ -234,6 +234,7 @@ thread_create(struct thread *td, struct rtprio *rtp,
bcopy(&td->td_startcopy, &newtd->td_startcopy,
__rangeof(struct thread, td_startcopy, td_endcopy));
newtd->td_proc = td->td_proc;
+ newtd->td_rb_list = newtd->td_rbp_list = newtd->td_rb_inact = 0;
thread_cow_get(newtd, td);
error = initialize_thread(newtd, thunk);
diff --git a/sys/sys/proc.h b/sys/sys/proc.h
index 6d03062..6162a16 100644
--- a/sys/sys/proc.h
+++ b/sys/sys/proc.h
@@ -282,9 +282,6 @@ struct thread {
int td_no_sleeping; /* (k) Sleeping disabled count. */
int td_dom_rr_idx; /* (k) RR Numa domain selection. */
void*td_su; /* (k) FFS SU private */
- uintptr_t td_rb_list; /* (k) Robust list head. */
- uintptr_t td_rbp_list;/* (k) Robust priv list head. */
- uintptr_t td_rb_inact;/* (k) Current in-action mutex loc. */
#definetd_endzero td_sigmask
/* Copied during fork1() or create_thread(). */
@@ -298,6 +295,9 @@ struct thread {
u_char td_base_user_pri; /* (t) Base user pri */
u_int td_dbg_sc_code; /* (c) Syscall code to debugger. */
u_int td_dbg_sc_narg; /* (c) Syscall arg count to debugger.*/
+ uintptr_t td_rb_list; /* (k) Robust list head. */
+ uintptr_t td_rbp_list;/* (k) Robust priv list head. */
+ uintptr_t td_rb_inact;/* (k) Current in-action mutex loc. */
#definetd_endcopy td_pcb
/*
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Samba 4.3 and 4.4 crashes on FreeBSD 11-ALPHA4
Hi, On Sat, Jun 25, 2016 at 12:20 PM, Konstantin Belousov wrote: > On Sat, Jun 25, 2016 at 11:49:39AM +0300, Konstantin Belousov wrote: >> On Sat, Jun 25, 2016 at 01:03:13AM +0300, Guy Yur wrote: >> > libtdb.so.1`tdb_runtime_check_for_robust_mutexes + 1475 at mutex.c:957 >> > ... >> >> The pointer to tdb_runtime_check_for_robust_mutexes() appeared to be >> most useful, thanks. >> >> The two patches below should fix samba use of robustness. First, >> kernel erronously reset robust lists locations on fork. Second, the >> pthread_mutex_trylock() for owned errorcheck mutex must return EDEADLK >> and not EBUSY. Try that. > > Correction, there was a reason why I initially put the rb list pointers > into zeroed region. It still needs to be zeroed on new thread creation. > Updated patch. > With the patch to the 3 files applied, smbd, nmbd and smbclient no longer crash on my machine. Tested with r302191+patch. Thanks, Guy ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: 11.0 -r301815 "kyua test -k /usr/tests/Kyuafile" on rpi2 [armv7-a/cortex-a7]: broken (24) and failing (59) lists
On 2016-Jun-25, at 12:44 AM, Ngie Cooper wrote: > On Fri, Jun 24, 2016 at 9:50 PM, Mark Millard wrote: >> On 2016-Jun-24, at 2:50 PM, Alan Somers wrote: >> >>> As of r302180., the usr.sbin/rpcbind, sys/acl, and sys/sys/bitstring >>> tests should all be fixed. I opened PR 210329 for the >>> usr.bin/lastcomm test. I haven't investigated the others. > > ... > >> This time the totals were 15 broken (down from 24) and 41 failed (down >> from 59). >> >> My results this time were. . . > > Hi Mark, >Please file bugs for all of the individual component failures, > e.g. lib/msun, and CC me on the bugs. > Thanks, > -Ngie Done. I generally omitted the --verbose Metadata output. For a few there was a large block of Standard output or Standard error text that I omitted. I was not sure of the intent but I put all the lib/msun "broken" items in one submittal, for example. Similarly for each of the other "broken" components. Similarly for lib/msun "failed" items (a separate submittal). Similarly for each of the other "failed" components. I noted explicitly in each submittal that I'd used -mcpu=cortex-a7 in my builds. But in more detail: A) buildworld and buildkernel had -march=armv7-a -mcpu=cortex-a7 both listed B) ports builds (such as kyua itself) had -mcpu=cortex-a7 (but not -march=armv7-a) This is because for ports I use options that do not complain for either clang 3.8.0 or for fairly modern gcc and gcc does complain about specifying both -mcpu=armv7-a and -mcpu=cortex-a7. Even the "armv7-a" notation is from gcc rejecting armv7a but both clang and gcc accepting armv7-a notation. (As I remember gcc uses -march=armv7-a where it conflicts with -mcpu=cortex-a7 when both are listed but gcc does warn about conflict despite having a resolution rule.) Other than the 11.0 -r302180 being more recent the "context details" in https://lists.freebsd.org/pipermail/freebsd-current/2016-June/061831.html still apply. I did not repeat that information in the submittals but at least the src.conf and the like are available for reference. === Mark Millard markmi at dsl-only.net ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: HEADS UP: caution required with updates using custom kernels
Am Sat, 25 Jun 2016 10:02:38 +0300 Konstantin Belousov schrieb: > On Fri, Jun 24, 2016 at 10:50:34PM +, Brooks Davis wrote: > > pipe(2) had an unnecessarily odd calling convention (ignoring the > > argument the user thought they were passing and returning the two file > > descriptors via the two return registers). This required machine > > dependent assembly for every target and special handling in tracing > > tools (ktrace, dtrace, etc). On 64-bit platforms, pipe(2)'s > > implementation is the only reason the two-register return model needs to > > exist at all (on 32-bit platforms it allows off_t to be returned from > > lseek). > getpid() is another instance. > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" That all is a nice explanation, but how to recover from a broken system, on which the order of installation wasn't performed the right way? pgpX7oS6migju.pgp Description: OpenPGP digital signature
Re: panic with tcp timers
So
All of our timers in TCP do something like
-
INFO-LOCK
INP_WLOCK
if (inp needs to be dropped) {
drop-it
}
do other work
UNLOCK-INP
UNLOCK-INFO
--
And generally the path “inp needs to be dropped” is rarely taken.
So why don’t we change the procedure to something like:
INP_WLOCK
if (inp needs to be dropped) {
inp_ref()
INP_WUNLOCK()
INFO_LOCK()
INP_WLOCK()
if (inp_release_ref) {
/* victory its dead */
INFO_UNLOCK
return
}
do-the-drop
}
This way we would only go grab the INFO lock in those rarer cases
when we *do* actually want to kill the tcb and at the same time
it would make the current callout system work correctly.. which as
many have pointed out would be much better if we could just let the
lock be gotten by the callout system. Hmm maybe with this scheme we
could even do that...
R
> On Jun 20, 2016, at 1:45 PM, Julien Charbon wrote:
>
>
> Hi,
>
> On 6/20/16 11:58 AM, Gleb Smirnoff wrote:
>> On Mon, Jun 20, 2016 at 11:55:55AM +0200, Julien Charbon wrote:
>> J> > On Fri, Jun 17, 2016 at 11:27:39AM +0200, Julien Charbon wrote:
>> J> > J> > Comparing stable/10 and head, I see two changes that could
>> J> > J> > affect that:
>> J> > J> >
>> J> > J> > - callout_async_drain
>> J> > J> > - switch to READ lock for inp info in tcp timers
>> J> > J> >
>> J> > J> > That's why you are in To, Julien and Hans :)
>> J> > J> >
>> J> > J> > We continue investigating, and I will keep you updated.
>> J> > J> > However, any help is welcome. I can share cores.
>> J> >
>> J> > Now, spending some time with cores and adding a bunch of
>> J> > extra CTRs, I have a sequence of events that lead to the
>> J> > panic. In short, the bug is in the callout system. It seems
>> J> > to be not relevant to the callout_async_drain, at least for
>> J> > now. The transition to READ lock unmasked the problem, that's
>> J> > why NetflixBSD 10 doesn't panic.
>> J> >
>> J> > The panic requires heavy contention on the TCP info lock.
>> J> >
>> J> > [CPU 1] the callout fires, tcp_timer_keep entered
>> J> > [CPU 1] blocks on INP_INFO_RLOCK(&V_tcbinfo);
>> J> > [CPU 2] schedules the callout
>> J> > [CPU 2] tcp_discardcb called
>> J> > [CPU 2] callout successfully canceled
>> J> > [CPU 2] tcpcb freed
>> J> > [CPU 1] unblocks... panic
>> J> >
>> J> > When the lock was WLOCK, all contenders were resumed in a
>> J> > sequence they came to the lock. Now, that they are readers,
>> J> > once the lock is released, readers are resumed in a "random"
>> J> > order, and this allows tcp_discardcb to go before the old
>> J> > running callout, and this unmasks the panic.
>> J>
>> J> Highly interesting. I should be able to reproduce that (will be useful
>> J> for testing the corresponding fix).
>> J>
>> J> Fix proposal: If callout_async_drain() returns 0 (fail) (instead of 1
>> J> (success) here) when the callout cancellation is a success _but_ the
>> J> callout is current running, that should fix it.
>> J>
>> J> For the history: It comes back to my old callout question:
>> J>
>> J> Does _callout_stop_safe() is allowed to return 1 (success) even if the
>> J> callout is still currently running; a.k.a. it is not because you
>> J> successfully cancelled a callout that the callout is not currently
>> running.
>> J>
>> J> We did propose a patch to make _callout_stop_safe() returns 0 (fail)
>> J> when the callout is currently running:
>> J>
>> J> callout_stop() should return 0 when the callout is currently being
>> J> serviced and indeed unstoppable
>> J>
>> https://reviews.freebsd.org/differential/changeset/?ref=62513&whitespace=ignore-most
>> J>
>> J> But this change impacted too many old code paths and was interesting
>> J> only for TCP timers and thus was abandoned.
>>
>> The fix I am working on now is doing exactly that. callout_reset must
>> return 0 if the callout is currently running.
>>
>> What are the old paths impacted?
>
> Actually all the paths that check the callout_stop() return value AND
> call both callout_reset() and callout_stop() AND use mpsafe callout().
> And for each path, we would have to check our patch was ok (or not).
>
> Thus, if you only do the change in callout_async_drain() context, you
> don't impact the "old" callout_stop() behavior and get the desired
> behavior for the TCP timers. Might be a good trade-off...
>
> My 2 cents.
>
> --
> Julien
Randall Stewart
r...@netflix.com
803-317-4952
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: HEADS UP: caution required with updates using custom kernels
On Sat, Jun 25, 2016 at 01:18:06PM +0200, O. Hartmann wrote: > Am Sat, 25 Jun 2016 10:02:38 +0300 > Konstantin Belousov schrieb: > > > On Fri, Jun 24, 2016 at 10:50:34PM +, Brooks Davis wrote: > > > pipe(2) had an unnecessarily odd calling convention (ignoring the > > > argument the user thought they were passing and returning the two file > > > descriptors via the two return registers). This required machine > > > dependent assembly for every target and special handling in tracing > > > tools (ktrace, dtrace, etc). On 64-bit platforms, pipe(2)'s > > > implementation is the only reason the two-register return model needs to > > > exist at all (on 32-bit platforms it allows off_t to be returned from > > > lseek). > > getpid() is another instance. > > ___ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > > That all is a nice explanation, but how to recover from a broken system, on > which the > order of installation wasn't performed the right way? Copy the libc.so.7 binary from the build area to /lib manually, e.g. using rescue shell. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: svn commit: r302185 - head/release/doc/en_US.ISO8859-1/relnotes
|Author: gjb |Date: Fri Jun 24 23:42:33 2016 Sigh. |Log: | Update the release notes following r302182. | A selection of system daemons, including: | fingerd, | ftpd, |- rlogind, |- rshd, and |- sshd have been modified to support |+ rlogind, and |+ rshd have been modified to support | sending notifications to the blacklistd | daemon. Allow me to continue hoping nonetheless. In this great future, you can't forget your past. --steffen ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: svn commit: r302185 - head/release/doc/en_US.ISO8859-1/relnotes
On Sat, Jun 25, 2016 at 03:02:11PM +0200, Steffen Nurpmeso wrote: > |Author: gjb > |Date: Fri Jun 24 23:42:33 2016 > > Sigh. > > |Log: > | Update the release notes following r302182. > > | A selection of system daemons, including: > | fingerd, > | ftpd, > |- rlogind, > |- rshd, and > |- sshd have been modified to support > |+ rlogind, and > |+ rshd have been modified to support > | sending notifications to the blacklistd > | daemon. > > Allow me to continue hoping nonetheless. > In this great future, you can't forget your past. > I hope the issues can be resolved before 11.0-RELEASE. I personally look forward to this change, but the revert was necessary. Glen signature.asc Description: PGP signature
Re: Samba 4.3 and 4.4 crashes on FreeBSD 11-ALPHA4
On 2016-06-25 12:16, Guy Yur wrote: > Hi, > > On Sat, Jun 25, 2016 at 12:20 PM, Konstantin Belousov > wrote: >> On Sat, Jun 25, 2016 at 11:49:39AM +0300, Konstantin Belousov wrote: >>> On Sat, Jun 25, 2016 at 01:03:13AM +0300, Guy Yur wrote: libtdb.so.1`tdb_runtime_check_for_robust_mutexes + 1475 at mutex.c:957 ... >>> >>> The pointer to tdb_runtime_check_for_robust_mutexes() appeared to be >>> most useful, thanks. >>> >>> The two patches below should fix samba use of robustness. First, >>> kernel erronously reset robust lists locations on fork. Second, the >>> pthread_mutex_trylock() for owned errorcheck mutex must return EDEADLK >>> and not EBUSY. Try that. >> >> Correction, there was a reason why I initially put the rb list pointers >> into zeroed region. It still needs to be zeroed on new thread creation. >> Updated patch. >> > > With the patch to the 3 files applied, smbd, nmbd and smbclient > no longer crash on my machine. > > Tested with r302191+patch. > > Thanks, > Guy > Hi, Sorry for being late to the party :/ I've tested your patch (the latest one) and it also seems to do the trick. Thanks! FreeBSD 11.0-ALPHA5 (AMD64) - Samba 4.4 Best regards, Daniel ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Cross-build Brokenness
It appears that the i386 cross-build is broken as follows. Builds on native
amd64 are OK.
--- main.o ---
cc -target i386-unknown-freebsd11.0 --sysroot=/usr/obj/i386.i386/opt/src/svn
-current/tmp -B/usr/obj/i386.i386/opt/src/svn-current/tmp/usr/bin -O2 -pipe
-pipe -DRDUMP -g -MD -MF.depend.main.o -MTmain.o -std=gnu99
-fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k
-Wno-uninitialized -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int
-Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value
-Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion
-Wno-unused-local-typedef -Wno-switch -Wno-switch-enum
-Wno-knr-promoted-parameter -Qunused-arguments -c
/opt/src/svn-current/sbin/dump/main.c -o main.o
--- all_subdir_lib ---
/usr/obj/i386.i386/opt/src/svn-current/tmp/usr/bin/ld:
/usr/bin/../lib/clang/3.8.0/lib/freebsd/libclang_rt.ubsan_standalone-i386.a:
No such file: No such file or directory
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** [h_raw.full] Error code 1
make[7]: stopped in /opt/src/svn-current/lib/libc/tests/ssp
1 error
make[7]: stopped in /opt/src/svn-current/lib/libc/tests/ssp
*** [h_raw] Error code 2
make[6]: stopped in /opt/src/svn-current/lib/libc/tests/ssp
1 error
make[6]: stopped in /opt/src/svn-current/lib/libc/tests/ssp
--- all_subdir_kerberos5 ---
A failure has been detected in another branch of the parallel make
make[5]: stopped in /opt/src/svn-current/kerberos5/lib/libheimbase
--- all_subdir_lib ---
*** [all_subdir_lib/libc/tests/ssp] Error code 2
make[5]: stopped in /opt/src/svn-current/lib/libc/tests
1 error
make[5]: stopped in /opt/src/svn-current/lib/libc/tests
--- all_subdir_kerberos5 ---
*** [all] Error code 2
make[4]: stopped in /opt/src/svn-current/kerberos5/lib
1 error
make[4]: stopped in /opt/src/svn-current/kerberos5/lib
--- all_subdir_lib ---
*** [all] Error code 2
make[4]: stopped in /opt/src/svn-current/lib/libc
1 error
make[4]: stopped in /opt/src/svn-current/lib/libc
--- all_subdir_kerberos5 ---
*** [all_subdir_kerberos5/lib] Error code 2
make[3]: stopped in /opt/src/svn-current/kerberos5
1 error
make[3]: stopped in /opt/src/svn-current/kerberos5
*** [all_subdir_kerberos5] Error code 2
make[2]: stopped in /opt/src/svn-current
--- all_subdir_lib ---
*** [all_subdir_lib/libc] Error code 2
make[3]: stopped in /opt/src/svn-current/lib
1 error
make[3]: stopped in /opt/src/svn-current/lib
*** [all_subdir_lib] Error code 2
make[2]: stopped in /opt/src/svn-current
--- all_subdir_rescue ---
A failure has been detected in another branch of the parallel make
make[6]: stopped in /opt/src/svn-current/bin/csh
*** [csh_make] Error code 2
make[5]: stopped in /export/obj/i386.i386/opt/src/svn-current/rescue/rescue
1 error
make[5]: stopped in /export/obj/i386.i386/opt/src/svn-current/rescue/rescue
*** [objs] Error code 2
make[4]: stopped in /opt/src/svn-current/rescue/rescue
1 error
make[4]: stopped in /opt/src/svn-current/rescue/rescue
*** [all] Error code 2
make[3]: stopped in /opt/src/svn-current/rescue
1 error
make[3]: stopped in /opt/src/svn-current/rescue
*** [all_subdir_rescue] Error code 2
make[2]: stopped in /opt/src/svn-current
--- all_subdir_sbin ---
A failure has been detected in another branch of the parallel make
make[4]: stopped in /opt/src/svn-current/sbin/dump
*** [all_subdir_sbin/dump] Error code 2
make[3]: stopped in /opt/src/svn-current/sbin
1 error
make[3]: stopped in /opt/src/svn-current/sbin
*** [all_subdir_sbin] Error code 2
make[2]: stopped in /opt/src/svn-current
4 errors
make[2]: stopped in /opt/src/svn-current
*** [everything] Error code 2
make[1]: stopped in /opt/src/svn-current
1 error
make[1]: stopped in /opt/src/svn-current
*** [buildworld] Error code 2
make: stopped in /opt/src/svn-current
1 error
make: stopped in /opt/src/svn-current
slippy#
The following patch fixes my i386 cross-builds on amd64 native architecture.
Index: lib/libc/tests/ssp/Makefile
===
--- lib/libc/tests/ssp/Makefile (revision 302191)
+++ lib/libc/tests/ssp/Makefile (working copy)
@@ -1,5 +1,6 @@
# $FreeBSD$
+.include
.include
NO_WERROR=
@@ -34,10 +35,12 @@
.if ${MACHINE_CPUARCH} == "i386" || ${MACHINE_CPUARCH} == "amd64"
.if ${COMPILER_TYPE} == "clang" && ${MK_TOOLCHAIN} == "yes"
.if ${COMPILER_VERSION} < 30500 || 30700 <= ${COMPILER_VERSION}
+.if ${MACHINE_CPUARCH} == ${_HOST_ARCH}
PROGS+=h_raw
.endif
.endif
.endif
+.endif
PROGS+=h_read
PROGS+=h_readlink
PROGS+=h_snprintf
--
Cheers,
Cy Schubert
FreeBSD UNIX: Web: http://www.FreeBSD.org
The need of the many outweighs the greed of the few.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-un
Re: svn commit: r302185 - head/release/doc/en_US.ISO8859-1/relnotes
Glen Barber wrote: |On Sat, Jun 25, 2016 at 03:02:11PM +0200, Steffen Nurpmeso wrote: |>| A selection of system daemons, including: |>| fingerd, |>| ftpd, |>|- rlogind, |>|- rshd, and |>|- sshd have been modified to support |>|+ rlogind, and |>|+ rshd have been modified to support |>| sending notifications to the blacklistd |>| daemon. |> |> Allow me to continue hoping nonetheless. |> In this great future, you can't forget your past. | |I hope the issues can be resolved before 11.0-RELEASE. I personally |look forward to this change, but the revert was necessary. It is very likely that you and D.E. Smørgrav are right, and then 11.0 is to be expected for September. In fact i was only looking at this from a very narrow user perspective and, in addition, never liked that log files have to be parsed to recollect states that were known by the generating daemon. It can only be that commercial software does this better, more integrated, but i don't know. So once the blacklistd idea came up, which was, if i recall correctly, shortly after DragonFly BSD introduced their own logfile analyzer, didn't they?, i was kind of thrilled, because isn't that the first time that the right thing is done to face that problem? In my opinion it would be great if all servers that listen to the outside world would gain the necessary hooks for "bad command", "bad login", "good login", possibly more. This would create an integrated, synchronous mesh of firewall and servers, so talking about clowds.., i am looking forward to this. If rules would become more sophisticated, e.g., "user IP tried to post messages with more than X KB the Y time", and that could be taken into account. And then it also requires less CPU time and thus energy, then having a logfile analyzer running in addition. Thank you. Have a nice weekend. --steffen ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: HEADS UP: caution required with updates using custom kernels
Am Fri, 24 Jun 2016 15:51:11 + Brooks Davis schrieb: > On Fri, Jun 24, 2016 at 06:00:19AM +0200, O. Hartmann wrote: > > Am Thu, 23 Jun 2016 21:07:51 + > > Brooks Davis schrieb: > > > > > Kernel config minimalists and those running aarch64 and riscv systems will > > > want to head this UPDATING message. > > > > > > In practice, if you're fairly up to date, doing installworld before > > > installkernel will also work (I've tested that case from ALPHA4), but is > > > always somewhat risky. > > > > > > -- Brooks > > > > > > - Forwarded message from Brooks Davis - > > > > > > Date: Thu, 23 Jun 2016 21:02:05 + (UTC) > > > From: Brooks Davis > > > To: src-committ...@freebsd.org, svn-src-...@freebsd.org, > > > svn-src-h...@freebsd.org > > > Subject: svn commit: r302152 - head > > > > > > Author: brooks > > > Date: Thu Jun 23 21:02:05 2016 > > > New Revision: 302152 > > > URL: https://svnweb.freebsd.org/changeset/base/302152 > > > > > > Log: > > > Add an UPDATING entry for the pipe() -> pipe2() transition. > > > > > > Approved by:re (gjb) > > > Sponsored by: DARPA, AFRL > > > > > > Modified: > > > head/UPDATING > > > > > > Modified: head/UPDATING > > > == > > > --- head/UPDATING Thu Jun 23 20:59:13 2016(r302151) > > > +++ head/UPDATING Thu Jun 23 21:02:05 2016(r302152) > > > @@ -31,6 +31,14 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 11 > > > disable the most expensive debugging functionality run > > > "ln -s 'abort:false,junk:false' /etc/malloc.conf".) > > > > > > +20160622: > > > + The the libc stub for the pipe(2) system call has been replaced with > > > + a wrapper which calls the pipe2(2) system call and the pipe(2) is now > > > + only implemented by the kernels which include "options > > > + FREEBSD10_COMPAT" in their config file (this is the default). > > > + Users should ensure that this option is enabled in their kernel > > > + or upgrade userspace to r302092 before upgrading their kernel. > > > + > > > 20160527: > > > CAM will now strip leading spaces from SCSI disks' serial numbers. > > > This will effect users who create UFS filesystems on SCSI disks using > > > > > > > > > - End forwarded message - > > > > Is this showing up, when one doesn't have the expected COMPAT_FREEBSD10 in > > kernel and > > updated kernel __before___ world?: > > You must include COMPAT_FREEBSD10 or have a new userspace. Otherwise > things like your shell are unlikely to work. How to fix this probblem? I have no other machine available to build a generic system/kernel. How to fix this mess with a running system, but with messages like below showing up when try building a new world? The warning came, as usual, too late! The source and changes were out. It seems that I'm not the only one with problems like that, so please provide some instructions to salvage the situation. Thanks in advance, oh > > -- Brooks > > > > > most recent CURRENT (FreeBSD 11.0-ALPHA4 #41 r302149: Thu Jun 23 21:58:25 > > CEST 2016 > > amd64, custom kernel) dies when trying to > > > > make buildworld > > > > or > > > > make buildkernel/kernel > > > > with the message shown below: > > > > root@localhost: [src] make buildkernel > > *** Signal 12 > > > > Stop. > > make: stopped in /usr/src > > .ERROR_TARGET='buildkernel' > > .ERROR_META_FILE='' > > .MAKE.LEVEL='0' > > MAKEFILE='' > > .MAKE.MODE='normal' > > .CURDIR='/usr/src' > > .MAKE='make' > > .OBJDIR='/usr/obj/usr/src' > > .TARGETS='buildkernel' > > DESTDIR='' > > LD_LIBRARY_PATH='' > > MACHINE='amd64' > > MACHINE_ARCH='amd64' > > MAKEOBJDIRPREFIX='/usr/obj' > > MAKESYSPATH='/usr/src/share/mk' > > MAKE_VERSION='20160606' > > PATH='/sbin:/bin:/usr/sbin:/usr/bin' > > SRCTOP='/usr/src' > > OBJTOP='/usr/obj/usr/src' > > > > Regards, > > > > oh > > pgpASHlJmVSyd.pgp Description: OpenPGP digital signature
Re: HEADS UP: caution required with updates using custom kernels
Am Sat, 25 Jun 2016 14:35:44 +0300 Konstantin Belousov schrieb: > On Sat, Jun 25, 2016 at 01:18:06PM +0200, O. Hartmann wrote: > > Am Sat, 25 Jun 2016 10:02:38 +0300 > > Konstantin Belousov schrieb: > > > > > On Fri, Jun 24, 2016 at 10:50:34PM +, Brooks Davis wrote: > > > > pipe(2) had an unnecessarily odd calling convention (ignoring the > > > > argument the user thought they were passing and returning the two file > > > > descriptors via the two return registers). This required machine > > > > dependent assembly for every target and special handling in tracing > > > > tools (ktrace, dtrace, etc). On 64-bit platforms, pipe(2)'s > > > > implementation is the only reason the two-register return model needs to > > > > exist at all (on 32-bit platforms it allows off_t to be returned from > > > > lseek). > > > getpid() is another instance. > > > ___ > > > freebsd-current@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > > To unsubscribe, send any mail to > > > "freebsd-current-unsubscr...@freebsd.org" > > > > That all is a nice explanation, but how to recover from a broken system, on > > which the > > order of installation wasn't performed the right way? > > Copy the libc.so.7 binary from the build area to /lib manually, e.g. using > rescue shell. > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" I did so, but there is no effect. Whenever I try to build a kernel/world, I receive this from the shell: root@localhost: [src] make update *** Signal 12 Stop. make: stopped in /usr/src .ERROR_TARGET='update' .ERROR_META_FILE='' .MAKE.LEVEL='0' MAKEFILE='' .MAKE.MODE='normal' .CURDIR='/usr/src' .MAKE='make' .OBJDIR='/usr/obj/usr/src' .TARGETS='update' DESTDIR='' LD_LIBRARY_PATH='' MACHINE='amd64' MACHINE_ARCH='amd64' MAKEOBJDIRPREFIX='/usr/obj' MAKESYSPATH='/usr/src/share/mk' MAKE_VERSION='20160606' PATH='/sbin:/bin:/usr/sbin:/usr/bin' SRCTOP='/usr/src' OBJTOP='/usr/obj/usr/src' These two machines in question are victims "of the early adopter" - the warning came way too late! At work, I did the same update - but I did an installworld prior to the usual installkernel - and everything seems so far to work, even without COMAPT_FREEBSD10 in the kernel. Is there a way to salvage the situation without relying on "customized" third party kernels? I usually use /bin/csh - so this might be of use. Thank you in advance for help, kind regards, Oliver pgp9B6NQYaEMA.pgp Description: OpenPGP digital signature
Re: HEADS UP: caution required with updates using custom kernels
On Sat, Jun 25, 2016 at 05:17:14PM +0200, O. Hartmann wrote: > Am Sat, 25 Jun 2016 14:35:44 +0300 > Konstantin Belousov schrieb: > > > On Sat, Jun 25, 2016 at 01:18:06PM +0200, O. Hartmann wrote: > > > Am Sat, 25 Jun 2016 10:02:38 +0300 > > > Konstantin Belousov schrieb: > > > > > > > On Fri, Jun 24, 2016 at 10:50:34PM +, Brooks Davis wrote: > > > > > pipe(2) had an unnecessarily odd calling convention (ignoring the > > > > > argument the user thought they were passing and returning the two file > > > > > descriptors via the two return registers). This required machine > > > > > dependent assembly for every target and special handling in tracing > > > > > tools (ktrace, dtrace, etc). On 64-bit platforms, pipe(2)'s > > > > > implementation is the only reason the two-register return model needs > > > > > to > > > > > exist at all (on 32-bit platforms it allows off_t to be returned from > > > > > lseek). > > > > getpid() is another instance. > > > > ___ > > > > freebsd-current@freebsd.org mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > > > To unsubscribe, send any mail to > > > > "freebsd-current-unsubscr...@freebsd.org" > > > > > > That all is a nice explanation, but how to recover from a broken system, > > > on which the > > > order of installation wasn't performed the right way? > > > > Copy the libc.so.7 binary from the build area to /lib manually, e.g. using > > rescue shell. > > ___ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > > I did so, but there is no effect. > > Whenever I try to build a kernel/world, I receive this from the shell: > > root@localhost: [src] make update > *** Signal 12 > > Stop. > make: stopped in /usr/src > .ERROR_TARGET='update' > .ERROR_META_FILE='' > .MAKE.LEVEL='0' > MAKEFILE='' > .MAKE.MODE='normal' > .CURDIR='/usr/src' > .MAKE='make' > .OBJDIR='/usr/obj/usr/src' > .TARGETS='update' > DESTDIR='' > LD_LIBRARY_PATH='' > MACHINE='amd64' > MACHINE_ARCH='amd64' > MAKEOBJDIRPREFIX='/usr/obj' > MAKESYSPATH='/usr/src/share/mk' > MAKE_VERSION='20160606' > PATH='/sbin:/bin:/usr/sbin:/usr/bin' > SRCTOP='/usr/src' > OBJTOP='/usr/obj/usr/src' > > These two machines in question are victims "of the early adopter" - the > warning came way > too late! At work, I did the same update - but I did an installworld prior to > the usual > installkernel - and everything seems so far to work, even without > COMAPT_FREEBSD10 in the > kernel. > > Is there a way to salvage the situation without relying on "customized" third > party > kernels? > > I usually use /bin/csh - so this might be of use. You need either kernel with COMPAT_FREEBSD10, or simpler, newer libc. You could take that libc anywhere, e.g. from the ALPHA5 binaries recently uploaded. There is no way to fix build with the combination of bits you have at present self-hosting. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: HEADS UP: caution required with updates using custom kernels
On Sat, 25 Jun 2016 18:26:45 +0300 Konstantin Belousov wrote: > On Sat, Jun 25, 2016 at 05:17:14PM +0200, O. Hartmann wrote: >> Is there a way to salvage the situation without relying on "customized" >> third party kernels? >> >> I usually use /bin/csh - so this might be of use. > > You need either kernel with COMPAT_FREEBSD10, or simpler, newer libc. > You could take that libc anywhere, e.g. from the ALPHA5 binaries recently > uploaded. There is no way to fix build with the combination of bits you > have at present self-hosting. Can't he boot kernel.old? ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: HEADS UP: caution required with updates using custom kernels
Am Sat, 25 Jun 2016 17:35:31 +0200 Tijl Coosemans schrieb: > On Sat, 25 Jun 2016 18:26:45 +0300 Konstantin Belousov > wrote: > > On Sat, Jun 25, 2016 at 05:17:14PM +0200, O. Hartmann wrote: > >> Is there a way to salvage the situation without relying on "customized" > >> third party kernels? > >> > >> I usually use /bin/csh - so this might be of use. > > > > You need either kernel with COMPAT_FREEBSD10, or simpler, newer libc. > > You could take that libc anywhere, e.g. from the ALPHA5 binaries recently > > uploaded. There is no way to fix build with the combination of bits you > > have at present self-hosting. > > Can't he boot kernel.old? ... he can, but it is the very same as with the recent kernel. Besides - I thought "I would have the most recent libc" in this scenario and the kernel is out of sync? pgpU6zNtTmYGa.pgp Description: OpenPGP digital signature
Re: panic with tcp timers
Ok
Lets try this again with my source changed to my @freebsd.net :-)
Now I am also attaching a patch for you Gleb, this will take some poking to
get in to your NF-head since it incorporates some changes we made earlier.
I think this will fix the problem.. i.e. dealing with two locks in the callout
system (which it was
never meant to have done)..
Note we probably can move the code to use the callout lock init now.. but lets
see if this works
on your setup on c096 and if so we can think about doing that.
R
for_gleb
Description: Binary data
> On Jun 25, 2016, at 4:48 AM, Randall Stewart wrote:
>
> So
>
> All of our timers in TCP do something like
> -
> INFO-LOCK
> INP_WLOCK
>
> if (inp needs to be dropped) {
>drop-it
> }
> do other work
>
> UNLOCK-INP
> UNLOCK-INFO
> --
>
> And generally the path “inp needs to be dropped” is rarely taken.
>
> So why don’t we change the procedure to something like:
>
> INP_WLOCK
> if (inp needs to be dropped) {
> inp_ref()
> INP_WUNLOCK()
> INFO_LOCK()
> INP_WLOCK()
> if (inp_release_ref) {
> /* victory its dead */
> INFO_UNLOCK
> return
> }
> do-the-drop
> }
>
> This way we would only go grab the INFO lock in those rarer cases
> when we *do* actually want to kill the tcb and at the same time
> it would make the current callout system work correctly.. which as
> many have pointed out would be much better if we could just let the
> lock be gotten by the callout system. Hmm maybe with this scheme we
> could even do that...
>
> R
>
>
>> On Jun 20, 2016, at 1:45 PM, Julien Charbon wrote:
>>
>>
>> Hi,
>>
>> On 6/20/16 11:58 AM, Gleb Smirnoff wrote:
>>> On Mon, Jun 20, 2016 at 11:55:55AM +0200, Julien Charbon wrote:
>>> J> > On Fri, Jun 17, 2016 at 11:27:39AM +0200, Julien Charbon wrote:
>>> J> > J> > Comparing stable/10 and head, I see two changes that could
>>> J> > J> > affect that:
>>> J> > J> >
>>> J> > J> > - callout_async_drain
>>> J> > J> > - switch to READ lock for inp info in tcp timers
>>> J> > J> >
>>> J> > J> > That's why you are in To, Julien and Hans :)
>>> J> > J> >
>>> J> > J> > We continue investigating, and I will keep you updated.
>>> J> > J> > However, any help is welcome. I can share cores.
>>> J> >
>>> J> > Now, spending some time with cores and adding a bunch of
>>> J> > extra CTRs, I have a sequence of events that lead to the
>>> J> > panic. In short, the bug is in the callout system. It seems
>>> J> > to be not relevant to the callout_async_drain, at least for
>>> J> > now. The transition to READ lock unmasked the problem, that's
>>> J> > why NetflixBSD 10 doesn't panic.
>>> J> >
>>> J> > The panic requires heavy contention on the TCP info lock.
>>> J> >
>>> J> > [CPU 1] the callout fires, tcp_timer_keep entered
>>> J> > [CPU 1] blocks on INP_INFO_RLOCK(&V_tcbinfo);
>>> J> > [CPU 2] schedules the callout
>>> J> > [CPU 2] tcp_discardcb called
>>> J> > [CPU 2] callout successfully canceled
>>> J> > [CPU 2] tcpcb freed
>>> J> > [CPU 1] unblocks... panic
>>> J> >
>>> J> > When the lock was WLOCK, all contenders were resumed in a
>>> J> > sequence they came to the lock. Now, that they are readers,
>>> J> > once the lock is released, readers are resumed in a "random"
>>> J> > order, and this allows tcp_discardcb to go before the old
>>> J> > running callout, and this unmasks the panic.
>>> J>
>>> J> Highly interesting. I should be able to reproduce that (will be useful
>>> J> for testing the corresponding fix).
>>> J>
>>> J> Fix proposal: If callout_async_drain() returns 0 (fail) (instead of 1
>>> J> (success) here) when the callout cancellation is a success _but_ the
>>> J> callout is current running, that should fix it.
>>> J>
>>> J> For the history: It comes back to my old callout question:
>>> J>
>>> J> Does _callout_stop_safe() is allowed to return 1 (success) even if the
>>> J> callout is still currently running; a.k.a. it is not because you
>>> J> successfully cancelled a callout that the callout is not currently
>>> running.
>>> J>
>>> J> We did propose a patch to make _callout_stop_safe() returns 0 (fail)
>>> J> when the callout is currently running:
>>> J>
>>> J> callout_stop() should return 0 when the callout is currently being
>>> J> serviced and indeed unstoppable
>>> J>
>>> https://reviews.freebsd.org/differential/changeset/?ref=62513&whitespace=ignore-most
>>> J>
>>> J> But this change impacted too many old code paths and was interesting
>>> J> only for TCP timers and thus was abandoned.
>>>
>>> The fix I am working on now is doing exactly that. callout_reset must
>>> return 0 if the callout is currently running.
>>>
>>> What are the old paths impacted?
>>
>> Actually all the paths that check the callout_stop() return value AND
>> call both callout_reset() and callout_stop() AND use mpsafe callout().
>> And for each path, we would have to check our patch was ok (or not).
>>
>> Thus, if you only
Re: HEADS UP: caution required with updates using custom kernels
On Sat, Jun 25, 2016 at 05:35:31PM +0200, Tijl Coosemans wrote:
> On Sat, 25 Jun 2016 18:26:45 +0300 Konstantin Belousov
> wrote:
> > On Sat, Jun 25, 2016 at 05:17:14PM +0200, O. Hartmann wrote:
> >> Is there a way to salvage the situation without relying on "customized"
> >> third party kernels?
> >>
> >> I usually use /bin/csh - so this might be of use.
> >
> > You need either kernel with COMPAT_FREEBSD10, or simpler, newer libc.
> > You could take that libc anywhere, e.g. from the ALPHA5 binaries recently
> > uploaded. There is no way to fix build with the combination of bits you
> > have at present self-hosting.
>
> Can't he boot kernel.old?
Presumably this was tried.
BTW, this situation is a stellar argument to finally switch toolchain
to dynamic linking. For dynamically-linked make, LD_PRELOADing a
dso like this would solve the problem
int
pipe(int[] fds)
{
return (pipe2(fds, 0));
}
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: New FreeBSD snapshots available: head (20160624 r302164)
On Sat, Jun 25, 2016 at 02:13:24AM +, Glen Barber wrote: > New FreeBSD development branch installation ISOs and virtual machine > disk images have been uploaded to the FTP mirrors. > There have been a few reports of "missing" files on the FTP mirrors. This happened last week with the i386 MANIFEST file for base.txz, etc., and FreeBSD-11.0-ALPHA5-arm-armv6-BANANAPI-20160624-r302164.img.xz was missing this week (in addition to other files). I'm looking into the root cause, but if you notice something missing that should exist, please email me privately. (This is not an re@ issue, this is an admins@ issue, and I'm still trying to figure out what is going wrong.) Thanks, and apologies for anything I've missed. Glen signature.asc Description: PGP signature
Re: PostgreSQL performance on FreeBSD
Sean, to the issue that you are describing it is also might be possible to do it some other way around. One, perhaps more portable, is to share a connected socketpair between two communicating processes, so that you can do non-blocking read on one of its ends from time to time and check if it returns EOF. Which would be the case if whatever process holds the other end of it is no longer there. So instead of shared memory segment, you can have pool of descriptors, one for each worker that you care about. Polling on those would be trivial with just regular poll(2). The only issue might be that postgres forks a lot, so we would probably need to implement FD_CLOFORK to avoid copying those extra fds into every child. Something akin to a solution that I recently posted to work around problem that you cannot really waitpid() on a grand-child see PG BUG #14199 for details & patch. But yes, it would be really nice to get rid of SYSV shared memory use in PG completely as some point one way or another. -Max On Thu, Jun 23, 2016 at 3:42 PM, Sean Chittenden wrote: > Small nit: > > PostgreSQL used SYSV because it allowed for the detection of dead > processes. If you `kill -9`’ed a process, PostgreSQL can detect that and > then shut down and perform an automatic recovery. In this regard, sysv is > pretty clever. The move to POSIX shared mem was done for a host of > reasons, but it means that you don’t have to adjust your SYSV limits. My > understanding from a few years ago is that there is still a ~64KB SYSV > memory segment that is still used to act as the latch to signal if a > process was killed, but all of the shared buffers are stored in posix > mmap’ed regions. > > At this point in time this could be replaced with kqueue(2) EVFILT_PROC, > but no one has done that yet. > > -sc > > > > -- > Sean Chittenden > s...@chittenden.org > > > On Jun 22, 2016, at 07:26 , Maxim Sobolev wrote: > > > > Konstantin, > > > > Not if you do sem_unlink() immediately, AFAIK. And that's what PG does. > So > > the window of opportunity for the leakage is quite small, much smaller > than > > for SYSV primitives. Sorry for missing your status update message, I've > > missed it somehow. > > > > > >mySem = sem_open(semname, O_CREAT | O_EXCL, > > (mode_t) IPCProtection, > > (unsigned) 1); > > > > #ifdef SEM_FAILED > >if (mySem != (sem_t *) SEM_FAILED) > >break; > > #else > >if (mySem != (sem_t *) (-1)) > >break; > > #endif > > > >/* Loop if error indicates a collision */ > >if (errno == EEXIST || errno == EACCES || errno == EINTR) > >continue; > > > >/* > > * Else complain and abort > > */ > >elog(FATAL, "sem_open(\"%s\") failed: %m", semname); > >} > > > >/* > > * Unlink the semaphore immediately, so it can't be accessed > > externally. > > * This also ensures that it will go away if we crash. > > */ > >sem_unlink(semname); > > > >return mySem; > > > > > > -Max > > > > On Wed, Jun 22, 2016 at 3:02 AM, Konstantin Belousov < > kostik...@gmail.com> > > wrote: > > > >> On Tue, Jun 21, 2016 at 12:48:00PM -0700, Maxim Sobolev wrote: > >>> Thanks, Konstantin for the great work, we are definitely looking > forward > >> to > >>> get all those improvements to be part of the default FreeBSD > kernel/port. > >>> Would be nice if you can post an update some day later as to what's > >>> integrated and what's not. > >> I did posted the update several days earlier. Since you replying to > this > >> thread, it would be not unreasonable to read recent messages that were > >> sent. > >> > >>> > >>> Just in case, I've opened #14206 with PG to switch us to using POSIX > >>> semaphores by default. Apart from the mentioned performance benefits, > >> SYSV > >>> semaphores are PITA to deal with as they come in very limited > quantities > >> by > >>> default. Also they might stay around if PG dies/gets nuked and prevent > it > >>> from starting again due to overflow. We've got some quite ugly code to > >>> clean up those using ipcrm(1) in our build scripts to deal with just > >> that. > >>> I am happy that code could be retired now. > >> > >> Named semaphores also stuck around if processes are killed without > cleanup. > >> > >> > > ___ > > freebsd-performa...@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-performance > > To unsubscribe, send any mail to " > freebsd-performance-unsubscr...@freebsd.org" > > ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
