Re: libcxx and buildroot toolchain
> > Since Xiang already added support for NuttX on libcxx mainstream: > https://reviews.llvm.org/D88718?id=303700 > > I think it is time to phase out my libcxx repository and we will stick > to libcxx mainline. > > Do you agree on that? In general, yes. However, with your modified version, I was able to build and use libcxx without libsupc++ at all. With the upstream version, libsupc++ is required. I'm still not sure how I was able to get away without this before. It seems building libsupc++.a is not the easiest. The gcc build system is a complicated beast. Instead, I have added support for building the libcxxabi library alongside libcxx. I have now successfully gotten it to build and run, but exception unwinding does not work. Xiang, are you actually using exceptions? Can you confirm whether exception unwinding is actually working for you with gcc and libsupc++.a? Best, Anthony On Sat, Jan 9, 2021 at 11:57 AM Xiang Xiao wrote: > On Sat, Jan 9, 2021 at 8:11 AM Anthony Merlino > wrote: > > > Thanks for the info Xiang! I was on the right track but it is good to > have > > some affirmation that I'm not missing something. > > > > The issue with the Arm maintained toolchain, as Greg has warned many > times, > > is its use of newlib. Are you doing anything to ensure the wrong math > > library is not used? > > > > > Yes, from the theory, the potential conflict may happen since NuttX has its > own libc/libm implementation which is totally different from newlib, but at > least the follow combination is good from the practice: > >1. libc and libm come from NuttX, libsupc++ come from newlib >2. libc come from NuttX, libm and libsupc++ come from newlib > > We use item 2 for all arm based products and sim develop environment. > > I will work on improving buildroot to also build and include libsupcxx.a. > > > > With Alan's modified version of libcxx, he disabled some exception logic > > and maybe some other logic that was allowing me to build without > libsupxx.a > > entirely. > > > > > libcxx will turn on/off exception and rtti automatically, you just need > specify the right compiler flag(e.g. -fno-exceptions and -fno-rtti): > > https://github.com/llvm/llvm-project/blob/main/libcxx/include/__config#L433-L435 > > https://github.com/llvm/llvm-project/blob/main/libcxx/include/__config#L1092-L1098 > > > > Also, may I ask why we are building libcxx by manually grabbing the > source > > files and building it? Are we against just driving the build from their > > existing build system? > > > libcxx uses CMake, it's hard to integrate their build system. But, it will > be great if we can find a clean solution to reuse their build script. > > > > I ask because there are options in the way libcxx is > > built that might be useful to us and we have to replicate all of that if > we > > just grab the source. > > > > > Except the build system integration, all NuttX specific changes are > upstream to the llvm project. We have to apply the following patches: > > https://github.com/apache/incubator-nuttx/blob/master/libs/libxx/0001-libc-Fix-a-few-warnings.patch > > https://github.com/apache/incubator-nuttx/blob/master/libs/libxx/0001-libc-Fix-tests-failing-with-Clang-after-removing-GCC.patch > > https://github.com/apache/incubator-nuttx/blob/master/libs/libxx/0001-libc-NFC-Fix-several-GCC-warnings-in-the-test-suite.patch > > https://github.com/apache/incubator-nuttx/blob/master/libs/libxx/0001-libcxx-Check-_LIBCPP_PROVIDES_DEFAULT_RUNE_TABLE-fir.patch > > https://github.com/apache/incubator-nuttx/blob/master/libs/libxx/0001-libcxx-Port-to-NuttX-https-nuttx.apache.org-RTOS.patch > just because the new libcxx contains our change doesn't release yet. Once > the new release is out, I will remove all patches from the NuttX repo. > > > > > > Best, > > Anthony > > > > > > On Sat, Jan 9, 2021, 7:32 AM Xiang Xiao > wrote: > > > > > On Fri, Jan 8, 2021 at 9:57 AM Anthony Merlino > > > wrote: > > > > > > > Hey all, > > > > > > > > Would someone be willing to share their experience with libcxx and > > their > > > > toolchain? > > > > > > > > > This toolchain should work: > > > > > > > > > https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm > > > > > > > > > > I've now caught up to the tip of master, which now downloads > > > > and builds libcxx for me instead of using Alan's modified version. I > am > > > > facing an issue I've faced before, and would really love to properly > > > > understand how this is supposed to work. > > > > > > > > My issues mostly center around libsupc++.a > > > > > > > > Currently, my build fails because it can't find libsupc++. From my > > > > understanding, this should be provided by my toolchain. But I am > using > > a > > > > toolchain built by the NuttX buildroot and it does not build the C++ > > > > libraries. I am not using RTTI or exceptions, so I don't need what > > > > > > > > > > libsupc++.a doesn't only provide RTTI and exectpions, but
Re: libcxx and buildroot toolchain
Hi Anthony, On 1/12/21, Anthony Merlino wrote: >> >> Since Xiang already added support for NuttX on libcxx mainstream: >> https://reviews.llvm.org/D88718?id=303700 >> >> I think it is time to phase out my libcxx repository and we will stick >> to libcxx mainline. >> >> Do you agree on that? > > > In general, yes. However, with your modified version, I was able to build > and use libcxx without libsupc++ at all. With the upstream version, > libsupc++ is required. I'm still not sure how I was able to get away > without this before. > > It seems building libsupc++.a is not the easiest. The gcc build system is a > complicated beast. > > Instead, I have added support for building the libcxxabi library alongside > libcxx. I have now successfully gotten it to build and run, but exception > unwinding does not work. > Ok, let's to keep it for a while until we have everything working with LLVM libcxx from mainline. BR, Alan
condition variables and signals
Hi, I'm having an issue when waiting on a pthread condition variable from the main thread and then a signal handler runs, which should cancel the wait since I have cancellation points enabled, however this did not happen. While debugging I see the main thread blocked when standing inside the signal handler. Here's the call graph at that point: If I continue execution, the main thread never runs. I'm wondering if that call to nxsem_wait_uninterruptible is correct since it would seem that is exactly why this is not being canceled. Note that I was initially signaling the condition variable from within the signal handler but this did not work either. However, looking at pthread documentation this is supposedly not supported: https://linux.die.net/man/3/pthread_cond_signal (I'm not sure if that really applies to NuttX though). Also, I'm mixing signals with threads because I'm using POSIX timers which require signals to be handled, so I'm stuck with this. I think supporting timerfd would be really useful (as I could then have a thread that waits on the fd of the timer). Best, Matias
Re: condition variables and signals
As a quick test, I made pthread_cond_wait call to pthread_sem_take with the last argument to true (which then use the interruptible version) and it now cancels the wait. So it would seem that this is a bug right? Best, Matias On Tue, Jan 12, 2021, at 20:13, Matias N. wrote: > Hi, > I'm having an issue when waiting on a pthread condition variable from the > main thread and then a signal handler runs, which should cancel the wait > since I have cancellation points enabled, however this did not happen. While > debugging I see the main thread blocked when standing inside the signal > handler. Here's the call graph at that point: > > If I continue execution, the main thread never runs. I'm wondering if that > call to nxsem_wait_uninterruptible is correct since it would seem that is > exactly why this is not being canceled. > > Note that I was initially signaling the condition variable from within the > signal handler but this did not work either. However, looking at pthread > documentation this is supposedly not supported: > https://linux.die.net/man/3/pthread_cond_signal > (I'm not sure if that really applies to NuttX though). > > Also, I'm mixing signals with threads because I'm using POSIX timers which > require signals to be handled, so I'm stuck with this. I think supporting > timerfd would be really useful (as I could then have a thread that waits on > the fd of the timer). > > Best, > Matias > > *Attachments:* > * image.png
Re: condition variables and signals
As a quick test, I made pthread_cond_wait call to pthread_sem_take with the last argument to true (which then use the interruptible version) and it now cancels the wait. So it would seem that this is a bug right? nxsem_wait_uninterruptible() should have no effect on thread cancellation (i.e., it should still return ECANCELED). This only effects signal interrupts (EINTR). So at this superficial level of analysis, nxsem_wait_uninterruptible() should be correct. There is something else going on.
Re: condition variables and signals
I'm having an issue when waiting on a pthread condition variable from the main thread and then a signal handler runs, which should cancel the wait since I have cancellation points enabled, however this did not happen. While debugging I see the main thread blocked when standing inside the signal handler. Here's the call graph at that point: There are two semaphores associated with a condition variable wait: a) One for the conditional variable and b) one for a mutex that protects the condition variable. When you wait for a condition, the mutex (b) is unlocked and the logic waits on (a). After (a) wakes up, then mutex (b) is re-relocked. So there can be two waits. What is not clear from the call graph is if the sem_wait() is the first for the condition variable or the second for the mutex. nxsem_wait_interruptible() is fine for for the first, but not for the second if ECANCELED was returned.
Re: condition variables and signals
I'm having an issue when waiting on a pthread condition variable from the main thread and then a signal handler runs, which should cancel the wait since I have cancellation points enabled, however this did not happen. I don't understand this either. A signal will not, in general, cause a task to be killed. The DEFAULT behavior of some signals (if enabled) will kill the task. However, if you have connected a signal handler via sigaction(), then you have also disabled the default behavior and the task will not be killed.
Re: condition variables and signals
The wait I'm referring to is the one on the semaphore underlying to the condition variable (pthread_sem_take on cond->sem). This ends up as a call to nxsem_wait_uninterruptible which loops on the wait to ignore EINTR errors. So I understand that a signal would not interrupt the wait (and thus, the call to pthread_cond_wait). On Tue, Jan 12, 2021, at 20:34, Gregory Nutt wrote: > > > I'm having an issue when waiting on a pthread condition variable from > > the main thread and then a signal handler runs, which should cancel > > the wait since I have cancellation points enabled, however this did > > not happen. While debugging I see the main thread blocked when > > standing inside the signal handler. Here's the call graph at that point: > > There are two semaphores associated with a condition variable wait: a) > One for the conditional variable and b) one for a mutex that protects > the condition variable. When you wait for a condition, the mutex (b) is > unlocked and the logic waits on (a). After (a) wakes up, then mutex (b) > is re-relocked. So there can be two waits. > > What is not clear from the call graph is if the sem_wait() is the first > for the condition variable or the second for the mutex. > nxsem_wait_interruptible() is fine for for the first, but not for the > second if ECANCELED was returned. > > >
Re: condition variables and signals
I'm not expecting the task to be killed, but the pthread_cond_wait to return when the process receives a signal (which I'm handling in a signal handler). What I need is to exit the pthread_cond_wait upon reception of the signal. As I mentioned, doing pthread_cond_signal from the handler did not work (and seems not to be safe) and now I was trying to enable cancellation points so that the signal itself cancels pthread_cond_wait. Best, Matias On Tue, Jan 12, 2021, at 21:00, Gregory Nutt wrote: > > > > I'm having an issue when waiting on a pthread condition variable from > > the main thread and then a signal handler runs, which should cancel > > the wait since I have cancellation points enabled, however this did > > not happen. > > I don't understand this either. A signal will not, in general, cause a > task to be killed. The DEFAULT behavior of some signals (if enabled) > will kill the task. > > However, if you have connected a signal handler via sigaction(), then > you have also disabled the default behavior and the task will not be killed. > >
Re: condition variables and signals
The wait I'm referring to is the one on the semaphore underlying to the condition variable (pthread_sem_take on cond->sem). This ends up as a call to nxsem_wait_uninterruptible which loops on the wait to ignore EINTR errors. So I understand that a signal would not interrupt the wait (and thus, the call to pthread_cond_wait). When the thread is canceled, nxsem_wait() will return ECANCELED. In that case, nxsem_wait_interruptible() will return the error. If you continue to loop, then cancellation is not ocurring (i.e., EINTR is received not ECANCELED). This would be the case if, for example, you attached a signal handler to the signal used for cancellation
Re: condition variables and signals
I'm not expecting the task to be killed, but the pthread_cond_wait to return when the process receives a signal (which I'm handling in a signal handler). What I need is to exit the pthread_cond_wait upon reception of the signal. As I mentioned, doing pthread_cond_signal from the handler did not work (and seems not to be safe) and now I was trying to enable cancellation points so that the signal itself cancels pthread_cond_wait. pthread_cond_wait() is not supposed to return if a signal is received: https://pubs.opengroup.org/onlinepubs/009696699/functions/pthread_cond_wait.html EINTR is not one of the valid return values from pthread_cond_wait().
Re: condition variables and signals
pthread_cond_wait() is not supposed to return if a signal is received: https://pubs.opengroup.org/onlinepubs/009696699/functions/pthread_cond_wait.html EINTR is not one of the valid return values from pthread_cond_wait(). This is the specific POSIX requirement that must be followed: "If a signal is delivered to a thread waiting for a condition variable, upon return from the signal handler the thread resumes waiting for the condition variable as if it was not interrupted, or it shall return zero due to spurious wakeup." "Spurious wakeups" are not handled.
Re: condition variables and signals
Ahh ok. Thanks for that, should've looked further into the spec. I assumed from other documentation I read that it could work. So, now I'm a bit stuck on mixing the signal-based POSIX timers with mutexes. I will see about other synchronization mechanism. Best, Matias On Tue, Jan 12, 2021, at 21:19, Gregory Nutt wrote: > > > pthread_cond_wait() is not supposed to return if a signal is received: > > https://pubs.opengroup.org/onlinepubs/009696699/functions/pthread_cond_wait.html > > > > EINTR is not one of the valid return values from pthread_cond_wait(). > > This is the specific POSIX requirement that must be followed: "If a > signal is delivered to a thread waiting for a condition variable, upon > return from the signal handler the thread resumes waiting for the > condition variable as if it was not interrupted, or it shall return zero > due to spurious wakeup." > > "Spurious wakeups" are not handled. > >
Re: condition variables and signals
> Note that I was initially signaling the condition variable from within the > signal handler but this did not work either. However, looking at pthread > documentation this is supposedly not supported: > https://linux.die.net/man/3/pthread_cond_signal > (I'm not sure if that really applies to NuttX though). > BTW, looking at the spec for pthread_cond_wait, there's actually no mention about a limitation regarding using pthread_cond_signal invoked from within a signal handler to unblock a pthread_cond_wait. However, this does not work either for me. If you follow the signal operation, I can see that when it reaches the nxsem_get_value() on the cond->sem, the semaphore value is zero and the sem_give is not done. Should this work? Best, Matia
Re: condition variables and signals
BTW, looking at the spec for pthread_cond_wait, there's actually no mention about a limitation regarding using pthread_cond_signal invoked from within a signal handler to unblock a pthread_cond_wait. However, this does not work either for me. If you follow the signal operation, I can see that when it reaches the nxsem_get_value() on the cond->sem, the semaphore value is zero and the sem_give is not done. Should this work? Yes, calling pthread_cond_signal() should work, however, no signal should be delivered. The word "signal" is overloaded: Here the condition is signaled but no signal should be sent so I am not quite sure what I am looking at. Apparently some other logic is signaling the task group asynchronously? I am thinking that there must be some race condition: When the condition variable is created the cond->sem value will be set to zero. While waiting for cond->sem, it will be set to -1. If you see cond->sem equal to zero, my guess would be that the signal was received and cond->sem was incremented asychronously by the signal delivery logic. But that is only a guess. It would be good to check the behavior without the signal interfering.
Re: libcxx and buildroot toolchain
On Wed, Jan 13, 2021 at 12:44 AM Anthony Merlino
wrote:
> >
> > Since Xiang already added support for NuttX on libcxx mainstream:
> > https://reviews.llvm.org/D88718?id=303700
> >
> > I think it is time to phase out my libcxx repository and we will stick
> > to libcxx mainline.
> >
> > Do you agree on that?
>
>
> In general, yes. However, with your modified version, I was able to build
> and use libcxx without libsupc++ at all. With the upstream version,
> libsupc++ is required. I'm still not sure how I was able to get away
> without this before.
>
> It seems building libsupc++.a is not the easiest. The gcc build system is a
> complicated beast.
>
> Instead, I have added support for building the libcxxabi library alongside
> libcxx. I have now successfully gotten it to build and run, but exception
> unwinding does not work.
>
> Xiang, are you actually using exceptions? Can you confirm whether exception
> unwinding is actually working for you with gcc and libsupc++.a?
>
>
Yes, the exception can work as expect, I just try cxxtest with
CONFIG_CXX_EXCEPTION:
https://github.com/apache/incubator-nuttx-apps/tree/master/testing/cxxtest
Here is the output from a board with Cortex M33:
test ofstream=== printf: Starting test_ostream printf:
Successfully opened /dev/console cout: Successfully opened /dev/console Writing
this to /dev/console test iostream=== Hello, this
is only a test Print an int: 190 Print a char: d test
vector= v1=1 2 3 Hello World Good Luck test
map test rtti===
extend test exception== Catch exception: runtime
error
The simulator has the same output too.
BTW, to make the exception work, the link script need .ARM.extab
and.ARM.exidx sections:
.ARM.extab : ALIGN(4) {
*(.ARM.extab*)
} > ROM
.ARM.exidx : ALIGN(4) {
__exidx_start = ABSOLUTE(.);
*(.ARM.exidx*)
__exidx_end = ABSOLUTE(.);
} > ROM
> Best,
> Anthony
>
>
>
>
>
> On Sat, Jan 9, 2021 at 11:57 AM Xiang Xiao
> wrote:
>
> > On Sat, Jan 9, 2021 at 8:11 AM Anthony Merlino
> > wrote:
> >
> > > Thanks for the info Xiang! I was on the right track but it is good to
> > have
> > > some affirmation that I'm not missing something.
> > >
> > > The issue with the Arm maintained toolchain, as Greg has warned many
> > times,
> > > is its use of newlib. Are you doing anything to ensure the wrong math
> > > library is not used?
> > >
> > >
> > Yes, from the theory, the potential conflict may happen since NuttX has
> its
> > own libc/libm implementation which is totally different from newlib, but
> at
> > least the follow combination is good from the practice:
> >
> >1. libc and libm come from NuttX, libsupc++ come from newlib
> >2. libc come from NuttX, libm and libsupc++ come from newlib
> >
> > We use item 2 for all arm based products and sim develop environment.
> >
> > I will work on improving buildroot to also build and include libsupcxx.a.
> > >
> > > With Alan's modified version of libcxx, he disabled some exception
> logic
> > > and maybe some other logic that was allowing me to build without
> > libsupxx.a
> > > entirely.
> > >
> > >
> > libcxx will turn on/off exception and rtti automatically, you just need
> > specify the right compiler flag(e.g. -fno-exceptions and -fno-rtti):
> >
> >
> https://github.com/llvm/llvm-project/blob/main/libcxx/include/__config#L433-L435
> >
> >
> https://github.com/llvm/llvm-project/blob/main/libcxx/include/__config#L1092-L1098
> >
> >
> > > Also, may I ask why we are building libcxx by manually grabbing the
> > source
> > > files and building it? Are we against just driving the build from their
> > > existing build system?
> >
> >
> > libcxx uses CMake, it's hard to integrate their build system. But, it
> will
> > be great if we can find a clean solution to reuse their build script.
> >
> >
> > > I ask because there are options in the way libcxx is
> > > built that might be useful to us and we have to replicate all of that
> if
> > we
> > > just grab the source.
> > >
> > >
> > Except the build system integration, all NuttX specific changes are
> > upstream to the llvm project. We have to apply the following patches:
> >
> >
> https://github.com/apache/incubator-nuttx/blob/master/libs/libxx/0001-libc-Fix-a-few-warnings.patch
> >
> >
> https://github.com/apache/incubator-nuttx/blob/master/libs/libxx/0001-libc-Fix-tests-failing-with-Clang-after-removing-GCC.patch
> >
> >
> https://github.com/apache/incubator-nuttx/blob/master/libs/libxx/0001-libc-NFC-Fix-several-GCC-warnings-in-the-test-suite.patch
> >
> >
> https://github.com/apache/incubator-nuttx/blob/master/libs/libxx/0001-libcxx-Check-_LIBCPP_PROVIDES_DEFAULT_RUNE_TABLE-fir.patch
> >
> >
> https://github.com/apache/incubator-nuttx/blob/master/libs/libxx/0001-libcxx-Port-to-NuttX-https-nuttx.apache.org-RTOS.patch
> > just because the new libcxx cont
