Hi everyone, I just enabled CONFIG_ARMV7M_STACKCHECK, as I would like to have this functionality in my project, but I am facing problems.
NuttX is not able to boot at all with this option selected. I stepped through the code and it seems that nxsig_initialize actually causes a stack overflow that is detected by the above check. Is this a bug, or I should configure something in a different way? Then I realized that __stack_overflow_trap is broken. When a stack overflow happens, this function is called which is supposed to cause a hardfault. However as __stack_overflow_trap is a function itself, __cyg_profile_func_enter is called again. Once again it detects the overflow and calls __stack_overflow_trap, and so on... Finally, as I see, the stack check is performed while entering a function, which is wrong. If there is a stack overflow, it will be detected at the next function call, which may be at a irrelevant part of the code. I believe that the check shall be performed on the exit of a function, in which case you will be sure that this specific function caused the overflow. And of course it will solve the issue above with __stack_overflow_trap.
