It looks like a good start. I really hope this tool will also be usable
with the future hardware testing farms.
Testing on riscv qemu is certainly important as it will provide info
about some kinds of regressions, but it is far from sufficient.
Thanks for this work.
Sebastien
On 08/02/2025 22:19, Tomek CEDRO wrote:
Aaah, so this is already runtime verification!! Awesome!! :-)
The nuttx and nuttx-apps versions compatibility corss-cheks issue:
https://github.com/apache/nuttx/issues/15791
And general top level CI Improvements Issue where we can attach sub issues :-)
https://github.com/apache/nuttx/issues/15792
Thank you Lup!! :-)
Tomek
On Sat, Feb 8, 2025 at 9:30 PM Lee, Lup Yuen <lu...@appkaki.com> wrote:
<< Maybe storing "working" commit and using git bisect would be faster and more
energy efficient? :-) >>
Thanks Tomek! Ah remember that we're fixing Runtime Bugs, not Compile Errors.
Git Bisect won't work if the Runtime Bug is Not Reliably Reproducible:
https://lupyuen.org/articles/bisect.html#good-commit-goes-bad
When we Rewind 20 Commits, we'll know if the bug is Reliably Reproducible. Then
we can run Git Bisect...
<< Do you think that we can also add to our CI test build with older nuttx-app to
cross-check if a change made on nuttx / nuttx-apps does not break nuttx-apps / nuttx ? :-)
>>
Yep sure! Can we add it to our NuttX Roadmap / Wishlist? I need to understand
the requirements. Thanks :-)
Lup
On Sun, Feb 9, 2025 at 4:15 AM Tomek CEDRO <to...@cedro.info> wrote:
On Sat, Feb 8, 2025 at 9:08 PM Lee, Lup Yuen <lu...@appkaki.com> wrote:
If the Daily Test fails for NuttX … Can we Auto-Rewind and discover the
Breaking Commit? Let’s try this:
(1) Every Day: Ubuntu Cron shall trigger a Daily Build and Test of NuttX for
QEMU RISC-V
(2) If The Test Fails: Our Machine will Backtrack The Commits, rebuilding and
retesting each commit (on QEMU Emulator)
(3) When it discovers the Breaking Commit: The Machine shall push to our phone
an Alert Message, that includes the (suspicious) Pull Request
(4) Bonus: The Machine will draft a Polite Note for our NuttX Colleague to
investigate the Pull Request, please
Why are we doing this? If NuttX Fails on QEMU RISC-V: High chance that NuttX
will also fail on RISC-V SBCs like Ox64 BL808 and Oz64 SG2000. Thus it’s
important to Nip the Bud and Fix the Bug early, before it hurts our RISC-V
Devs. (Be Kind, Rewind!)
Check out the article: https://lupyuen.org/articles/rewind.html
Thank You Lup Amazing Work!! :-)
Maybe storing "working" commit and using git bisect would be faster
and more energy efficient? :-)
Do you think that we can also add to our CI test build with older
nuttx-app to cross-check if a change made on nuttx / nuttx-apps does
not break nuttx-apps / nuttx ? :-)
--
CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
--
CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
