On Tue, Nov 21, 2023 at 09:56:24AM +0100, Thomas Huth wrote:
> On 17/11/2023 08.35, Nicholas Piggin wrote:
> > On Fri Nov 17, 2023 at 4:11 AM AEST, Thomas Huth wrote:
> > > On 16/11/2023 12.53, Nicholas Piggin wrote:
> > > > Let's try enable reverse_debugging.py in gitlab CI.
> > > >
> > > > Signed-off-by: Nicholas Piggin <npig...@gmail.com>
> > > > ---
> > > > Maybe we could try this again at some point? The bug might have been
> > > > noticed sooner.
> > > >
> > > > They only take a couple of seconds to run so should not take too much
> > > > overhead. But my gitlab CI pipeline doesn't run the avocado tests for
> > > > some reason, so I can't see if it's still causing problems.
> > > >
> > > > Thanks,
> > > > Nick
> > > > ---
> > > >
> > > > tests/avocado/reverse_debugging.py | 7 -------
> > > > 1 file changed, 7 deletions(-)
> > >
> > > FYI, I gave it a try, and it survived my CI run:
> > >
> > > https://gitlab.com/thuth/qemu/-/jobs/5552213972#L403
> > >
> > > So I went ahead and put it (together with the first patch) in my current
> > > pull request, let's see how it goes...
> >
> > Great, thank you.
>
> ... and here it's failing again (current master branch):
>
> https://gitlab.com/thuth/qemu/-/jobs/5582657378#L404
>
> According to the debug.log in the artifacts, it's failing here:
>
> 08:28:32 DEBUG| [ 0.230392217,5] OPAL v7.0 starting...
>
> 08:28:32 DEBUG| [ 0.230674939,7] initial console log level: memory 7,
> driver 5
>
> 08:28:32 DEBUG| [ 0.231048494,6] CPU: P9 generation processor (max 4
> threads/core)
>
> 08:28:32 DEBUG| [
> 08:28:32 DEBUG| [ 0.231412547,7] CPU: Boot CPU PIR is 0x0000 PVR is
> 0x004e1202
>
> 08:28:32 DEBUG| [
> 08:28:32 ERROR|
> 08:28:32 ERROR| Reproduced traceback from:
> /builds/thuth/qemu/build/pyvenv/lib64/python3.8/site-packages/avocado/core/test.py:770
> 08:28:32 ERROR| Traceback (most recent call last):
> 08:28:32 ERROR| File
> "/builds/thuth/qemu/build/tests/avocado/reverse_debugging.py", line 262, in
> test_ppc64_powernv
> 08:28:32 ERROR| self.reverse_debugging()
> 08:28:32 ERROR| File
> "/builds/thuth/qemu/build/tests/avocado/reverse_debugging.py", line 178, in
> reverse_debugging
> 08:28:32 ERROR| g.cmd(b'c')
> 08:28:32 ERROR| File
> "/builds/thuth/qemu/build/pyvenv/lib64/python3.8/site-packages/avocado/utils/gdb.py",
> line 783, in cmd
> 08:28:32 ERROR| response_payload = self.decode(result)
> 08:28:32 ERROR| File
> "/builds/thuth/qemu/build/pyvenv/lib64/python3.8/site-packages/avocado/utils/gdb.py",
> line 738, in decode
> 08:28:32 ERROR| raise InvalidPacketError
> 08:28:32 ERROR| avocado.utils.gdb.InvalidPacketError
> 08:28:32 ERROR|
> 08:28:32 DEBUG| Local variables:
> 08:28:32 DEBUG| -> self <class 'reverse_debugging.ReverseDebugging_ppc64'>:
> 79-tests/avocado/reverse_debugging.py:ReverseDebugging_ppc64.test_ppc64_powernv
> 08:28:32 DEBUG| Shutting down VM appliance; timeout=30
> 08:28:32 DEBUG| Attempting graceful termination
> 08:28:32 DEBUG| Closing console socket
> 08:28:32 DEBUG| Politely asking QEMU to terminate
>
> So unless someone has a clue how to fix that, I guess it's
> likely best to revert this enablement patch again...
A little further in the log we see
08:28:32 DEBUG| Politely asking QEMU to terminate
08:28:32 DEBUG| --> {
"execute": "quit"
}
08:28:32 DEBUG| <-- {
"timestamp": {
"seconds": 1700555312,
"microseconds": 86122
},
"event": "RESUME"
}
08:28:32 ERROR| Task.Reader: BrokenPipeError: [Errno 32] Broken pipe
With seeing a bad packet from GDB and seeing Broken pipe from QMP,
my impression is that the QEMU process is no longer present, most
likely it has SEGV'd I reckon.
IOW, I think we might well have a genuine bug here, not merely an
unreliable test suite.
None the less, unless someone can guess what the problem is, we'll
need to disable the test to get reliable CI.
A bug should be opened though with the CI logs.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
