On 3/6/25 15:49, Daniel P. Berrangé wrote:
On Tue, Jun 03, 2025 at 01:18:55PM +0200, Thomas Huth wrote:
On 22/05/2025 12.29, Daniel P. Berrangé wrote:
The set_pixel_conversion() method is responsible for determining whether
the VNC client pixel format matches the server format, and thus whether
we can use the fast path "copy" impl for sending pixels, or must use
the generic impl with bit swizzling.
The VNC server format is set at build time to VNC_SERVER_FB_FORMAT,
which corresponds to PIXMAN_x8r8g8b8.
The qemu_pixman_get_format() method is then responsible for converting
the VNC pixel format into a pixman format.
The VNC client pixel shifts are relative to the associated endianness.
The pixman formats are always relative to the host native endianness.
The qemu_pixman_get_format() method does not take into account the
VNC client endianness, and is thus returning a pixman format that is
only valid with the host endianness matches that of the VNC client.
...
Hi Daniel,
this patch breaks the output in the TigerVNC viewer for me.
If I run "./qemu-system-x86_64 -vnc :1" on my laptop, and then connect to it
via "vncviewer :1", the output of the BIOS now appears in yellow letters
(instead of grey ones).
It turns out that historically we never set the 'client_be' flag
when a client does NOT send a "set pixel format" message. By luck
this was OK for little endian platforms as the default value of
0 matched little endian.
When I replaced 'client_be' with "client_endian", the default
value of 0 matches neither big or little endian.
I didn't see this with remote-viewer as it unconditionally
sends "set pixel format", but tigervnc always uses the server's
default pixel format.
So this patch is fine, but it exposes a pre-existing latent
bug there was probably causing problems on big endian platforms
in the past, but now causes problems on little endian platforms.
Nice :)
