On Tue, Jun 03, 2025 at 01:18:55PM +0200, Thomas Huth wrote: > On 22/05/2025 12.29, Daniel P. Berrangé wrote: > > The set_pixel_conversion() method is responsible for determining whether > > the VNC client pixel format matches the server format, and thus whether > > we can use the fast path "copy" impl for sending pixels, or must use > > the generic impl with bit swizzling. > > > > The VNC server format is set at build time to VNC_SERVER_FB_FORMAT, > > which corresponds to PIXMAN_x8r8g8b8. > > > > The qemu_pixman_get_format() method is then responsible for converting > > the VNC pixel format into a pixman format. > > > > The VNC client pixel shifts are relative to the associated endianness. > > > > The pixman formats are always relative to the host native endianness. > > > > The qemu_pixman_get_format() method does not take into account the > > VNC client endianness, and is thus returning a pixman format that is > > only valid with the host endianness matches that of the VNC client. > ... > > Hi Daniel, > > this patch breaks the output in the TigerVNC viewer for me. > If I run "./qemu-system-x86_64 -vnc :1" on my laptop, and then connect to it > via "vncviewer :1", the output of the BIOS now appears in yellow letters > (instead of grey ones).
It turns out that historically we never set the 'client_be' flag when a client does NOT send a "set pixel format" message. By luck this was OK for little endian platforms as the default value of 0 matched little endian. When I replaced 'client_be' with "client_endian", the default value of 0 matches neither big or little endian. I didn't see this with remote-viewer as it unconditionally sends "set pixel format", but tigervnc always uses the server's default pixel format. So this patch is fine, but it exposes a pre-existing latent bug there was probably causing problems on big endian platforms in the past, but now causes problems on little endian platforms. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
