On both macOS and Linux, the default send buffer size too small causing poor performance when reading and writing to qemu-nbd. A simple way to experience this is to compare TCP and unix sockets, showing that TCP socket is much faster. Programs like nbdcopy partly mitigate this by using multiple NBD connections.
On macOS the default send buffer size is 8192 bytes. Increasing the send buffer size to 2 MiB shows up to *12.6 times higher throughput* and lower cpu usage. On Linux the default and maximum buffer size is 212992 bytes. Increasing the send buffer size to 2 MiB shows up to *2.7 times higher throughput* and lower cpu usage. On older machine we see very little improvement, up to 1.03 times higher throughput. We likely have the same issue on other platforms. It should be easy to enable this change for more platform by defining UNIX_STREAM_SOCKET_SEND_BUFFER_SIZE. Changes since v2: - Test with different receive buffer size (Daniel) - Test with larger send buffer size (2m, 4m) - Set only send buffer size - setting receive buffer size has no effect - Increase send buffer size to 2m (based on new tests) - Enable the change also for Linux - Change only unix stream socket - datagram socket need different configuration - Modify the code to make it easy to support unix datagram socket v2 was here: https://lists.gnu.org/archive/html/qemu-devel/2025-04/msg03167.html Nir Soffer (2): io: Increase unix socket buffers size on macOS io: Increase unix socket buffers on Linux io/channel-socket.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) -- 2.39.5 (Apple Git-154)
