Further information regarding this issue:

With the default GDM configuration, there is no X server active on the 
login screen, because the GDM greeter is using Wayland.  When you log in, 
either Xwayland or Xorg (depending on whether you chose a Wayland or an 
Xorg session) is started on Display :0, so there is no possibility of a 
conflict with TurboVNC.  However, if you use vglserver_config in VirtualGL 
to configure the host for use with VirtualGL's GLX back end, 
vglserver_config will disable Wayland in GDM.  That causes the GDM greeter 
to use Xorg rather than Wayland, and the greeter's Xorg instance listens on 
Display :0.  Thus, VirtualGL can use the greeter's X server to access the 
GPU while the host is sitting at the login prompt.

On hosts configured thusly (with WaylandEnable=false in GDM's custom.conf 
file), a conflict occurs if you start a TurboVNC session while the host is 
sitting at the login prompt, then you log in locally.  When you start a 
TurboVNC session while the host is sitting at the login prompt, TurboVNC 
(rightfully) chooses Display :1 for the session, because nothing is using 
the resources associated with Display :1.  However, if you then log in 
locally, GDM starts a second Xorg instance for the local session (so as not 
to conflict with the greeter's Xorg instance.)  GDM indiscriminately 
chooses Display :1 for the local session without checking whether something 
else is already using the resources associated with Display :1.  Thus, it 
effectively stomps all over the TurboVNC session that is listening on 
Display :1.  That causes the problems that you observed.

I can (and will) make the vncserver script more robust, in the sense that 
it will check whether an abstract socket for a display number is in use 
before deciding to use that display number.  However, GDM needs to do 
likewise.

The only apparent workarounds for this are:

1. If you have used vglserver_config to configure a host for use with 
VirtualGL's GLX back end, then be very cautious when logging in locally.  
That is true irrespective of TurboVNC, because logging in locally on such a 
host will cause GDM to suspend the greeter's X server, which has the effect 
of causing any applications currently running with VirtualGL to freeze.  
Use '/opt/TurboVNC/bin/vncserver -list' to ensure that no TurboVNC sessions 
are listening on :1 before you log in locally, and avoid local logins if at 
all possible.

2. Comment out 'WaylandEnable=false' in /etc/gdm3/custom.conf, and restart 
GDM.  This is not true of all modern Linux systems, but with Ubuntu 22 at 
least, GDM will continue to start an Xorg instance at the login prompt even 
though the greeter is using Wayland.  You can set VGL_DISPLAY=:1024 to use 
that Xorg instance as a 3D X server with VirtualGL's GLX back end.

I'm also going to investigate whether it might be possible for VirtualGL to 
play more nicely with Wayland on modern Linux distributions.

On Tuesday, December 19, 2023 at 1:40:38 PM UTC-5 DRC wrote:

> No, by deleting /tmp/.X11-unix/X1, I was able to exactly reproduce the 
> issue you reported with the Session Manager.  That error occurred when 
> the Session Manager tried to start a new TurboVNC session through SSH.  
> In order for Port=5901 to have any effect, it would either have to be 
> specified in the default TurboVNC connection info file 
> (~/.vnc/default.turbovnc), or it would have to be specified in another 
> connection info file (either a TightVNC-compatible connection info file, 
> which has a .vnc extension, or a TurboVNC connection info file, which 
> has a .turbovnc extension) that you pass to the TurboVNC Viewer on the 
> command line.  Otherwise, the mere existence of such a connection info 
> file on the machine wouldn't matter.  Also, the error occurred prior to 
> the TurboVNC Viewer attempting to establish the RFB connection with the 
> TurboVNC Server, so the viewer had not yet used the value of the Port 
> parameter.
>
> DRC
>
> On 12/19/23 12:40 PM, Torsten Kupke wrote:
> > Hi CRC,
> >
> > Am 19.12.2023 um 16:41 schrieb 'DRC' via TurboVNC User 
> > Discussion/Support:
> >> Apparently something happened that caused /tmp/.X11-unix/X1 to be 
> >> deleted, so the vncserver script didn't know that the primary X 
> >> server was using Display :1, and it tried to use that display number 
> >> itself.
> >
> > Maybe I have another explantion for this: I have a .vnc file stored on 
> > that machine specifying "port=5901". I think, the viewer found this 
> > file and therefore tried to connect to the XServer :1. Could this be 
> > the case? So if :1 now is in use by the native XServer, I simply would 
> > have to specify "port=5902" in my .vnc file. Can you agree? Would this 
> > solve my issue?
> >
> > However, I'm on Christmas vacation now. So I will need a proper 
> > solution not before 2nd of Jan. '24.
> >
> > Merry Christmas and a happy new year for you and all other readers!
> >
> > tkansgar
> >
>

-- 
You received this message because you are subscribed to the Google Groups 
"TurboVNC User Discussion/Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to turbovnc-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/turbovnc-users/2e98ca31-b6d8-4971-bc59-1f1edf5f508en%40googlegroups.com.

Reply via email to