I have experienced much the same issue: I leave an X session to run as another user on another VT, open X for that user, exit X, switch back to the original session and it crashes. The log shows:
II) AIGLX: Suspending AIGLX clients for VT switch [dix] couldn't enable device 9 [dix] couldn't enable device 10 [dix] couldn't enable device 11 [dix] couldn't enable device 8 [dix] couldn't enable device 6 [dix] couldn't enable device 7 (II) AIGLX: Suspending AIGLX clients for VT switch (EE) Fatal server error: (EE) EnterVT failed for screen 0 (EE) (EE) Please consult the The X.Org Foundation support_ at http://wiki.x.org for help.