Lately after having stability problems probably related to the Nvidia driver, I decided to try the Nouveau driver again. At first I got the same problem: Xorg crashing at startup. Then I edited xorg.conf and commented out all the lines concerning the HDMI output (Device, Monitor, Screen, ServerLayout), which I haven't been actually using, and everything works again.
-- Harri