There are some files which I imagine are part of one of the X client or server packages, but they are unaccounted for in dpkg. I imagine that these could be marked as conffiles. Is there a reason that dpkg shouldn't know about these?
$ dpkg -S /etc/X11/X /etc/X11/XF86Config-4 /etc/X11/Xwrapper.config /etc/X11/default-display-manager dpkg: /etc/X11/X not found. dpkg: /etc/X11/XF86Config-4 not found. dpkg: /etc/X11/Xwrapper.config not found. dpkg: /etc/X11/default-display-manager not found. Frank