I never used virtual environments and wouldn't like to start with it.
There's your problem - everything else is a result of this. There is just no nice way to work with a combination of pypi, apt-get and system-wide python.
Everyone uses virtual environments. Sorry. -- https://mail.python.org/mailman/listinfo/python-list