Marc-Andre Lemburg <[email protected]> added the comment: STINNER Victor wrote: > > STINNER Victor <[email protected]> added the comment: > >> I think it would be best to backport the handler (even though >> it is not needed in Python 2.7), since it makes porting apps >> to 3.x easier. > > surrogateescape should not be used directly be applications. It's used by > Python3 internals using unicode by default. > > I don't know if it's would help porting applications from Python2 to Python3. > I don't know a use case of surrogateescape in Python2. By default, Python2 > uses byte string everywhere, especially for filenames, and so it doesn't need > any unicode error handler. > > Another point to consider is that utf8 encoder rejects surrogates in Python3, > whereas surrogates are accepted by the Python2 utf8 encoder.
Sorry, I think I need to correct myself: I mixed up the handlers surrogateescape and surrogatepass. I was actually thinking of the surrogatepass handler which makes the Python3 UTF-8 codec have like the Python2 UTF-8 codec (without extra handler), not the surrogatescape handler which implements the UTF-8b logic of escaping non-encodable bytes to lone surrogates. * The surrogatepass handler is needed in Python 2.7 to make it possible to write applications that work in both 2.7 and 3.x without changing the code. I consider this an important missing backport for 2.7, since without this handler, the UTF-8 codecs in 2.7 and 3.x are incompatible and there's no other way to work around this other than to make use of the errorhandler conditionally depend on the Python version. As such, it's a bug rather than a new feature. * The surrogateescape handler implements the UTF-8b escaping logic: b'\x91\x92' In Python 3.x this is needed to work around problems with wrong I/O encoding settings or situations where you have mixed encoding settings used in external resources such as environment variable content, filesystems using different encodings than the system one, remote shell output, pipes which don't carry any encoding information, etc. etc. Backporting this handler would be useful for Python 2.7 as well, since it allows preparing 2.7 applications for use in 3.x and again allows using the same code for 2.7 and 3.x. Not having this handler in 2.7 is not as serious as the surrogatepass handler, but still useful for applications to use that are meant to run in 2.7 and 3.x unchanged. ---------- _______________________________________ Python tracker <[email protected]> <http://bugs.python.org/issue8438> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
