[issue39174] unicodedata.normalize failing with NFD and NFKD for some characters in Python3

2020-01-05 Thread Lee Collins
Lee Collins added the comment: I did some more investigation by running cat on a file with the decomposed characters and saw that the output was composed. So, this does look like a problem with the Mac OS terminal. It can be resolved as 3rd party -- resolution: -> third party st

[issue39174] unicodedata.normalize failing with NFD and NFKD for some characters in Python3

2019-12-31 Thread Lee Collins
Lee Collins added the comment: On further investigation, it appears that the problem is the interaction between Python3 and the MacOS terminal. unicodedata.normalize() produces the correct sequence u'a\u0300' but when printed it comes out

[issue39174] unicodedata.normalize failing with NFD and NFKD for some characters in Python3

2019-12-31 Thread Lee Collins
New submission from Lee Collins : A script that works in 2.7.17 is now failing for some Unicode characters in 3.7.5 on MacOS 10.14.6. For example unicodedata.normalize('NFD', 'à') used to return the correct decomposition u'a\u0300', but in 3.7 it returns the sin