Hello, Weird issue I've found on Windows images in Azure Devops Pipelines and Github actions. Printing Unicode characters fails on these images because, for some reason, the encoding is mapped to cp1252. What is particularly weird about the code page being set to 1252 is that if you execute "chcp" it shows that the code page is 65001.
At the end of this email are the cleaned up logs from GH actions. The actions are very simple - print out unicode characters using echo to prove the characters can be printed to the console. The rest of the commands are in Python, and they include printing out the "encoding" variable of sys.stdout, as well as printing sys.flags and sys.getfilesystemencoding. Then print the same unicode character using print, which causes a UnicodEncodeError because the character isn't in the cp1252 charmap. I've also uploaded the logs to pastebin here: https://pastebin.com/ExzGRHav I also uploaded a screenshot to imgur, since the logs are not the easiest to read. https://imgur.com/a/dhvLWOJ I'm trying to determine why this issue only happens on these images - I can replicate it on multiple versions of Python (from 3.9 to 3.7 at least, haven't tried more), but I can't replicate this on my own machines. There are a few issues on GH regarding this issue but they seem to stay open since they are hard to replicate. Here are the ones I have stumbled upon while researching this. https://github.com/databrickslabs/dbx/issues/455 https://github.com/PrefectHQ/prefect/issues/5754 https://github.com/pallets/click/issues/2121 Any insight or ideas on how to test and validate the cause would be great. I'm pulling my hair out trying to find the root cause of this - not because it really matters to any of my processes but because it is weird and broken. Thanks for any help, Jessica Begin Logs: 2022-11-10T23:54:51.7272453Z Requested labels: windows-latest 2022-11-10T23:54:51.7272494Z Job defined at: NodeJSmith/wsl_home/.github/workflows/blank.yml@refs/heads/main 2022-11-10T23:54:51.7272514Z Waiting for a runner to pick up this job... 2022-11-10T23:54:52.3387510Z Job is waiting for a hosted runner to come online. 2022-11-10T23:55:04.8574435Z Job is about to start running on the hosted runner: Hosted Agent (hosted) 2022-11-10T23:55:15.8332600Z Current runner version: '2.298.2' 2022-11-10T23:55:15.8366947Z ##[group]Operating System 2022-11-10T23:55:15.8367650Z Microsoft Windows Server 2022 2022-11-10T23:55:15.8367954Z 10.0.20348 2022-11-10T23:55:15.8368389Z Datacenter 2022-11-10T23:55:15.8368696Z ##[endgroup] 2022-11-10T23:55:15.8369023Z ##[group]Runner Image 2022-11-10T23:55:15.8369654Z Image: windows-2022 2022-11-10T23:55:15.8369931Z Version: 20221027.1 2022-11-10T23:55:15.8370539Z Included Software: https://github.com/actions/runner-images/blob/win22/20221027.1/images/win/Windows2022-Readme.md 2022-11-10T23:55:15.8371174Z Image Release: https://github.com/actions/runner-images/releases/tag/win22%2F20221027.1 2022-11-10T23:55:15.8371622Z ##[endgroup] 2022-11-10T23:55:15.8371955Z ##[group]Runner Image Provisioner 2022-11-10T23:55:15.8372277Z 2.0.91.1 2022-11-10T23:55:15.8372514Z ##[endgroup] 2022-11-10T23:55:16.3619998Z ##[group]Run echo " └── ID:" 2022-11-10T23:55:16.3620626Z echo " └── ID:" 2022-11-10T23:55:16.3927292Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'" 2022-11-10T23:55:16.3927894Z ##[endgroup] 2022-11-10T23:55:32.9958751Z └── ID: 2022-11-10T23:55:34.0835652Z ##[group]Run chcp 2022-11-10T23:55:34.0836104Z chcp 2022-11-10T23:55:34.0878901Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'" 2022-11-10T23:55:34.0879350Z ##[endgroup] 2022-11-10T23:55:34.4878247Z Active code page: 65001 2022-11-10T23:55:34.7917219Z ##[group]Run python -c "import sys; print('sys.stdout.encoding', sys.stdout.encoding); print('sys.flags',sys.flags);print('sys.getfilesystemencoding',sys.getfilesystemencoding())" 2022-11-10T23:55:34.7918148Z python -c "import sys; print('sys.stdout.encoding', sys.stdout.encoding); print('sys.flags',sys.flags);print('sys.getfilesystemencoding',sys.getfilesystemencoding())" 2022-11-10T23:55:34.7960873Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'" 2022-11-10T23:55:34.7961202Z ##[endgroup] 2022-11-10T23:55:36.2324642Z sys.stdout.encoding cp1252 2022-11-10T23:55:36.2325910Z sys.flags sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0, dev_mode=False, utf8_mode=0) 2022-11-10T23:55:36.2327055Z sys.getfilesystemencoding utf-8 2022-11-10T23:55:36.4553957Z ##[group]Run python -c "print('└── ID:')" 2022-11-10T23:55:36.4554395Z python -c "print('└── ID:')" 2022-11-10T23:55:36.4595413Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'" 2022-11-10T23:55:36.4595740Z ##[endgroup] 2022-11-10T23:55:36.8739309Z Traceback (most recent call last): 2022-11-10T23:55:37.1316425Z File "<string>", line 1, in <module> 2022-11-10T23:55:37.1317452Z File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\encodings\cp1252.py", line 19, in encode 2022-11-10T23:55:37.1324632Z return codecs.charmap_encode(input,self.errors,encoding_table)[0] 2022-11-10T23:55:37.1325913Z UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-2: character maps to <undefined> -- https://mail.python.org/mailman/listinfo/python-list
