[issue1602] windows console doesn't print or input Unicode
stijn added the comment:
New here, but I think this is the correct issue to get info about this unicode
problem. On the windows console:
> chcp
Active code page: 437
> type utf.txt
Привет
> chcp 65001
Active code page: 65001
> type utf.txt
Привет
> python --version
Python 3.5.0a0
> cat utf.py
f = open('utf.txt')
l = f.readline()
print(l)
print(len(l))
> python utf.py
Привет
�²ÐµÑ‚
�‚
13
> cat utf_explicit.py
import codecs
f = codecs.open('utf.txt', encoding='utf-8', mode='r')
l = f.readline()
print(l)
print(len(l))
> python utf_explicit.py
Привет
ет
7
I partly read through the page but these things are a bit above my head. Could
anyone explain
- how to figure out what codec files returned by open()?
- is there a way to change it globally to utf-8?
- the last case is almost correct: it has the correct number of characters, but
the print() still does something wrong. I got this working by using the stream
patch, but got another example on which is is not correct, see below. Any way
around this?
> type utf2.txt
aαbβcγdδ
> cat utf2.py
import streams
import codecs
streams.enable()
f = codecs.open('utf2.txt', encoding='utf-8', mode='r')
print(f.read(1))
print(f.read(1))
print(f.read(2))
print(f.read(4))
> python utf2.py
a
α
bβc
γdδ
--
nosy: +stijn
___
Python tracker
<http://bugs.python.org/issue1602>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
stijn added the comment: Drekin: you're right for both input and output. Using encoding with plain open() works just fine and using the latest win-unicode-console does give correct output for the second example as well. Thanks! -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5004] socket.getfqdn() doesn't cope properly with purely DNS-based setups
Stijn Hoop added the comment: I hereby put my patch in the public domain and/or under any desired copyright license as required by the Python project to accept it. Regards, Stijn Hoop On Fri, 22 Oct 2021 21:03:26 + Richard van den Berg wrote: > Richard van den Berg added > the comment: > > In that case Stijn Hope should create the PR since he wrote the > patch. Anyone else could get in trouble for using his code without > proper permission. > > -- > > ___ > Python tracker > <https://bugs.python.org/issue5004> > ___ -- nosy: +shoop ___ Python tracker <https://bugs.python.org/issue5004> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5004] socket.getfqdn() doesn't cope properly with purely DNS-based setups
Stijn Hoop added the comment: Still seeing this on Fedora 18 / Python 2.7.3. I only have loopback in /etc/hosts [TUE\shoop@pclin281] <~> cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 I search in .campus.tue.nl and .win.tue.nl: [TUE\shoop@pclin281] <~> grep search /etc/resolv.conf search campus.tue.nl. win.tue.nl. Hostname -f reliably returns .campus.tue.nl as it should [TUE\shoop@pclin281] <~> hostname -f pclin281.campus.tue.nl [TUE\shoop@pclin281] <~> hostname -f pclin281.campus.tue.nl But socket.getfqdn disagrees, even with itself when run multiple times: [TUE\shoop@pclin281] <~> python Python 2.7.3 (default, Aug 9 2012, 17:23:57) [GCC 4.7.1 20120720 (Red Hat 4.7.1-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> socket.getfqdn() 'pclin281' >>> socket.getfqdn() 'pclin281.win.tue.nl' >>> socket.getfqdn() 'pclin281' >>> socket.getfqdn() 'pclin281.win.tue.nl' >>> Note that pclin281.win.tue.nl is in fact also a valid DNS entry, but not one that I would expect the function to ever return given the search order. -- nosy: +Stijn.Hoop ___ Python tracker <http://bugs.python.org/issue5004> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5004] socket.getfqdn() doesn't cope properly with purely DNS-based setups
Stijn Hoop added the comment: OK, fair enough. >From reading sources, it appears that hostname is using getaddrinfo(3) on >kernelhostname with hints->ai_flags & AI_CANONNAME, while Lib/socket.py simply >uses gethostbyaddr(kernelhostname), and falls back on kernelhostname in case >of errors. Unfortunately I am not entirely sure who is "correct" here, as I don't know the intent of socket.getfqdn(). In my case, kernelhostname is set to 'pclin281' e.g. without the dots. I believe this to be correct, but I know that this is already "controversial" as in there exists software that expects an FQDN there, and internet folklore is split about 50/50 about this necessity. Then, apparently, there is confusion about AI_CANONNAME and what it actually should do. glibc upstream does address lookups but Fedora patches this out. See this recent glibc bug for more pointers: http://sourceware.org/bugzilla/show_bug.cgi?id=15218 As mentioned in that bug, a lot of software runs on Fedora and works using that definition of AI_CANONNAME. However, switching Lib/socket.py / getfqdn from gethostbyaddr to getaddrinfo might have more implications than just fixing this case. I can try to write a patch, but is this the right direction? -- ___ Python tracker <http://bugs.python.org/issue5004> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5004] socket.getfqdn() doesn't cope properly with purely DNS-based setups
Stijn Hoop added the comment: Attached is a very lightly tested patch that matches hostname -f behaviour on my system. I suspect this should be OK but it definitely needs more testing than just my system... -- keywords: +patch Added file: http://bugs.python.org/file29919/python2.7-socket-getfqdn.patch ___ Python tracker <http://bugs.python.org/issue5004> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5004] socket.getfqdn() doesn't cope properly with purely DNS-based setups
Stijn Hoop added the comment: OK, dumping my current findings here, as I'm still not sure what the expected results should be. First of all, Lib/socket.py calls gethostbyaddr with a name. As in, gethostby _ADDR_ with a name. This works because Modules/socketmodule.c internally uses setipaddr() to resolve the name to an address. setipaddr() does this using a call to getaddrinfo() with hints.ai_family == AF_UNSPEC and no further flags. On my system (confirmed using the test program attached) this results in SIX entries, and this is the part that confused me. Due to virtualization I have a virtual bridge virbr0 configured with an internal IP address 192.168.122.1, as well as my LAN-connected bridge br0 with IP address 131.155.71.8. Both of these addresses are returned in the call to getaddrinfo() (each one 3 times), but NOT ALWAYS IN THE SAME ORDER. And this is the clue as to why python's socket.getfqdn() does not behave consistently. For 192.168.122.1 does not resolve to anything, hence it will return "pclin281". And 131.155.71.8 will backwards resolve to pclin281.win.tue.nl as the PTR record points to that entry. Now, again, I'm not entirely sure what to do here. I agree that this is not a simple bugfix. I also think that, apart from the weirdness of getaddrinfo() return order, socket.getfqdn() is doing it's documented job of returning /an/ FQDN for a given host. But in case of the guaranteed LOCAL canonical hostname, another function is warranted, imho. Does this make sense? For the record, output of a given run on my system: [TUE\shoop@pclin281] <~/tmp> ./test gai canon result 0: pclin281.campus.tue.nl 192.168.122.1 gai canon result 1: (null) 131.155.71.8 gai result 0: (null) 131.155.71.8 gai result 1: (null) 131.155.71.8 gai result 2: (null) 131.155.71.8 gai result 3: (null) 192.168.122.1 gai result 4: (null) 192.168.122.1 gai result 5: (null) 192.168.122.1 ghbn result 0 h_name: pclin281.campus.tue.nl ghbn result 0 h_alias: __NONE__ ghbn result 1 h_name: pclin281.campus.tue.nl ghbn result 1 h_alias: __NONE__ ghbn result 2 h_name: pclin281.campus.tue.nl ghbn result 2 h_alias: __NONE__ ghbn result 3 h_name: pclin281.campus.tue.nl ghbn result 3 h_alias: __NONE__ ghbn result 4 h_name: pclin281.campus.tue.nl ghbn result 4 h_alias: __NONE__ ghbn result 5 h_name: pclin281.campus.tue.nl ghbn result 5 h_alias: __NONE__ ghbn result 6 h_name: pclin281.campus.tue.nl ghbn result 6 h_alias: __NONE__ ghbn result 7 h_name: pclin281.campus.tue.nl ghbn result 7 h_alias: __NONE__ ghbn result 8 h_name: pclin281.campus.tue.nl ghbn result 8 h_alias: __NONE__ ghbn result 9 h_name: pclin281.campus.tue.nl ghbn result 9 h_alias: __NONE__ -- Added file: http://bugs.python.org/file29921/python5004-test.c ___ Python tracker <http://bugs.python.org/issue5004> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5004] socket.getfqdn() doesn't cope properly with purely DNS-based setups
Stijn Hoop added the comment: So after a good nights sleep: does it not make sense to use the canonical hostname iff the name argument is not present / empty? Otherwise, fall back to the documented steps? That way extra API is avoided, and I can't think of a case where you would rather have my weird results vs "the output of hostname -f". -- ___ Python tracker <http://bugs.python.org/issue5004> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2865] syntax error in fix_imports.py
New submission from Stijn van Drongelen <[EMAIL PROTECTED]>: In the current revision of 2to3 (63319), somebody forgot a comma at the end of line 27 of lib2to3/fix_imports.py, resulting in a syntax error. -- assignee: collinwinter components: 2to3 (2.x to 3.0 conversion tool) files: fix_imports-syntaxerror-63319.patch keywords: patch messages: 66867 nosy: Tinctorius, collinwinter severity: normal status: open title: syntax error in fix_imports.py type: compile error Added file: http://bugs.python.org/file10333/fix_imports-syntaxerror-63319.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2865> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2866] syntax error in fix_imports.py
New submission from Stijn van Drongelen <[EMAIL PROTECTED]>: In the current revision of 2to3 (63319), somebody forgot a comma at the end of line 27 of lib2to3/fix_imports.py, resulting in a syntax error. -- assignee: collinwinter components: 2to3 (2.x to 3.0 conversion tool) files: fix_imports-syntaxerror-63319.patch keywords: patch messages: 66869 nosy: Tinctorius, collinwinter severity: normal status: open title: syntax error in fix_imports.py type: compile error Added file: http://bugs.python.org/file10334/fix_imports-syntaxerror-63319.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue2866> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1610] test_socket.py fails
Stijn van Drongelen <[EMAIL PROTECTED]> added the comment: Reproducable with Python3.0rc1 on Debian lenny, only when nscd is running. Not a Python bug. -- nosy: +Tinctorius versions: +Python 3.0 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1610> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42906] python3 -m doctest test.py tests the stdlib "time" module instead
New submission from Stijn van Drongelen : I have two files, named time.py and time_.py, with the same contents: def foo(): """Returns 1. >>> foo() 1 """ return 1 When I run python3 -m doctest -v time_.py I see the expected behaviour: Trying: foo() Expecting: 1 ok 1 items had no tests: time_ 1 items passed all tests: 1 tests in time_.foo 1 tests in 2 items. 1 passed and 0 failed. Test passed. However, when I run python3 -m doctest -v time_.py it seems like doctest is testing the standard library 'time' module, while I expected it to have similar output as seen above: 30 items had no tests: time time.asctime (... skipped 24 lines for the bug report ...) time.time_ns time.tzset 0 tests in 30 items. 0 passed and 0 failed. Test passed. The same happens when I provide a path with slashes in it. -- components: Library (Lib) messages: 384913 nosy: stijnvandrongelen priority: normal severity: normal status: open title: python3 -m doctest test.py tests the stdlib "time" module instead type: behavior versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue42906> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42906] python3 -m doctest test.py tests the stdlib "time" module instead
Stijn van Drongelen added the comment: Reproducible in Python 3.8.7 and 3.9.1. -- versions: +Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue42906> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
