Re: [Python-Dev] converting the stdlib to str.format
> Antoine Pitrou wrote:
>>
>> Not to mention that e.g. "%r" % s is much simpler than "{0!r}".format(s)
>> (if I got the format spec right).
>
> repr(s) is even simpler :)
Yes, of course, but in the non-trivial case, e.g. "value=%r" % my_value,
it's much easier to use %-style formatting than playing with repr() and
the other string formatting facilities.
> Note that the old code in this case wouldn't display dict or tuple
> instances correctly. Avoiding that ambiguity is a major advantage of the
> new approach.
I know. For me the problem is not about ditching the % operator for an
intuitively-named method like format(). It's the format syntax which has
become much more complicated and error-prone without any clear advantage.
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytearray and array.array are not thread-safe
> Unfortunately, it's also a significant change at this point. I > personally won't have time to provide a patch, but I think a patch > is needed before the last beta. IOW, the issue should become a > release blocker. Agreed. Unfortunately I don't have much time to write a patch either. Perhaps in one or two weeks, but it would be better if someone beats me to it. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbots
Hi and thanks for your answers, > If you are able to offer something that's not on the list, that'd be > good. But any help at all is appreciated. > > I believe Windows has traditionally been under-represented in all > buildbot farms, and it's likely to stay that way... Well what I could provide is a 32-bit x86 Debian stable. Rather common I fear... > For Pybots, we're testing third-party apps and libraries against > changes made to Python core. If you're interested in a 3rd party > project, and you're willing to stay on top of that project's buildbot > status, and notify both the project leaders and the Python core devs > whenever you notice an ugly breakage Not interested /enough/ I think... by your description it sounds the job should really be done by a core developer of each of those packages (even if the machine is donated by someone else). What I could be interested in is to provide a buildbot for Python itself, but I don't know if that's needed right now. Especially for such a common platform as a x86 Debian. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: string formatting and i18n
> Yes. If you think I was arguing the opposite, then I failed to > communicate clearly and I apologize. Actually, I didn't interpret your message like that, but as I had already seen that proposal (to suppress string formatting), I thought it would be the right time to react ;) > For instance, take your I18N example. Not all languages have the > same word order, as you've observed. When there's more than one > parameter, Python's %-interpolation isn't enough in general; > you'd need something that can reorder the parameters. I don't > know whether this is worth complicating string formatting for, > but it's not obvious that it isn't. Well, I totally agree. I think it could be nice to both: - introduce positional formatting : "%1", "%2"... - make type specification optional, since Python can figure out the type by itself and use the right method; you would only specify the type when you want to have a different formatting (for example, for floats, you could use "%g2" instead of "%2" which would be equivalent to "%f2") Regards Antoine. -- « On dit que pétrir c'est modeler, Moi je dis que péter c'est démolir. » Stupéflip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - cooperative MT
Hi Martin, [snip] The "confusion" stems from the fact that two issues are mixed up in this discussion thread: - improving concurrency schemes to make it easier to write well-behaving applications with independent parallel flows - improving concurrency schemes to improve performance when there are several hardware threads available The respective solutions to these problems do not necessarily go hand in hand. > To implement that explicitly, you would need an > asynchronous version of all the functions that may block on > resources (e.g. file open, socket write, etc.), in order to be > able to insert a yield statement at that point, after the async > call, and there should be a way for the scheduler to check if the > resource is "ready" to be able to put your generator back in the > runnable queue. You can also use a helper thread and signal the scheduling loop when some action in the helper thread has finished. It is an elegant solution because it helps you keep a small generic scheduling loop instead of putting select()-like calls in it. (this is how I've implemented timers in my little cooperative multi-threading system, for example) > (A question comes to mind here: Twisted must be doing something > like this with their "deferred objects", no? I figure they would > need to do something like this too. I will have to check.) A Deferred object is just the abstraction of a callback - or, rather, two callbacks: one for success and one for failure. Twisted is architected around an event loop, which calls your code back when a registered event happens (for example when an operation is finished, or when some data arrives on the wire). Compared to generators, it is a different way of expressing cooperative multi-threading. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7 patch levels turning two digit
Le 23/06/2014 15:27, M.-A. Lemburg a écrit : Not sure what you mean. We've had binary wininst distributions for Windows for more than a decade, and egg and msi distributions for 8 years :-) But without access to the VS 2008 compiler that is needed to compile those extensions, It does seem to be available: http://www.microsoft.com/en-us/download/details.aspx?id=13276 What am I missing? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fix Unicode-disabled build of Python 2.7
Le 24/06/2014 07:04, Skip Montanaro a écrit : I can't see any reason to make a backwards-incompatible change to Python 2 to only support Unicode. You're bound to break somebody's setup. Apparently, that setup would already have been broken for years. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fix Unicode-disabled build of Python 2.7
Le 25/06/2014 19:28, Nick Coghlan a écrit : OK, *that* sounds like an excellent reason to keep the Unicode disabled builds functional, and make sure they stay that way with a buildbot: to help make sure we're not accidentally running afoul of the implicit interoperability between str and unicode when backporting fixes from Python 3. Helping to ensure correct handling of str values makes this capability something of benefit to *all* Python 2 users, not just those that turn off the Unicode support. It also makes it a potentially useful testing tool when assessing str/unicode handling in general. Hmmm... From my perspective, trying to enforce unicode-disabled builds will only lower the (already low) chance that I may want to write / backport bug fixes for 2.7. For the same reason, I agree with Victor that we should ditch the threading-disabled builds. It's too much of a hassle for no actual, practical benefit. People who want a threadless unicodeless Python can install Python 1.5.2 for all I care. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Network Security Backport Status
Le 01/07/2014 14:26, Alex Gaynor a écrit : I can do all the work of reviewing each commit, but I need some help from a mercurial expert to automate the cherry-picking/rebasing of every single commit. What do folks think? Does this approach make sense? Anyone willing to help with the mercurial scripting? I don't think this makes much sense; Mercurial won't be smarter than you are. I think you'd have a better chance of succeeding by backporting one feature at a time. IMO, you'd first want to backport the _SSLContext base class and SSLContext.wrap_socket(). The latter *will* require some manual coding to adapt to 2.7's different SSLSocket implementation, not just applying patch hunks around. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Memory BIO for _ssl
Hi, Le 05/07/2014 14:04, Geert Jansen a écrit : Since I need this for my Gruvi async framework, I want to volunteer to write a patch. It should be useful as well to Py3K's asyncio and other async frameworks. It would be good to get some feedback before I start on this. Thanks for volunteering! This would be a very welcome addition. Thoughts: I was thinking of the following approach: * Add a new type to _ssl: PySSLMemoryBIO * PySSLMemoryBIO has a public constructor, and at least the following methods: puts() puts_eof() and gets(). I aligned the terminology with the method names in OpenSSL. puts_eof() does a BIO_set_mem_eof_return(-1). Hmm... I haven't looked in detail, but at least I'd like those to be called read() and write() (and write_eof()), like most other I/O methods in Python. Or if we want to avoid confusion, add an explicit suffix (write_incoming?). * All accesses to the memory BIO as non-blocking. Sounds sensible indeed (otherwise what would they wait for?). * Update PySSLSocket to add support for SSL_set_bio(). The fact that the memory BIO is non-blocking makes it easier. None of the logic in and around check_socket_and_wait_for_timeout() for example needs to be changed. For the parts that deal with the socket directly, and that are in the code path for non-blocking IO, I think the preference would be i) try to change the code to use BIO methods that works for both sockets and memory BIOs, and ii) if not possible, special case it. That sounds good in the principle. I don't enough about memory BIOs to know whether you will have issues doing so :-) * At this point the PySSLSocket name is a bit of a misnomer as it does more than sockets. Probably not an issue. Agreed. * Add a method _wrap_bio(rbio, wbio, ...) to _SSLContext. * Expose the low-level methods via the "ssl" module. Creating an SSLSocket with a memory BIO would work something like this: context = SSLContext() rbio = ssl.MemoryBIO() wbio = ssl.MemoryBIO() sslsock = ssl.wrap_bio(rbio, wbio) The one thing I find confusing is the r(ead)bio / w(rite)bio terminology (because you actually read and write from both). Perhaps incoming and outgoing would be clearer. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot.python.org down again?
Le 07/07/2014 13:22, Guido van Rossum a écrit : It's a reference to Neil Stephenson's Anathem. According to Google, it doesn't look like he played the trombone, though. Regards Antoine. On Jul 7, 2014 8:55 AM, "Benjamin Peterson" mailto:[email protected]>> wrote: On Mon, Jul 7, 2014, at 08:44, Guido van Rossum wrote: > It would still be nice to know who "the appropriate persons" are. Too > much > of our infrastructure seems to be maintained by house elves or the ITA. :) Is ITA "International Trombone Association"? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] == on object tests identity in 3.x
Le 09/07/2014 00:21, Stephen J. Turnbull a écrit : Steven D'Aprano writes: > I don't think so. Floating point == represents *numeric* equality, There is no such thing as floating point == in Python. You can apply == to two floating point numbers, but == (at the language level) handles any two numbers, as well as pairs of things that aren't numbers in the Python language. This is becoming pointless hair-splitting. >>> float.__eq__(1.0, 2.0) False >>> float.__eq__(1.0, 2) False >>> float.__eq__(1.0, 1.0+0J) NotImplemented >>> float.__eq__(1, 2) Traceback (most recent call last): File "", line 1, in TypeError: descriptor '__eq__' requires a 'float' object but received a 'int' Please direct any further discussion of this to python-ideas. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cStringIO vs io.BytesIO
Hi, Le 16/07/2014 19:07, [email protected] a écrit : Attached is a (rough) patch implementing this, feel free to try it with hg tip. Thanks for your work. Please post any patch to http://bugs.python.org Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remaining decisions on PEP 471 -- os.scandir()
Hi, > Thanks Victor, Nick, Ethan, and others for continued discussion on the scandir PEP 471 (most recent thread starts at https://mail.python.org/pipermail/python-dev/2014-July/135377.html). Have you tried modifying importlib's _bootstrap.py to use scandir() instead of listdir() + stat()? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remaining decisions on PEP 471 -- os.scandir()
Le 20/07/2014 17:34, Ben Hoyt a écrit : Have you tried modifying importlib's _bootstrap.py to use scandir() instead of listdir() + stat()? No, I haven't -- I'm not familiar with that code. What does _bootstrap.py do -- does it do a lot of listdir calls and stat-ing of many files? Quite a bit, although that should be dampened in recent 3.x versions, thanks to the caching of directory contents. Even though there is tangible performance improvement from scandir(), it would be useful to find out if the API fits well. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP466] SSLSockets, and sockets, _socketobjects oh my!
Le 22/07/2014 17:03, Alex Gaynor a écrit : The question is: a) Should we backport weak referencing _socket.sockets (changing the structure of the module seems overly invasive, albeit completely backwards compatible)? b) Does anyone know why weak references are used in the first place? The commit message just alludes to fixing a leak with no reference to an issue. Because : - the SSLSocket has a strong reference to the ssl object (self._sslobj) - self._sslobj having a strong reference to the SSLSocket would mean both would only get destroyed on a GC collection I assume that's what "leak" means here :-) As for 2.x, I don't see why you couldn't just continue using a strong reference. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP466] SSLSockets, and sockets, _socketobjects oh my!
Le 22/07/2014 17:44, Nick Coghlan a écrit : > > As for 2.x, I don't see why you couldn't just continue using a strong reference. As Antoine says, if the cycle already exists in Python 2 (and it sounds like it does), we can just skip backporting the weak reference change. No, IIRC there shouldn't be a cycle. It's just complicated in a different way than 3.x :-) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 471 "scandir" accepted
Le 21/07/2014 18:26, Victor Stinner a écrit : I'm happy because the final API is very close to os.path functions and pathlib.Path methods. Python stays consistent, which is a great power of this language! By the way, http://bugs.python.org/issue19767 could benefit too. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP466] SSLSockets, and sockets, _socketobjects oh my!
Le 23/07/2014 15:36, Alex Gaynor a écrit : That said, I've hit another issue, with SNI callbacks. The first argument to an SNI callback is the socket. The callback is set up by some C code, which right now has access to only the _socket.socket object, not the ssl.SSLSocket object, which is what the public API needs there. Possible solutions are: * Pass the SSLObject *in addition* to the _socket.socket object to the C code. This generates some additional divergence from the Python3 code, but is probably basically straightforward. You mean for use with SSL_set_app_data? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #22003: When initialized from a bytes object, io.BytesIO() now
Le 30/07/2014 02:11, Serhiy Storchaka a écrit : 30.07.14 06:59, Serhiy Storchaka написав(ла): 30.07.14 02:45, antoine.pitrou написав(ла): http://hg.python.org/cpython/rev/79a5fbe2c78f changeset: 91935:79a5fbe2c78f parent: 91933:fbd104359ef8 user:Antoine Pitrou date:Tue Jul 29 19:41:11 2014 -0400 summary: Issue #22003: When initialized from a bytes object, io.BytesIO() now defers making a copy until it is mutated, improving performance and memory use on some use cases. Patch by David Wilson. Did you compare this with issue #15381 [1]? Not really, but David's patch is simple enough and does a good job of accelerating the read-only BytesIO case. $ ./python -m timeit -s 'import i' 'i.readlines()' Before patch: 10 loops, best of 3: 46.9 msec per loop After issue22003 patch: 10 loops, best of 3: 36.4 msec per loop After issue15381 patch: 10 loops, best of 3: 27.6 msec per loop I'm surprised your patch does better here. Any idea why? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #22003: When initialized from a bytes object, io.BytesIO() now
Le 30/07/2014 15:48, Serhiy Storchaka a écrit : Ignoring tests and comments my patch adds/removes/modifies about 200 lines, and David's patch -- about 150 lines of code. But it's __sizeof__ looks not correct, correcting it requires changing about 50 lines. In sum the complexity of both patches is about equal. I meant that David's approach is conceptually simpler, which makes it easier to review. Regardless, there is no exclusive-OR here: if you can improve over the current version, there's no reason not to consider it/ I didn't look at David's patch too close yet. But my patch includes optimization for end-of-line scanning. Ahah, unrelated stuff :-) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Surely "nullable" is a reasonable name?
Le 04/08/2014 03:35, Stephen Hansen a écrit : Before you say "the term 'nullable' will confuse end users", let me remind you: this is not user-facing. This is a parameter for an Argument Clinic converter, and will only ever be seen by CPython core developers. A group which I hope is not so easily confused Yet, my lurking observation of argument clinic is it is all about clearly defining the C-side of how things are done in Python API's. It may not confuse 'end users', but it may confuse possible contributors, and simply add a lack of clarity to the situation. That's a rather good point, and I agree with Stephen here. Even core contributors can deserve clarity and the occasional non-confusing notation :-) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Surely "nullable" is a reasonable name?
Le 04/08/2014 13:36, Alexander Belopolsky a écrit : On Mon, Aug 4, 2014 at 12:57 PM, Ethan Furman mailto:[email protected]>> wrote: 'allow_none' is definitely clearer. I disagree. Unlike "nullable", "allow_none" does not tell me what happens on the C side when I pass in None. If the receiving type is PyObject*, either NULL or Py_None is a valid choice. But here the receiving type can be an int. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Surely "nullable" is a reasonable name?
Le 04/08/2014 14:18, Larry Hastings a écrit : On 08/05/2014 03:53 AM, Antoine Pitrou wrote: Le 04/08/2014 13:36, Alexander Belopolsky a écrit : If the receiving type is PyObject*, either NULL or Py_None is a valid choice. But here the receiving type can be an int. Just to be precise: in the case where the receiving type *would* have been an int, and "nullable=True", the receiving type is actually a structure containing an int and a "you got a None" flag. I can't stick a magic value in the int and say "that represents you getting a None" because any integer value may be valid. Also, I'm pretty sure there are places in builtin argument parsing that accept either NULL or Py_None, and I *think* maybe in one or two of them they actually mean different things. What fun! For small values of "fun", Is -909 too large a value to be fun? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pathlib handling of trailing slash (Issue #21039)
Le 06/08/2014 18:36, Isaac Schwabacher a écrit :
If a symbolic link is encountered during pathname resolution, the
behavior shall depend on whether the pathname component is at the
end of the pathname and on the function being performed. If all of
the following are true, then pathname resolution is complete:
1. This is the last pathname component of the pathname. 2. The
pathname has no trailing . 3. The function is required to
act on the symbolic link itself, or certain arguments direct that
the function act on the symbolic link itself.
In all other cases, the system shall prefix the remaining pathname,
if any, with the contents of the symbolic link. [...]
So the only case where this would make a difference is when calling a
"function acting on the symbolic link itself" (such as lstat() or
unlink()) on a path with a trailing slash:
>>> os.lstat('foo')
os.stat_result(st_mode=41471, st_ino=1981954, st_dev=2050, st_nlink=1,
st_uid=1000, st_gid=1000, st_size=4, st_atime=1407370025,
st_mtime=1407370025, st_ctime=1407370025)
>>> os.lstat('foo/')
os.stat_result(st_mode=17407, st_ino=917505, st_dev=2050, st_nlink=7,
st_uid=0, st_gid=0, st_size=4096, st_atime=1407367916,
st_mtime=1407369857, st_ctime=1407369857)
>>> pathlib.Path('foo').lstat()
os.stat_result(st_mode=41471, st_ino=1981954, st_dev=2050, st_nlink=1,
st_uid=1000, st_gid=1000, st_size=4, st_atime=1407370037,
st_mtime=1407370025, st_ctime=1407370025)
>>> pathlib.Path('foo/').lstat()
os.stat_result(st_mode=41471, st_ino=1981954, st_dev=2050, st_nlink=1,
st_uid=1000, st_gid=1000, st_size=4, st_atime=1407370037,
st_mtime=1407370025, st_ctime=1407370025)
But you can also call resolve() explicitly if you want to act on the
link target rather than the link itself:
>>> pathlib.Path('foo/').resolve().lstat()
os.stat_result(st_mode=17407, st_ino=917505, st_dev=2050, st_nlink=7,
st_uid=0, st_gid=0, st_size=4096, st_atime=1407367916,
st_mtime=1407369857, st_ctime=1407369857)
Am I overlooking other cases?
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pathlib handling of trailing slash (Issue #21039)
Le 06/08/2014 20:50, Alexander Belopolsky a écrit : On Wed, Aug 6, 2014 at 8:11 PM, Antoine Pitrou mailto:[email protected]>> wrote: Am I overlooking other cases? There are many interfaces where trailing slash is significant. For example, rsync uses trailing slash on the target directory to avoid creating an additional directory level at the destination. Loosing it when passing path strings through pathlib.Path() may be a source of bugs. pathlib is generally concerned with filesystem operations written in Python, not arbitrary third-party tools. Also it is probably easy to append the trailing slash in your command-line invocation, if so desired. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pathlib handling of trailing slash (Issue #21039)
Le 06/08/2014 22:12, Ben Finney a écrit :
You seem to be saying that ‘pathlib’ is not intended to be helpful for
constructing a shell command.
pathlib lets you do operations on paths. It also gives you a string
representation of the path that's expected to designate that path when
talking to operating system APIs. It doesn't give you the possibility to
store other semantic variations ("whether a new directory level must be
created"); that's up to you to add those.
(similarly, it doesn't have separate classes to represent "a file", "a
directory", "a non-existing file", etc.)
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sum(...) limitation
Le 09/08/2014 01:08, Steven D'Aprano a écrit : On Fri, Aug 08, 2014 at 10:20:37PM -0400, Alexander Belopolsky wrote: On Fri, Aug 8, 2014 at 8:56 PM, Ethan Furman wrote: I don't use sum at all, or at least very rarely, and it still irritates me. You are not alone. When I see sum([a, b, c]), I think it is a + b + c, but in Python it is 0 + a + b + c. If we had a "join" operator for strings that is different form + - then sure, I would not try to use sum to join strings, but we don't. I've long believed that + is the wrong operator for concatenating strings, and that & makes a much better operator. We wouldn't be having these interminable arguments about using sum() to concatenate strings (and lists, and tuples) if the & operator was used for concatenation and + was only used for numeric addition. Come on. These arguments are interminable because many people (including you) love feeding interminable arguments. No need to blame Python for that. And for that matter, this interminable discussion should probably have taken place on python-ideas or even python-list. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.walk() is going to be *fast* with scandir
Le 09/08/2014 12:43, Ben Hoyt a écrit : Just thought I'd share some of my excitement about how fast the all-C version [1] of os.scandir() is turning out to be. Below are the results of my scandir / walk benchmark run with three different versions. I'm using an SSD, which seems to make it especially faster than listdir / walk. Note that benchmark results can vary a lot, depending on operating system, file system, hard drive type, and the OS's caching state. Anyway, os.walk() can be FIFTY times as fast using os.scandir(). Very nice results, thank you :-) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 467: Minor API improvements for bytes & bytearray
Le 17/08/2014 13:07, Raymond Hettinger a écrit : FWIW, I've been teaching Python full time for three years. I cover the use of bytearray(n) in my classes and not a single person out of 3000+ engineers have had a problem with it. This is less about bytearray() than bytes(), IMO. bytearray() is sufficiently specialized that only experienced people will encounter it. And while preallocating a bytearray of a certain size makes sense, it's completely pointless for a bytes object. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 467: Minor API improvements for bytes & bytearray
Le 16/08/2014 01:17, Nick Coghlan a écrit : * Deprecate passing single integer values to ``bytes`` and ``bytearray`` I'm neutral. Ideally we wouldn't have done that mistake at the beginning. * Add ``bytes.zeros`` and ``bytearray.zeros`` alternative constructors * Add ``bytes.byte`` and ``bytearray.byte`` alternative constructors * Add ``bytes.iterbytes``, ``bytearray.iterbytes`` and ``memoryview.iterbytes`` alternative iterators +0.5. "iterbytes" isn't really great as a name. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 467: Minor API improvements for bytes & bytearray
Le 17/08/2014 19:41, Raymond Hettinger a écrit : The APIs have been around since 2.6 and AFAICT there have been zero demonstrated need for a special case for a single byte. We already have a perfectly good spelling: NUL = bytes([0]) That is actually a very cumbersome spelling. Why should I first create a one-element list in order to create a one-byte bytes object? The Zen tells us we really don't need a second way to do it (actually a third since you can also write b'\x00') and it suggests that this special case isn't special enough. b'\x00' is obviously the right way to do it in this case, but we're concerned about the non-constant case. The reason to instantiate bytes from non-constant integer comes from the unfortunate indexing and iteration behaviour of bytes objects. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: PEP 467: Minor API improvements for bytes & bytearray
Le 17/08/2014 20:08, Nick Coghlan a écrit : On 18 Aug 2014 09:57, "Barry Warsaw" mailto:[email protected]>> wrote: > > On Aug 18, 2014, at 09:12 AM, Nick Coghlan wrote: > > >I'm talking more generally - do you *really* want to be explaining that > >"bytes" behaves like a tuple of integers, while "bytes.bytes" behaves like > >a tuple of bytes? > > I would explain it differently though, using concrete examples. > > data = bytes(...) > for i in data: # iterate over data as integers > for i in data.bytes: # iterate over data as bytes > > But whatever. I just wish there was something better than iterbytes. There's actually another aspect to your idea, independent of the naming: exposing a view rather than just an iterator. So that view would actually be the bytes object done right? Funny :-) Will it have lazy slicing? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 4000 to explicitly declare we won't be doing a Py3k style compatibility break again?
Le 18/08/2014 13:22, Mark Dickinson a écrit : [Moderately off-topic] On Sun, Aug 17, 2014 at 3:39 AM, Steven D'Aprano mailto:[email protected]>> wrote: I used to refer to Python 4000 as the hypothetical compatibility break version. Now I refer to Python 5000. I personally think it should be Python 500, or Py5M. When we come to create the mercurial branch, that should of course, following tradition, be called p5ym. I would suggest "NaV", for "not-a-version". It would compare greater than all other version numbers (in the spirit of Numpy's "not-a-time", slightly tweaked). Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytes path support
Le 19/08/2014 13:43, Ben Hoyt a écrit : The official policy is that we want them [support for bytes paths in stdlib functions] to go away, but reality so far has not budged. We will continue to hold our breath though. :-) Does that mean that new APIs should explicitly not support bytes? I'm thinking of os.scandir() (PEP 471), which I'm implementing at the moment. I was originally going to make it support bytes so it was compatible with listdir, but maybe that's a bad idea. Bytes paths are essentially broken on Windows. Bytes paths are "essential" on Unix, though, so I don't think we should create new low-level APIs that don't support bytes. Fair enough. I don't quite understand, though -- why is the "official policy" to kill something that's "essential" on *nix? PEP 383 should actually work on Unix quite well, AFAIR. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytes path support
Le 20/08/2014 07:08, Nick Coghlan a écrit : It's not just the JVM that says text and binary APIs should be separate - it's every widely used operating system services layer except POSIX. The POSIX way works well *if* everyone reliably encodes things as UTF-8 or always uses encoding detection, but its failure mode is unfortunately silent data corruption. That said, there's a lot of Python software that is POSIX specific, where bytes paths would be the least of the barriers to porting to Windows or Jython. I'm personally +1 on consistently allowing binary paths in lower level APIs, but disallowing them in higher level explicitly cross platform abstractions like pathlib. I fully agree with Nick's position here. To elaborate specifically about pathlib, it doesn't handle bytes paths but allows you to generate them if desired: https://docs.python.org/3/library/pathlib.html#operators Adding full bytes support to pathlib would have added a lot of complication and fragility in the implementation *and* in the API (is it allowed to combine str and bytes paths? should they have separate classes?), for arguably little benefit. I think if you want low-level features (such as unconverted bytes paths under POSIX), it is reasonable to point you to low-level APIs. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytes path support
Le 21/08/2014 00:52, Cameron Simpson a écrit : The "bytes in some arbitrary encoding where at least the slash character (and maybe a couple others) is ascii compatible" notion is completely bogus. There's only one special byte, the slash (code 47). There's no OS-level need that it or anything else be ASCII compatible. Of course there is. Try to split an UTF-16-encoded file path on the byte 47 and you'll get a lot of garbage. So, yes, POSIX implicitly mandates an ASCII-compatible encoding for file paths. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytes path support
Le 21/08/2014 18:27, Cameron Simpson a écrit : As remarked, codes 0 (NUL) and 47 (ASCII slash code) _are_ special to UNIX filename bytes strings. So you admit that POSIX mandates that file paths are expressed in an ASCII-compatible encoding after all? Good. I've nothing to add to your rant. Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytes path related questions for Guido
Le 24/08/2014 09:04, Nick Coghlan a écrit : On 24 August 2014 14:44, Nick Coghlan wrote: 2. Should we add some additional helpers to the string module for dealing with surrogate escaped bytes and other techniques for smuggling arbitrary binary data as text? My proposal [3] is to add: * string.escaped_surrogates (constant with the 128 escaped code points) * string.clean(s): replaces surrogates with '\ufffd' or another specified code point * string.redecode(s, encoding): encodes a string back to bytes and then decodes it again using the specified encoding (the old encoding defaults to 'latin-1' to match the assumptions in WSGI) Serhiy & Ezio convinced me to scale this one back to a proposal for "codecs.clean_surrogate_escapes(s)", which replaces surrogates that may be produced by surrogateescape (that's what string.clean() above was supposed to be, but my description was not correct, and the name was too vague for that error to be obvious to the reader) "clean" conveys the wrong meaning. It should use a scary word such as "trap". "Cleaning" surrogates is unlikely to be the right procedure when dealing with surrogates produced by undecodable byte sequences. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Fri, 29 Aug 2014 17:11:35 -0400 Donald Stufft wrote: > > Another problem with this is that I don’t think it’s actually > possible to do. Python itself isn’t validating the TLS certificates, > OpenSSL is doing that. To my knowledge OpenSSL doesn’t > have a way to say “please validate these certificates and if > they don’t validate go ahead and keep going and just let me > get a warning from it”. Actually, there may be a solution. In client mode, OpenSSL always verifies the server cert chain and stores the verification result in the SSL structure. It will then only report an error if the verify mode is not SSL_VERIFY_NONE. (see ssl3_get_server_certificate() in s3_clnt.c) The verification result should then be readable using SSL_get_verify_result(), even with SSL_VERIFY_NONE. (note this is only from reading the source code and needs verifying) Then we could have the following transition phase: - define a new CERT_WARN value for SSLContext.verify_mode - use that value as the default in the HTTP stack (people who want the old silent default will have to set verify_mode explicitly to VERIFY_NONE) - with CERT_WARN, SSL_VERIFY_NONE is passed to OpenSSL and Python manually calls SSL_get_verify_result() after a handshake; if there was a verification error, a warning is printed out And in the following version we switch the HTTP default to CERT_REQUIRED. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Fri, 29 Aug 2014 17:42:34 -0400 "R. David Murray" wrote: > > Especially if you want an accelerated change, there must be a way to > *easily* get back to the previous behavior, or we are going to catch a > lot of flack. There may be only 7% of public certs that are problematic, > but I'd be willing to bet you that there are more not-really-public ones > that are critical to day to day operations *somewhere* :) Actually, by construction, there are certs which will always fail verification, for example because they are embedded in telco equipments which don't have a predefined hostname or IP address. (I have encountered some of those) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Fri, 29 Aug 2014 18:08:19 -0400 Donald Stufft wrote: > > > > Are you sure that's possible ? Python doesn't load the > > openssl.cnf file and the SSL_CERT_FILE, SSL_CERT_DIR env > > vars only work for the openssl command line binary, AFAIK. > > I’m not 100% sure on that. I know they are not limited to the command > line binary as ruby uses those environment variables in the way I > described above. SSL_CERT_DIR and SSL_CERT_FILE are used, if set, when SSLContext.load_verify_locations() is called. Actually, come to think of it, this allows us to write a better test for that method. Patch welcome! Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Sat, 30 Aug 2014 12:19:11 +0200 "M.-A. Lemburg" wrote: > > To add to the PEP: > > > > * Emit a warning in 3.4.next for cases that would raise a Exception in 3.5 > > * Clearly state that the existing OpenSSL environment variables will be > > respected for setting the trust root > > I'd also suggest to compile Python with OPENSSL_LOAD_CONF, since that > causes OpenSSL to read the global openssl.cnf file for additional > configuration. Python links against OpenSSL as a shared library, not statically. It's unlikely that setting a compile constant inside Python would affect OpenSSL at all. > > Discussion points: > > > > * Disabling verification entirely externally to the program, through a CLI > > flag > > or environment variable. I'm pretty down on this idea, the problem you > > hit is > > that it's a pretty blunt instrument to swing, and it's almost impossible > > to > > imagine it not hitting things it shouldn't; it's far too likely to be > > used in > > applications that make two sets of outbound connections: 1) to some > > internal > > service which you want to disable verification on, and 2) some external > > service which needs strong validation. A global flag causes the latter to > > fail silently when subjected to a MITM attack, and that's exactly what > > we're > > trying to avoid. It also makes things much harder for library authors: I > > write an API client for some API, and make TLS connections to it. I want > > those to be verified by default. I can't even rely on the httplib > > defaults, > > because someone might disable them from the outside. > > The reasoning here is the same as for hash randomization. There > are cases where you want to test your application using self-signed > certificates which don't validate against the system CA root list. That use case should be served with the SSL_CERT_DIR and SSL_CERT_FILE env vars (or, better, by specific settings *inside* the application). I'm against multiplying environment variables, as it makes it more difficult to assess the actual security of a setting. The danger of an ill-secure setting is much more severe than with hash randomization. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Sat, 30 Aug 2014 12:46:47 +0200 "M.-A. Lemburg" wrote: > The change is to the OpenSSL API, not the OpenSSL lib. By setting > the variable you enable a few special calls to the config loader > functions in OpenSSL when calling the initializer it: > > https://www.openssl.org/docs/crypto/OPENSSL_config.html Ah, ok. Do you have experience with openssl.cnf? Apparently, it is meant for offline tools such as certificate generation, I am not sure how it could impact certification validation. > > That use case should be served with the SSL_CERT_DIR and SSL_CERT_FILE > > env vars (or, better, by specific settings *inside* the application). > > > > I'm against multiplying environment variables, as it makes it more > > difficult to assess the actual security of a setting. The danger of an > > ill-secure setting is much more severe than with hash randomization. > > You have a point there. So how about just a python run-time switch > and no env var ? Well, why not, but does it have a value over letting the code properly configure their SSLContext? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Sun, 31 Aug 2014 09:26:30 +1000 Nick Coghlan wrote: > >> > >> * configuration: > >> > >> It would be good to be able to switch this on or off > >> without having to change the code, e.g. via a command > >> line switch and environment variable; perhaps even > >> controlling whether or not to raise an exception or > >> warning. > >> > >> * choice of trusted certificate: > >> > >> Instead of hard wiring using the system CA roots into > >> Python it would be good to just make this default and > >> permit the user to point Python to a different set of > >> CA roots. > >> > >> This would enable using self signed certs more easily. > >> Since these are often used for tests, demos and education, > >> I think it's important to allow having more control of > >> the trusted certs. > > > > > > +1 for PEP with above changes. > > Ditto from me. > > In relation to changing the Python CLI API to offer some of the wget/curl > style command line options, I like the idea of providing recipes in the > docs for implementing them at the application layer, but postponing making > the *default* behaviour configurable that way. I'm against any additional environment variables and command-line options. It will only complicate and obscure the security parameters of certificate validation. The existing knobs have already been mentioned in this thread, I won't mention them here again. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
Le 31/08/2014 19:03, Paul Moore a écrit : On 31 August 2014 17:27, Christian Heimes wrote: It's very simple to trust a self-signed certificate: just download it and stuff it into the trust store. "Stuff it into the trust store" is the hard bit, though. I have honestly no idea how to do that. You certainly shouldn't do so. If an application has special needs that require trusting a self-signed certificate, then it should expose a configuration setting to let users specify the cert's location. Stuffing self-signed certs into the system trust store is really a measure of last resort. There's another case which isn't solved by this, though, which is when a cert is invalid. The common situation being that it has expired (renewing certs is a PITA and therefore expired certs are more common than it sounds they should be). In this case, there is no way to whitelist it: you have to disable certificate checking altogether. This can be exposed by the application as configuration option if necessary, as well. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
Le 31/08/2014 20:28, Paul Moore a écrit : I can't see how that would be something the application would know. For example, pip allows me to specify an "alternate cert bundle" but not a single additional cert. So IIUC, I can't use my local index that serves https using a self-signed cert. I'd find it hard to argue that it's pip's responsibility to think of that use case - pretty much any program that interacts with a web service *might* need to interact with a self-signed dummy version, if only under test conditions. Well, it's certainly pip's responsibility more than Python's. What would Python do? Provide a setting that would blindly add a cert for all uses of httplib? pip knows about the use cases here, Python doesn't. (perhaps you want to serve your local index using http, though) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
Le 31/08/2014 21:12, Paul Moore a écrit : On 31 August 2014 19:37, Antoine Pitrou wrote: Well, it's certainly pip's responsibility more than Python's. What would Python do? Provide a setting that would blindly add a cert for all uses of httplib? That's more or less my point, pip doesn't have that much better idea than Python. I was talking about putting the cert in my local cert store, so that *I* can decide, and applications don't need to take special care to allow me to handle this case. What do you call your local cert store? If you mean the system cert store, then that will affect all users. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
Le 31/08/2014 23:41, Nick Coghlan a écrit : Right, this is why I came to the conclusion we need to follow the browser vendors lead here and support a per-user Python specific supplementary certificate cache before we can start validating certs by default at the *Python* level. There are still too many failure modes for cert management on private networks for us to safely ignore the use case of needing to force connections to services with invalid certs. We are not ignoring that use case. The proper solution is simply to disable cert validation in the application code (or, for more sophisticated needs, provide an application configuration setting for cert validation). > In the meantime, we can update the security considerations for the ssl > module to make it clearer that the defaults are set up for trusted networks > and that using it safely on the public internet may mean you're better off > with a third party library like requests or Twisted. No, you simply have to select the proper validation settings. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
On Mon, 01 Sep 2014 01:15:12 +0300 Marko Rauhamaa wrote: > > If a signal is received when read() or write() has completed its task > partially (> 0 bytes), no EINTR is returned but the partial count. > Obviously, Python should take that possibility into account so that > raising an exception in the signal handler (as mandated by the PEP) > doesn't cause the partial result to be lost on os.read() or os.write(). If the signal handler is called, the exception *will* be raised. There's no guarantee at which point in the Python code it will be raised (it's implementation-dependent), but it's near impossible to protect regular Python code against such asynchronous exceptions. Which is why you should switch to a wakeup fd scheme as mentioned by Victor, if you want to rely on signals at all. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 477: selected ensurepip backports for Python 2.7
On Mon, 1 Sep 2014 08:00:14 +1000 Nick Coghlan wrote: > > That part of the proposal proved to be controversial, so we dropped it from > the original PEP in order to focus on meeting the Python 3.4 specific > release deadlines. This also had the benefit of working out the kinks in > the bootstrapping processing as part of the Python 3.4 release cycle. > > However, we still think we should start providing pip by default to Python > 2.7 users as well, at least as part of the Windows and Mac OS X installers. I don't agree with this. pip is simply not part of the 2.7 feature set. If you add pip to a bugfix version, then you have bugfix versions which are more featureful than others, which makes things more complicated to explain. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
Le 01/09/2014 10:09, Nick Coghlan a écrit : > On 1 September 2014 17:13, Christian Heimes wrote: >> On 01.09.2014 08:44, Nick Coghlan wrote: >>> Yes, it would have exactly the same security failure modes as >>> sitecustomize, except it would only fire if the application >>> imported the ssl module. >>> >>> The "-S" and "-I" switches would need to disable the implied >>> "sslcustomize", just as they disable "import site". >> >> A malicious package can already play havoc with your installation with >> a custom ssl module. If somebody is able to sneak in a ssl.py then you >> are screwed anyway. sslcustomize is not going to make the situation worse. > > That's not quite true - we're fairly careful about putting the > standard library before userspace directories, so aside from the > "current directory" problem, shadowing "ssl" itself can be tricky to > arrange. Not sure why. Just put another module named "ssl" in sys.modules directly. You can also monkeypatch the genuine ssl module. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
On Mon, 01 Sep 2014 19:15:33 +1200 Greg Ewing wrote: > Victor Stinner wrote: > > > > Le 1 sept. 2014 00:17, "Marko Rauhamaa" > <mailto:[email protected]>> a écrit : > > > If a signal is received when read() or write() has completed its task > > > partially (> 0 bytes), no EINTR is returned but the partial count. > > > Obviously, Python should take that possibility into account so that > > > raising an exception in the signal handler (as mandated by the PEP) > > > doesn't cause the partial result to be lost on os.read() or os.write(). > > > > This case is unrelated to the PEP, the PEP only changes the behaviour > > when a syscall fails with EINTR. > > I think there's a problem here, though. As thing stand, a > signal handler that doesn't raise an exception can set a flag, > and code after the read() can test it. > > Under the proposed scheme, the signal handler has to > be made to raise an exception so that the read will be > broken out of in the EINTR case. The PEP already addresses this remark: ""Applications relying on the fact that system calls are interrupted with ``InterruptedError`` will hang. The authors of this PEP don't think that such application exist. If such applications exist, they are not portable and are subject to race conditions (deadlock if the signal comes before the system call).""" Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
Hi, I'm +1 on the whole PEP. > Writing a signal handler is difficult, only "async-signal safe" > functions can be called. You mean a C signal handler? Python signal handlers are not restricted. > Some signals are not interesting and should not interrupt the the > application. There are two options to only interrupt an application > on some signals: > > * Raise an exception in the signal handler, like ``KeyboardInterrupt`` for > ``SIGINT`` > * Use a I/O multiplexing function like ``select()`` with the Python > signal "wakeup" file descriptor: see the function > ``signal.set_wakeup_fd()``. This section looks a bit incomplete. Some calls such as os.read() or os.write() will (should) return a partial result when interrupted and they already handled >0 bytes. Perhaps other functions have a similar behaviour? > On Unix, the ``asyncio`` module uses the wakeup file descriptor to > wake up its event loop. How about Windows? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Mon, 1 Sep 2014 23:24:39 +1000 Chris Angelico wrote: > On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou wrote: > > Not sure why. Just put another module named "ssl" in sys.modules directly. > > You can also monkeypatch the genuine ssl module. > > That has to be done inside the same process. But imagine this > scenario: You have a program that gets invoked as root (or some other > user than yourself), and you're trying to fiddle with what it sees. > You don't have root access, but you can manipulate the file system, to > the extent that your userid has access. What can you do to affect this > other program? If you're root you shouldn't run untrusted code. See https://docs.python.org/3/using/cmdline.html#cmdoption-I Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Mon, 1 Sep 2014 23:42:10 +1000 Chris Angelico wrote: > On Mon, Sep 1, 2014 at 11:34 PM, Antoine Pitrou wrote: > > On Mon, 1 Sep 2014 23:24:39 +1000 > > Chris Angelico wrote: > >> On Mon, Sep 1, 2014 at 10:41 PM, Antoine Pitrou wrote: > >> > Not sure why. Just put another module named "ssl" in sys.modules > >> > directly. > >> > You can also monkeypatch the genuine ssl module. > >> > >> That has to be done inside the same process. But imagine this > >> scenario: You have a program that gets invoked as root (or some other > >> user than yourself), and you're trying to fiddle with what it sees. > >> You don't have root access, but you can manipulate the file system, to > >> the extent that your userid has access. What can you do to affect this > >> other program? > > > > If you're root you shouldn't run untrusted code. See > > https://docs.python.org/3/using/cmdline.html#cmdoption-I > > Right, which is why sslcustomize has to be controlled by that, but the > possibility of patching (or monkeypatching) ssl.py isn't as big a > deal. To be frank I don't understand what you're arguing about. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Tue, 2 Sep 2014 00:53:11 +1000 Nick Coghlan wrote: > On 2 Sep 2014 00:08, "Antoine Pitrou" wrote: > > > > On Mon, 1 Sep 2014 23:42:10 +1000 > > Chris Angelico wrote: > > > >> > > > >> That has to be done inside the same process. But imagine this > > > >> scenario: You have a program that gets invoked as root (or some other > > > >> user than yourself), and you're trying to fiddle with what it sees. > > > >> You don't have root access, but you can manipulate the file system, > to > > > >> the extent that your userid has access. What can you do to affect > this > > > >> other program? > > > > > > > > If you're root you shouldn't run untrusted code. See > > > > https://docs.python.org/3/using/cmdline.html#cmdoption-I > > > > > > Right, which is why sslcustomize has to be controlled by that, but the > > > possibility of patching (or monkeypatching) ssl.py isn't as big a > > > deal. > > > > To be frank I don't understand what you're arguing about. > > When I said "shadowing ssl can be tricky to arrange", Chris correctly > interpreted it as referring to the filesystem based privilege escalation > scenario that isolated mode handles, not to normal in-process > monkeypatching or module injection. There's no actual difference. You can have a sitecustomize.py that does the monkeypatching or the shadowing. There doesn't seem to be anything "tricky" about that. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
On Mon, 01 Sep 2014 11:47:07 -0400 "R. David Murray" wrote: > > > > The two requirements are: > > > > * Allow the application to react to signals immediately in the main > >flow. > > You don't want to be writing your code in Python then. In Python > you *never* get to react immediately to signals. The interpreter > sets a flag and calls the python signal handler later. Yes, the > call is ASAP, but ASAP is *not* "immediately". Especially if the signal is delivered to another thread (which is OS-dependent), and the main thread is blocked in *another* system call ;-) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Daily reference leaks (640c575ab3e1): sum=151940
Is anyone working on those? On Mon, 01 Sep 2014 10:41:45 +0200 [email protected] wrote: > results for 640c575ab3e1 on branch "default" > > > test_codecs leaked [5825, 5825, 5825] references, sum=17475 > test_codecs leaked [1172, 1174, 1174] memory blocks, sum=3520 > test_collections leaked [0, 2, 0] references, sum=2 > test_distutils leaked [37735, 37735, 37735] references, sum=113205 > test_distutils leaked [5909, 5911, 5911] memory blocks, sum=17731 > test_functools leaked [0, 0, 3] memory blocks, sum=3 > test_site leaked [0, 0, 2] references, sum=2 > test_site leaked [0, 0, 2] memory blocks, sum=2 > > > Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', > '3:3:/home/antoine/cpython/refleaks/reflogHPiXJp', '-x'] ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR
On Mon, 1 Sep 2014 21:17:33 + (UTC) Matthew Woodcraft wrote: > > > If such applications exist, they are not portable and are subject to > > race conditions (deadlock if the signal comes before the system call). > > The program is certainly not portable (which is not any kind of a > problem), and as pselect is unavailable there is indeed the usual > theoretical race (which has not been a problem in practice in the ten > years it's been running). > > (The program handles SIGTERM so that it can do a bit of cleanup before > exiting, and it uses the signal-handler-sets-a-flag technique. The call > that might be interrupted is sleep(), so the program doesn't strictly > _rely_ on the existing behaviour; it would just become very slow to > exit.) So, if it's just for process exit, just let the signal handler raise an exception and catch the exception at the top-level. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Tue, 2 Sep 2014 14:00:02 -0700 Glyph Lefkowitz wrote: > > I would strongly recommend against such a mechanism. > > For what it's worth, Twisted simply unconditionally started verifying > certificates in 14.0 with no "disable" switch, and (to my knowledge) > literally no users have complained. And how many people are using Twisted as an HTTPS client? (compared to e.g. Python's httplib, and all the third-party libraries building on it?) > Furthermore, "disable verification" is a nonsensical thing to do with TLS. It's not. For example, if you have an expired cert, all you can do AFAIK is to disable verification. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Tue, 2 Sep 2014 22:16:18 + (UTC) Alex Gaynor wrote: > > > Furthermore, "disable verification" is a nonsensical thing to do with TLS. > > > > It's not. For example, if you have an expired cert, all you can do > > AFAIK is to disable verification. > > It really is a nonsensical operation, accepting any random TLS certificate > without pinning or use of a certificate authorities makes a valid connection > completely indistinguishable from a MITM attack. It's still distinguishable from a passively-eavesdroppable clear-text connection, though. > This whole scenario seems to be predicated on a siutation where: You have a > peer whose certificate you can't change, and you have a piece of code you > can't > change, and you're going to upgrade your Python installation, and you want to > talk to this peer, and you need to use an encrypted channel, but don't really > care if it's being MITM'd. It doesn't seem to me that this is reasonably > Python's responsibility to deal with the fact that you have no ability to > upgrade any of your infrastructure, except your Python version. Oh, I agree that whoever upgrades their Python version should be able to fix any of their applications should they start failing. That's why I don't want to let new command-line options and environment variables proliferate in the name of damage control. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Tue, 2 Sep 2014 16:47:35 -0700 Glyph Lefkowitz wrote: > > On Sep 2, 2014, at 4:28 PM, Nick Coghlan wrote: > > > On 3 Sep 2014 09:08, "David Reid" wrote: > > > > > > Nick Coghlan gmail.com> writes: > > > > > > > Creating *new* incompatibilities between Python 2 & Python 3 is a major > > > > point > > > > of concern. > > > > > > Clearly this change should be backported to Python2. > > > > Proposing to break backwards compatibility in a maintenance release (...) > > > > As we keep saying, this is not a break in backwards compatibility, it's a bug > fix. Keeping saying it doesn't make it magically true. Besides, it can perfectly well be a bug fix *as well as* a break in backwards compatibility. Which is why we sometimes choose to fix bugs only in the feature development branch. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sad status of Python 3.x buildbots
On Wed, 3 Sep 2014 00:13:22 +0200 Victor Stinner wrote: > > AMD64 Snow Leop 3.x: many tests are not reliable (stable) on this > platform. For example, test_logging.test_race() sometimes fail with > PermissionError(1, "Operation not permitted: > '/tmp/test_logging-3-bjulw8iz.log'"). Another example, > test_asyncio.test_stdin_broken_pipe() sometimes fail with an assertion > error because BrokenPipeError was not raised. Do we still support this > old version of Mac OS X? Released in 2009, it is 5 years old. 5 years old is not that old. I don't know what the upgrade expectations are on OS X, but I'm sure Python runs well on 5-yeard old Windows, Linuces and perhaps even FreeBSDs. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sad status of Python 3.x buildbots
On Wed, 3 Sep 2014 08:53:51 +1000 Nick Coghlan wrote: > On 3 Sep 2014 08:15, "Victor Stinner" wrote: > > > > x86 RHEL 6 3.x: TestReadline.test_init() fails, issue #19884. I don't > > have to this platform, I don't know how to fix it. > > Sorry, I haven't been a very good maintainer for that buildbot (the main > reason it never graduated to the "stable" list). If you send me your public > SSH key, I can add it (I think - if not, I can ask Luke to do it). > Alternatively, CentOS 6 may exhibit the same problem. > > From a completely different perspective, does anyone have experience with > using BuildBot with OpenStack hosted clients? We may be able to take > advantage of the PSF's new(ish) Rackspace infrastructure to provide more > stable test VMs. I'm not sure that's an answer to the problem. What we need is not more machines, but dedicated buildbot maintainers. That also means being willing to diagnose issues themselves rather than kindly offer SSH access, by the way ;-) (I can't speak for Victor, but being offered SSH access when I point out an issue on a remote machine is really a depressing answer to get, as far as I'm concerned) And, we also need some people to look after the master configuration and status - yes, that would have been me, but I've not been very motivated recently :-) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Wed, 3 Sep 2014 20:34:32 +1000 Nick Coghlan wrote: > > The backwards compatibility argument only applies to Python 2 maintenance > releases (where dreid indicated an intention to request backporting the > change), and there I'm quite happy to take the position of "use requests, > Twisted or Python 3.5+ to get HTTPS done right". or asyncio. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Tue, 02 Sep 2014 21:29:16 -0400 "R. David Murray" wrote: > > The top proposal so far is an sslcustomize.py file that could be used to > either decrease or increase the default security. This is a much less > handy solution than application options (eg, curl, wget) that allow > disabling security for "this cert" or "this CLI session". It also is > more prone to unthinking abuse since it is persistent. So perhaps > it is indeed not worth it. (That's why I suggested an environment > variable...something you could specify on the command line for a one-off.) I'll be fine with not adding any hooks at all, and letting people configure their application code correctly :-) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Wed, 3 Sep 2014 10:54:55 -0700 Guido van Rossum wrote: > > Let's take the plunge on this issue for the next 2.7 release (3.5 being a > done deal). I'm entirely against this. > Yes, some people will find that they have an old script > accessing an old service which breaks. Surely some of the other changes in > the same 2.7 bugfix release will also break some other scripts. People deal > with it. Probably 90% of the time it's an annoyance (but no worse than any > other minor-release upgrade -- you should test upgrades before committing > to them, and if all else fails, roll it back). Python is routinely updated to bugfix releases by Linux distributions and other distribution channels, you usually have no say over what's shipped in those updates. This is not like changing the major version used for executing the script, which is normally a manual change. > Today (working at Dropbox, a much smaller company!) I don't > even remember the last time I had to deal with such a browser > complaint -- internal services here all redirect to SSL, and not a > browser that can find fault with their certs. Good for you. I still sometimes get warnings about expired certificates - and sometimes ones that don't exactly match the domain being fetched (for example, the certificate wouldn't be valid for that specific subdomain - note that CAs often charge a premium for multiple subdomains, which why small or non-profit Web sites sometimes skimp on them). You shouldn't assume that the experience of well-connected people in the Silicon Valley is representative of what people over the world encounter. Yes, where there's a lot of money and a lot of accumulated domain competence, security procedures are updated and followed more scrupulously... > But at least some of the > time it will be a wake-up call and an expired certificate will be replaced, > resulting in more security for all. Only if you are actually the one managing that certificate and the machine it's installed one... Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Thu, 4 Sep 2014 09:19:56 +1000 Nick Coghlan wrote: > > > > Python is routinely updated to bugfix releases by Linux distributions > > and other distribution channels, you usually have no say over what's > > shipped in those updates. This is not like changing the major version > > used for executing the script, which is normally a manual change. > > We can potentially deal with the more conservative part of the user base on > the redistributor side - so long as the PEP says it's OK for us to not > apply this particular change if we deem it appropriate to do so. So people would believe python.org that they would get HTTPS cert validation by default, but their upstream distributor would have disabled it for them? That's even worse... Of course, people could read distribution-specific package changelogs, but nobody does that. > 2.7.9 is going to be a somewhat "interesting" release that requires careful > attention anyway (due to the completion of the PEP 466 backports), so if > Guido's OK with it, sure, let's kill the "HTTPS isn't" problem for Python 2 > as well. Possible unvoluntary breakage due to a large backport is one thing. Deliberate breakage is another. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Thu, 4 Sep 2014 13:11:38 +1000 Nick Coghlan wrote: > That leaves Python 2.7, and I have to say I'm now persuaded that a > backport (including any required httplib and urllib features) is the > right way to go. One of the tasks I'd been dreading as a follow-on > from PEP 466 was organising the code audit to make sure our existing > Python 2 applications are properly configuring SSL. If we instead > change Python 2.7.9 to validate certificates by default, then the need > to do that audit *goes away*, replaced by the far more mundane tasking > of doing integration testing on 2.7.9, which we'd have to do *anyway*. What are "our existing Python 2 applications"? Is it a Red Hat-specific statement? What is the "code audit" you are talking about? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython and parallel make
On Thu, 4 Sep 2014 12:06:25 +0200 Jonas Wagner wrote: > > One additional issue appeared, though: it seems that extension modules are > always built sequentially no matter what the value of MAKEFLAGS is. I've > seen that these are being built by a custom setup.py script. Do you think > it would be possible to parallelize this script? Would people be interested > in having a parallel version? See http://bugs.python.org/issue5309 Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.4): Issue #22295: Adopt 'python -m pip' as the preferred invocation
On Sat, 6 Sep 2014 12:40:19 +0200 (CEST) nick.coghlan wrote: > > The following command will install the latest version of a module and its > dependencies from the Python Package Index:: > > -pip install SomePackage > +python -m pip install SomePackage > + > +.. note:: > + > + For POSIX users (including Mac OS X and Linux users), the examples in > + this guide assume the use of a :term:`virtual environment`. Why not advocate --user instead? It is simpler than messing around with virtual environments and will suffice for most use cases. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed schedule for 3.4.2
On Mon, 8 Sep 2014 10:44:51 -0700 Alex Gaynor wrote: > *Shifts uncomfortably* it looks like presently there's not a good way to > change anything about the SSL configuration for urllib.request.urlopen. It > does not take a `context` argument, as the http.client API does: > https://docs.python.org/3/library/urllib.request.html#module-urllib.request > and instead takes the cafile, capath, cadefault args. The reason it doesn't take a context argument is that SSL contexts are hard to understand and configure for the average urlopen() user (why should people know about ssl module specifics to open an HTTPS URL?). http.client is more low-level and therefore more amenable to such powerful knobs. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed schedule for 3.4.2
On Tue, 9 Sep 2014 08:20:52 +1000
Nick Coghlan wrote:
> On 9 Sep 2014 04:00, "Barry Warsaw" wrote:
> > >
> > >This would need to be updated first, once it *did* take such an argument,
> > >this would be accomplished by:
> > >
> > >context = ssl.create_default_context()
> > >context.verify_mode = CERT_OPTIONACERT_NONE
> > >context.verify_hostname = False
> > >urllib.request.urlopen("
> https://something-i-apparently-dont-care-much-about";,
> > >context=context)
> >
> > There's probably an ugly hack possibility that uses unittest.mock.patch.
> ;)
>
> We could actually make it an "official" hack:
>
> import urllib.request
> urllib.request.urlopen = urllib.request._unverified_urlopen
-1. Instead of disabling cert verification for *one* urlopen() call
site, you're doing it for *every* urlopen call site in the program - or,
even worse, for every urlopen that's imported after the monkey-patching
(which makes the final effect potentially dependent on module import
order, and import style). It may affect third-party libraries that have
their own REST calls, or whatever.
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multilingual programming article on the Red Hat Developer blog
On Fri, 12 Sep 2014 07:54:56 +0100 Jeff Allen wrote: > Simply having a block "for private use" seems to create an unmanaged > space for conflict, reminiscent of the "other 128 characters" in > bilingual programming. I wondered if the way to respect use by > applications might be to make it private to a particular sub-class of > str, idly however. It's not private from Python's point of view, it's actually specified in a PEP. So all Python 3 code has to follow the rule, and there's no conflict internally. The characters shouldn't leak out to other applications, unless the user's code does its I/O very badly :-) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuples bug between 3.3.2 and 3.4.1
On Mon, 15 Sep 2014 02:13:53 +1000
Chris Angelico wrote:
> On Sun, Sep 14, 2014 at 8:13 PM, Brynjar Smári Bjarnason
> wrote:
> > I am using Python 3.4.1 installed with Anaconda. I tried the following
> > (expecting an OrderedDict as result):
> >
> >>>>from collections import namedtuple
> >>>>NT = namedtuple("NT",["a","b"])
> >>>>nt = NT(1,2)
> >>>>print(vars(nt))
> > {}
> >
> > so the result is an empty dict. In Python 3.3.2 (downgraded in the
> > same Anaconda environment) results in:
> >
> >>>>print(vars(nt))
> > OrderedDict([('a', 1), ('b', 2)])
>
> For what it's worth, I can't reproduce the issue on trunk CPython
> (built from default branch on Aug 21, so it's a little old now), nor
> on 3.4.1 as currently shipping with Debian Jessie, nor with 3.4.0 on
> Windows. So this may be an Anaconda issue. Do you know if it's meant
> to be a patched install of Python?
There may be a couple distutils-specific patches, but that's all.
Anaconda issues should be reported at
https://github.com/ContinuumIO/anaconda-issues/issues/
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] List insert at index that is well out of range - behaves like append
On Mon, 15 Sep 2014 23:46:03 +0100 Mark Lawrence wrote: > > I assume it's based on the concepts of slicing. From the docs > "s.insert(i, x) - inserts x into s at the index given by i (same as > s[i:i] = [x])". Although shouldn't that read s[i:i+1] = [x] ? No, the latter would replace the contents at index i, while the former inserts it (formally, it replaces the 0-length slice with a 1-length slice). Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multilingual programming article on the Red Hat Developer blog
Seriously, can this discussion move somewhere else? This has nothing to do on python-dev. Thank you Antoine. On Wed, 17 Sep 2014 18:56:02 +1000 Steven D'Aprano wrote: > On Wed, Sep 17, 2014 at 09:21:56AM +0900, Stephen J. Turnbull wrote: > > > Guido's mantra is something like "Python's str doesn't contain > > characters or even code points[1], it contains code units." > > But is that true? If it were true, I would expect to be able to make > Python text strings containing code units that aren't code points, e.g. > something like "\U1234" or chr(0x1234) should work, but neither > do. As far as I can tell, there is no way to build a string containing > items which aren't code points. > > I don't think it is useful to say that strings *contain* code units, > more that they *are made up from* code units. Code units are the > implementation: 16-bit code units in narrow builds, 32-bit code units > in wide builds, and either 8-, 16- or 32-bit code units in Python 3.3 and > beyond. (I don't know of any Python implementation which uses UTF-8 > internally, but if there was one, it would use 8-bit code units.) > > It isn't very useful to say that in Python 3.3 the string "A" *contains* > the 8-bit code unit 0x41. That's conflating two different levels of > explanation (the high-level interface and the underlying implemention) > and potentially leads to user confusion like > > # 8-bit code units are bytes, right? > assert b'\41' in "A" > > which is Not Even Wrong. > http://rationalwiki.org/wiki/Not_even_wrong > > I think it is correct to say that Python strings are sequences of > Unicode code points U+ through U+10. There are no other > restrictions, e.g. strings can contain surrogates, noncharacters, or > nonsensical combinations of code points such as a U+0300 COMBINING GRAVE > ACCENT combined with U+000A (newline). > > > > Implying > > that dealing with characters (or the grapheme globs that occasionally > > raise their ugly heads here) is an issue for higher-level facilities > > than str to deal with. > > Agreed that Python doesn't offer a string type based on graphemes, and > that such a facility belongs as a high-level library, not a built-in > type. > > Also agreed that talking about characters is sloppy. Nevertheless, for > English speakers at least, "code point = character" isn't too awful a > first approximation. > > > > The point being that > > > > > Basically, we are pretending that the each smuggled byte is single > > > character > > > > is something of a misstatement (good enough for present purpose of > > discussing email, but not good enough for the general case of > > understanding how this is supposed to work when porting the construct > > to other Python implementations), while > > > > > for string parsing purposes...but they don't match any of our > > > parsing constants. > > > > is precisely Pythonically correct. You might want to add "because all > > parsing constants contain only valid characters by construction." > > I don't understand what you are trying to say here. > > > > > [*] I worried a lot that this was re-introducing the bytes/string > > > problem from python2. > > > > It isn't, because the bytes/str problem was that given a str object > > out of context you could not tell whether it was a binary blob or > > text, and if text, you couldn't tell if it was external encoded text > > or internal abstract text. > > > > That is not true here because the representations of characters vs. > > smuggled bytes in str are disjoint sets. > > Nor am I sure what you are trying to say here either. > > > > Footnotes: > > [1] In Unicode terminology, a code unit is the smallest computer > > object that can represent a character (this is uniquely and sanely > > defined for all real Unicode transformation formats aka UTFs). A code > > point is an integer 0 - (17*256*256-1) that can represent a character, > > but many code points such as surrogates and 0x are defined to be > > non-characters. > > Actually not quite. "Noncharacter" is concretely defined in Unicode, and > there are only 66 of them, many fewer than the surrogate code points > alone. Surrogates are reserved, not noncharacters. > > http://www.unicode.org/glossary/#surrogate_code_point > http://www.unicode.org/faq/private_use.h
Re: [Python-Dev] PEP 394 - Clarification of what "python" command should invoke
On Fri, 19 Sep 2014 08:20:48 -0700 Guido van Rossum wrote: > "python" should always be the same as "python2". "Always" as in "eternally"? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.5 release schedule PEP
On Wed, 24 Sep 2014 17:12:35 +1000 Nick Coghlan wrote: > On 24 Sep 2014 15:15, "Tim Golden" wrote: > > > > On 23/09/2014 18:05, Steve Dower wrote: > >> > >> I'm also considering/experimenting with installing into "Program > >> Files" by default, but I suspect that isn't going to work out yet. > > > > > > I'd like to see that go forward: I think it's increasingly difficult to > justify Python's position at c:\pythonxx. But it does run the risk of > > breaking All The Things. When it was discussed, people spoke about > > symlinks (or hardlinks / junctions / whatever) in the appropriate > > places to support hardcoded paths, but I don't know how feasible > > that will turn out to be. > > It might be better to offer that as a supported option in 3.5, and then > make it the default in 3.6. I would be surprised if this weren't already a supported option. Decent Windows installers generally allow you to override the installation path (it's been a while I haven't used the Windows Python installer, sorry). I think making the move in 3.5 would be a good idea. Experts can override the installation path and choose C:\PythonXY. There's no actual breakage since the path changes for every major release, anyway (i.e. people would have had to change the path from "C:\Python34" to "C:\Python35"; instead, they will have to change it to "C:\Program Files\Python35"). Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.5 release schedule PEP
On Wed, 24 Sep 2014 21:32:52 +0200 Victor Stinner wrote: > Most Windows setup are desktop configured with a single user. I would not > be shocked if pip installs modules only for the current user by default. > Maybe it could be an option in Python installer (pip system wide or user). pip install --user ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.5 release schedule PEP
On Wed, 24 Sep 2014 22:56:20 +0100 Paul Moore wrote: > On 24 September 2014 22:29, Steve Dower wrote: > >> In my experience pip --user works just fine also. We use it on our unmanned > >> media players successfully. > > > > This is good to know. Maybe we aren't as far away as we think. > > Indeed. Moving towards having --user as the norm is definitely > something we want to look at for pip. One of the biggest concerns is > how well-exercised the whole user site directory area is in practice. What do you mean by well-exercised? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.5 release schedule PEP
On Thu, 25 Sep 2014 07:34:31 +0100 Paul Moore wrote: > On 25 September 2014 02:08, Antoine Pitrou wrote: > >> Indeed. Moving towards having --user as the norm is definitely > >> something we want to look at for pip. One of the biggest concerns is > >> how well-exercised the whole user site directory area is in practice. > > > > What do you mean by well-exercised? > > Basically, although --user is available in pip (and the underlying > facilities in Python have been around for some time), it's difficult > to gauge how many people are using them, and as a result what level of > testing has happened in real-life situations. I'm using it often. I'm also unsure how broken it could be. The user site-packages is just another site-packages directory. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.5 release schedule PEP
Le 25/09/2014 09:22, INADA Naoki a écrit : > FYI, homebrew's Python uses prefix option, so I can't use `--user`. > Is it a bug? > > $ /usr/local/bin/pip -V > pip 1.5.6 from > /usr/local/Cellar/python/2.7.8_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg > (python 2.7) > > $ /usr/local/bin/pip install --user tornado > ... > error: can't combine user with prefix, exec_prefix/home, or install_(plat)base Yes, it is. http://bugs.python.org/issue22269 Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX
On Thu, 25 Sep 2014 13:00:16 -0700 Bob Hanson wrote: > Critical bash vulnerability CVE-2014-6271 may affect Python on > *n*x and OSX: > > <http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-6271> > > <http://arstechnica.com/security/2014/09/bug-in-bash-shell-creates-big-security-hole-on-anything-with-nix-in-it/> > > <http://lcamtuf.blogspot.com/2014/09/quick-notes-about-bash-bug-its-impact.html> > > <http://www.openwall.com/lists/oss-security/2014/09/24/17> > > Also see <news:gmane.comp.security.fulldisclosure> for thread on > same being started today. Fortunately, Python's subprocess has its `shell` argument default to False. However, `os.system` invokes the shell implicitly and is therefore a possible attack vector. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX
On Fri, 26 Sep 2014 09:40:17 +1000
Steven D'Aprano wrote:
> Perhaps I'm missing something, but aren't there easier ways to attack
> os.system than the bash env vulnerability? If I'm accepting and running
> arbitrary strings from an untrusted user, there's no need for them to go
> to the trouble of feeding me:
>
> "env x='() { :;}; echo gotcha' bash -c 'echo do something useful'"
>
> when they can just feed me:
>
> "echo gotcha"
>
> In other words, os.system is *already* an attack vector, unless you only
> use it with trusted strings. I don't think the bash env vulnerability
> adds to the attack surface.
>
> Have I missed something?
The part where the attack payload is passed through the environment, not
through hypothetical user-injected command-line arguments.
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX
On Fri, 26 Sep 2014 01:10:53 -0700 Hasan Diwan wrote: > Matěj, > > On 26 September 2014 00:28, Matěj Cepl wrote: > > > Where does your faith that other /bin/sh implementations (dash, > > busybox, etc.) are less buggy comes from? > > > The fact that they are simpler, in terms of lines of code. It's no > guarantee, but the less a given piece of code does, the less bugs it will > have. -- H And that they have less "features" (which is certainly correlated to their simplicity). IIUC, the misimplemented feature leading to this vulnerability is a bash-ism. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Critical bash vulnerability CVE-2014-6271 may affect Python on *n*x and OSX
On Fri, 26 Sep 2014 14:56:05 +0200 Stefan Behnel wrote: > Jeremy Sanders schrieb am 26.09.2014 um 09:28: > > Antoine Pitrou wrote: > > > >> Fortunately, Python's subprocess has its `shell` argument default to > >> False. However, `os.system` invokes the shell implicitly and is > >> therefore a possible attack vector. > > > > Of course anything called by subprocess with shell=False may invoke the > > shell itself if it runs other processes. > > Ok, but does that really make it a relevant topic for python-dev? No. I don't know why the OP posted here. (but we have all kinds of borderline discussion threads these days, and people don't seem to care when they are asked to move the discussion elsewhere, so...) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Microsoft Visual C++ Compiler for Python 2.7
On Fri, 26 Sep 2014 18:01:31 + Steve Dower wrote: > Hi all, > > (This is advance notice since people on this list will be interested. > Official announcements are coming when setuptools makes their next release.) When you mention "setuptools", do you imply it doesn't work with plain distutils? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Microsoft Visual C++ Compiler for Python 2.7
On Sat, 27 Sep 2014 14:10:48 +0100 Paul Moore wrote: > On 27 September 2014 14:01, Nick Coghlan wrote: > > I personally believe we should treat handling both this and the SDK > > compilers properly as a platform-enablement bug for distutils and > > ensure they work properly with the currently maintained branches > > (including 2.7). > > +1 +1 as well. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fixing 2.7.x
On Mon, 06 Oct 2014 09:08:23 -0700 Ethan Furman wrote: > With the incredibly long life span of 2.7, which bugs should we *not* fix?* Those that are not bugs but enhancement requests. On that issue, you pointed out there was no regression and that enums were never meant to be supported by the json module at that time, so IMHO it's a won't fix. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] mUTF-7 support?
On Fri, 10 Oct 2014 00:47:46 +0200 Jesus Cea wrote: > I miss mUTF-7 support (as used to encode IMAP4 mailbox names) in Python, > in the codecs module. As an european with a language with 27 different > letters (instead of english 26), tildes, opening question marks, etc., I > find it very inconvenient. > > This encoding is used basically only in IMAP4, I know. But IMAP4 is an > important protocol and all projects related to it needs mUTF-7 support > if they care about non-english alphabets. Everybody has already an > implementation, waste of effort. This sounds good to me. Feel free to propose a patch for 3.5. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] mUTF-7 support?
On Thu, 9 Oct 2014 19:12:29 -0700 Dan Stromberg wrote: > On Thu, Oct 9, 2014 at 3:47 PM, Jesus Cea wrote: > > I miss mUTF-7 support (as used to encode IMAP4 mailbox names) in Python, > > in the codecs module. As an european with a language with 27 different > > letters (instead of english 26), tildes, opening question marks, etc., I > > find it very inconvenient. > > > > This encoding is used basically only in IMAP4, I know. But IMAP4 is an > > important protocol and all projects related to it needs mUTF-7 support > > if they care about non-english alphabets. Everybody has already an > > implementation, waste of effort. > > I've been parsing up a huge gmail account with no encoding errors, > using CPython 2.x and CPython 3.x. I'd be surprised if there are no > foreign characters in any of the thousands of messages there - but > maybe I'm just being very lucky. I'm not specifying a codec, and I > don't see a way of specifying one offhand. AFAIU, this is specifically about mailbox names, not messages. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] mUTF-7 support?
On Fri, 10 Oct 2014 13:36:49 +1100 Chris Angelico wrote: > On Fri, Oct 10, 2014 at 11:52 AM, Jesus Cea wrote: > > Actually, it doesn't work in Python 2 either. It never supported > > international mailbox names. > > > > Should I dare to suggest to port this to 2.7, since 2.7 is special and > > will be supported for a long time?. Or maybe this is something like > > "Yes, Python 2 is broken, the real deal is Python 3"? :). > > That's ultimately up to the release manager, but IMO this sounds like > a bug to be fixed, more than a feature being added. Well, it would be a bug if we had claimed to support it. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sad status of Python 3.x buildbots
On Fri, 10 Oct 2014 12:08:54 -0400 "R. David Murray" wrote: > On Fri, 10 Oct 2014 18:00:06 +0200, Jesus Cea wrote: > > On 10/10/14 17:56, Chris Angelico wrote: > > > Could I write a little > > > monitor at my end that asks every hour if my buildbots can be seen? > > > > AFAIK maintainers already get an email if the buildbot vanishes long > > enough. I am more interested in getting an email when my buildbot is > > consistently red because somebody committed something it doesn't like > > two months ago... > > I think they are supposed to, but I've never gotten one, not even when > my gentoo buildbots suffered a hardware failure. As far as I remember it's a relaying problem on the SMTP server. Perhaps Benjamin can confirm. > That said, there has never as far as I know been an active hook to email > the maintainer when the buildbot is red. I don't know if buildbot > supports that or not, so that would be the first thing to check. I don't know if it would support e-mailing the maintainer rather than whoever committed the last changes. Besides, I'm afraid our test suite isn't stable enough for such e-mail notifications to be bearable... Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of C compilers for Python on Windows
On Sat, 11 Oct 2014 00:30:51 + (UTC) Sturla Molden wrote: > Larry Hastings wrote: > > > CPython doesn't require OpenBLAS. Not that I am not receptive to the > > needs of the numeric community... but, on the other hand, who in the > > hell releases a library with Windows support that doesn't work with MSVC?! > > It uses AT&T assembly syntax instead of Intel assembly syntax. But you can compile OpenBLAS with one compiler and then link it to Python using another compiler, right? There is a single C ABI. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of C compilers for Python on Windows
On Sat, 11 Oct 2014 13:59:52 + (UTC) Sturla Molden wrote: > Antoine Pitrou wrote: > > > But you can compile OpenBLAS with one compiler and then link it to > > Python using another compiler, right? There is a single C ABI. > > BLAS and LAPACK are actually Fortran, which does not have a single C ABI. > The ABI depends on the Fortran compiler. g77 and gfortran will produce > different C ABIs. This is a consistent source of PITA in any scientific > programming that combines C and Fortran. But is that CPython's fault? I don't think so. > Another thing is that libraries are different. MSVC wants a .lib file, but > MinGW produces .a files like GCC does on Linux. It sound like whatever MSVC produces should be the defacto standard under Windows. If Microsoft released a (GNU/)Linux compiler which produced incompatible library files, I don't think anyone would regard them very highly. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Disabling SSL 3.0
On Wed, 15 Oct 2014 01:16:26 +0200 Victor Stinner wrote: > Hi, > > I opened an issue to track this vulnerability: > http://bugs.python.org/issue22638 > > SSL 3.0 is 8 years old, I guess that TLS is now widely deployed and > well supported? > > I guess that Linux vendors will have to fix the issues directly in > OpenSSL directly. Should Python only be changed on Windows? If OpenSSL gets a patch, we can simply update the OpenSSL version used for Windows installers. > Or do you want to modify Python to disable SSLv3 in the ssl module? > OpenSSL provides a SSL_OP_NO_SSLv2 option for SSL context. Is there a > SSL_OP_NO_SSLv3 option? Or only change the constructor of > ssl.SSLContext? Please let's not have this discussion on two different channels. *Either* the bug tracker or the mailing-list. Thank you Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How io.IOBase.readline() should behave when used on non-blocking obj and no data available?
On Thu, 16 Oct 2014 03:54:32 +0300
Paul Sokolovsky wrote:
> Hello,
>
> io.RawIOBase.read() is well specified for behavior in case it
> immediately gets a would-block condition: "If the object is in
> non-blocking mode and no bytes are available, None is returned."
> (https://docs.python.org/3/library/io.html#io.RawIOBase.read).
>
> However, nothing is said about such condition for io.IOBase.readline(),
> which is mixin method in a base class, default implementation of which
> thus would use io.RawIOBase.read(). Looking at 3.4.0 source, iobase.c:
> iobase_readline() has:
>
> b = _PyObject_CallMethodId(self, &PyId_read, "n", nreadahead);
> [...]
> if (!PyBytes_Check(b)) {
> PyErr_Format(PyExc_IOError,
> "read() should have returned a bytes object, "
> "not '%.200s'", Py_TYPE(b)->tp_name);
>
> I.e. it's not even ready to receive legitimate return value of None
> from read(). I didn't try to write a testcase though, so may be missing
> something.
>
> So, how readline() should behave in this case, and can that be
> specified in the Library Reference?
Well, the problem is that it's not obvious how to implement such methods
in a non-blocking context.
Let's says some data is received but there isn't a complete line.
Should readline() return just that data (an incomplete line)? That
breaks the API's contract. Should readline() buffer the incomplete line
and keep it for the next readline() call? But then the internal buffer
becomes unbounded: perhaps there is no new line in the next 4GB of
incoming data...
And besides, raw I/O objects *shouldn't* have an internal buffer. That's
the role of the buffered I/O layer.
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XP buildbot problem cloning from hg.python.org
On Fri, 24 Oct 2014 23:47:05 -0400 David Bolen wrote: > Starting yesterday, my XP buildbot began failing to execute clone > operations against hg.python.org. There's not a lot of data being > given aside from a transaction abort message (and my buildbot log > showing the hg command exiting), and I'm wondering if something may be > amiss on the server or its configuration? Have you tried running the hg clone manually from the buildbot? You could try to add --debug to get more info where the thing breaks. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
