Re: [Python-Dev] How VPython 6 differs from VPython 5
Le Sun, 13 Jan 2013 19:58:52 -0700, Bruce Sherwood a écrit : > Since this was copied to the Python-Dev list, I want to go on record > as stating firmly that there is no evidence whatsoever to > substantiate claims that there has ever been some kind of conflict > between VPython and Python. > > Since __future__ was also mentioned, I'll take the opportunity to say > that I've been very impressed by the way the Python community has > handled the difficult 2->3 transition. For example, it has been a big > help to the educational uses of VPython that we could tell students > simply to start with "from __future__ import division, > print_function", put parens around print arguments, and thereby make > it irrelevant whether they used Python 2 or Python 3. Many thanks to > the Python development community! You're welcome :) Best regards Antoine. > > Bruce Sherwood > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors
On 13/01/2013 00:41, Victor Stinner wrote: PEP: 433 Title: Add cloexec argument to functions creating file descriptors Status: Draft The PEP is still a draft. I'm sending it to python-dev to get a first review. The main question is the choice between the 3 different options: * don't set close-on-exec flag by default * always set close-on-exec flag * add sys.setdefaultcloexec() to leave the choice to the application Victor Nice clear explanation. I think io, meaning _io and _pyio really, would be amongst the impacted modules, and should perhaps be in the examples. (I am currently working on the Jython implementation of the _io module.) It seems to me that io.open, and probably all the constructors, such as _io.FileIO, would need the extra information as a mode or a boolean argument like closefd. This may be a factor in your choice above. Other things I noticed were minor, and I infer that they should wait until principles are settled. Jeff Allen ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Visualpython-users] How VPython 6 differs from VPython 5
Since this was copied to the Python-Dev list, I want to go on record as stating firmly that there is no evidence whatsoever to substantiate claims that there has ever been some kind of conflict between VPython and Python. Since __future__ was also mentioned, I'll take the opportunity to say that I've been very impressed by the way the Python community has handled the difficult 2->3 transition. For example, it has been a big help to the educational uses of VPython that we could tell students simply to start with "from __future__ import division, print_function", put parens around print arguments, and thereby make it irrelevant whether they used Python 2 or Python 3. Many thanks to the Python development community! Bruce Sherwood On Sun, Jan 13, 2013 at 6:41 PM, Mark Janssen wrote: > On Sun, Jan 13, 2013 at 12:14 PM, Bruce Sherwood > wrote: > > For the record, I do not know of any evidence whatsoever for a supposed > > "split" between the tiny VPython community and the huge Python community > > concerning floating point variables. Nor do I see anything in Python that > > needs to be "fixed". > > Well it was bit enough that the python community created a brand-new > language construct called import __future__ -- something never > considered before then and which actually changes the behavior of > python unlike any other module. And perhaps I've just felt it more > because I was a big proponent of both 3-d graphics as a way to keep > python a draw for beginning programmers and also a big fan of > scientific simulation. No one had anything near vpython back then. > (But ultimately I need to stop mentioning this issue to this vpython > list because it's really the Python group which need to get back in > sync.) > > > The new (currently experimental) version of VPython based on wxPython > must, > > in order to run in a Cocoa environment on the Mac, make the interact > loop be > > the primary thread, with the user's Python calculations at worst a > secondary > > thread, which is the opposite of the older VPython, where the > > interval-driven rendering thread was secondary to the user's > calculations. > > By the unusual stratagem of having VPython import the user's program it > has > > been possible to make this work, and at the same time greatly simplify > the > > C++ component of VPython by eliminating threading, with additional > important > > simplification from eliminating essentially all platform-dependent code > > thanks to the multiplatform character of wxPython. The result is that > nearly > > all existing VPython programs will work without change, at the very small > > cost of a few marginal cases requiring minor tweaking. I should alter the > > documentation to make this important property of the new version more > > salient. > > I need to analyze this more carefully before commenting further > > mark > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Visualpython-users] How VPython 6 differs from VPython 5
For the record, I do not know of any evidence whatsoever for a supposed "split" between the tiny VPython community and the huge Python community concerning floating point variables. Nor do I see anything in Python that needs to be "fixed". The new (currently experimental) version of VPython based on wxPython must, in order to run in a Cocoa environment on the Mac, make the interact loop be the primary thread, with the user's Python calculations at worst a secondary thread, which is the opposite of the older VPython, where the interval-driven rendering thread was secondary to the user's calculations. By the unusual stratagem of having VPython import the user's program it has been possible to make this work, and at the same time greatly simplify the C++ component of VPython by eliminating threading, with additional important simplification from eliminating essentially all platform-dependent code thanks to the multiplatform character of wxPython. The result is that nearly all existing VPython programs will work without change, at the very small cost of a few marginal cases requiring minor tweaking. I should alter the documentation to make this important property of the new version more salient. Minor point: The meaning of the new VPython "sleep" is precisely to relinquish control to the OS, including the GUI. During sleep one wants the wxPython window controls to remain active, which will not be the case with time.sleep, which knows nothing about the wxPython interact loop. So I think the meaning of "sleep" is the usual meaning. Bruce Sherwood On Sun, Jan 13, 2013 at 9:13 AM, Mark Janssen wrote: > On Sun, Jan 13, 2013 at 1:15 AM, Bruce Sherwood > wrote: > > Here is detailed information on how VPython 6 differs from VPython 5, > which > > will be incorporated in the Help for upcoming releases of VPython 6. Note > > that the fact that in a main program __name__ isn't '__main__' is an > > unavoidable "feature", not a bug. That is, there doesn't seem to be a > way to > > make this work. > > Hi, Bruce. I think there are two ways of fixing this. One is to get > down to the C language level, and share the C main() loop, which will > percolate back up into the CPython loop. The other involves mending > the split that I've been mentioning that happened over floating point > between the VPython developers and the Python-dev group. Then perhaps > Guido will accept VPython using/sharing the __main__ "loop". (I'm > CC'ing him in this message.) So, either of these tracks should fix > this problem. This is why I keep mentioning the important of healing > that split between the parties arguing in the two camps. Perhaps Tim > Peters will be able help bridge this gap. > > > Changes from VPython 5 > > > > The new version makes an essential change to the syntax of VPython > programs. > > Now, an animation loop MUST contain a rate or sleep statement, which > limits > > the number of loop iterations per second as before but also when > appropriate > > (about 30 times per second) updates the 3D scene and handles mouse and > > keyboard events. Without a rate or sleep statement, the scene will not be > > updated until and unless the loop is completed. > > I think this is perfectly acceptible and is just a necessary > restriction wherefrom the OS *must* maintain ultimate control of I/O. > Python sits in userspace surrounded by the larger computation > environment, so this is only fair that an OS call is made so that it > can keep things in "check". Naming it "sleep" is okay, but makes it > sound more like a voluntary thing, and as you probably know, is the > traditional name for relinquishng some control to the OS. (cf. Unix > Systems Programming, Robbins, et al.) > > > You should use the new function sleep rather than time.sleep. The new > > function periodically renders the scene and processes mouse events, > making > > it possible to continue using zoom and rotate, whereas time.sleep does > not > > do this. > > This is strange to me. I think this is where VPython must be ahead of > the curve and not close enough to the Linux development communities. > > > For technical reasons, it is necessary for VPython 6 to do something > rather > > unusual. When you import visual (or vis), your own program is in turn > > imported by the visual module. > > Again this is because of the faction that was created by in 200?, > regarding the Vpython community vs. the python-dev. Really, this > should be mended. > > Anyway, I hope that any of this made sense. > > Thanks for your help! > > Mark > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors
On Mon, Jan 14, 2013 at 8:06 AM, Greg Ewing wrote: > Nick Coghlan wrote: >> >> Agreed, but it's the security implications that let us even >> contemplate the backwards compatibility break. > > > I don't think that's a sufficient criterion for choosing a > name. The backwards compatibility issue is a transitional > thing, but we'll be stuck with the name forever. > > IMO the name should simply describe the mechanism, and be > neutral as to what the reason might be for using the > mechanism. The problem is the mechanism is *not the same* on Windows and POSIX - the Windows mechanism (noinherit) means the file won't be accessible in child processes, the POSIX mechanism (cloexec) means it won't survive exec(). This difference really shows up with multiprocessing, as that uses a bare fork() without exec() on POSIX systems, meaning even flagged descriptors will still be inherited by the child processes. A flag like "sensitive" or "protected" or "private" tells the interpreter "keep this to yourself, as best you can", leaving Python implementors with a certain amount of freedom in the way they translate that into platform appropriate settings (e.g. does "cloexec" or "noinherit" even make sense in Java?) I agree with Glenn's point that "sensitive" in particular is a bad choice, but a more value neutral term like "protect" may still work. We can then explain what it means to protect a file descriptor on the various platforms, and potentially even change the meaning over time as the platforms provide additional capabilities. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors
Hi, Thanks for your feedback, I already updated the PEP for most of your remarks. 2013/1/13 Charles-François Natali : > Also, ISTM that Windows also supports this flag. Yes it does, see Appendix: Operating system support > Windows. >> .. note:: >>OpenBSD older 5.2 does not close the file descriptor with >>close-on-exec flag set if ``fork()`` is used before ``exec()``, but >>it works correctly if ``exec()`` is called without ``fork()``. > > That would be *really* surprising, are your sure your test case is correct? > Otherwise it could be a compilation issue, because I simply can't > believe OpenBSD would ignore the close-on-exec flag. I had this issue with a short Python script. I will try to reproduce it with a C program to double check. >> Impacted modules: >> >> * ``multiprocessing`` >> * ``socketserver`` >> * ``subprocess`` >> * ``tempfile`` > > Hum, I thought temporay file are already created with the close-on-exec flag. "Impacted" is maybe not the best word. I mean that these modules should be modified or that their behaviour may change. It's more a TODO list to ensure that these modules are consistent with PEP. >> XXX Should ``subprocess.Popen`` set the close-on-exec flag on file XXX >> XXX descriptors of the constructor the ``pass_fds`` argument? XXX > > What? > Setting them cloexec would prevent them from being inherited in the > child process! Oops, it's just the opposite: pass_fds should (must?) *clear* the flag :-) (I'm not sure of what should be done here.) >> Add a new optional ``cloexec`` argument to: >> >> * Maybe also: ``os.open()``, ``os.openpty()`` > > Open can be passed O_CLOEXEC directly. See Portability section of Rational: O_CLOEXEC is not portable, even under Linux! It's tricky to use these atomic flags and to implement a fallback. It looks like most developers agree that we should expose an helper or something like that to access the "cloexec" feature for os.open(), os.pipe(), etc. I propose to add it directly in the os module because I don't want to add a new function just for that. (In the kernel/libc) OS functions don't support keyword arguments, so I consider cloexec=True explicit enough. The parameter may be a keyword-only argument, I chose a classic keyword parameter because it's simpler to use them in C. If you don't want to add extra syscalls to os functions using an optional keyword parameter, what do you propose to expose cloexec features to os.open(), os.pipe(), etc.? A new functions? A new module? >> * Many functions of the Python standard library creating file >>descriptors are cannot be changed by this proposal, because adding >>a ``cloexec`` optional argument would be surprising and too many >>functions would need it. For example, ``os.urandom()`` uses a >>temporary file on UNIX, but it calls a function of Windows API on >>Windows. Adding a ``cloexec`` argument to ``os.urandom()`` would >>not make sense. See `Always set close-on-exec flag`_ for an >>incomplete list of functions creating file descriptors. >> * Checking if a module creates file descriptors is difficult. For >>example, ``os.urandom()`` creates a file descriptor on UNIX to read >>``/dev/urandom`` (and closes it at exit), whereas it is implemented >>using a function call on Windows. It is not possible to control >>close-on-exec flag of the file descriptor used by ``os.urandom()``, >>because ``os.urandom()`` API does not allow it. > > I think that the rule of thumb should be simple: > if a library opens a file descriptor which is not exposed to the user, > either because it's opened and closed before returning (e.g. > os.urandom()) or the file descriptor is kept private (e.g. poll(), it > should be set close-on-exec. Ok, it looks fair. I agree *but* only if the file descriptor is closed when the function does exit. select.epoll() doesn't return directly a file descriptor, but an object having a fileno() method. A server may rely on the inherance of this file descriptor, we cannot set close-on-exec flag in this case (if the default is False). It becomes unclear if a function returns an opaque object which contains a file descriptor, but the file descriptor is not accessible. I don't know if Python contains such function. >> Always set close-on-exec flag on new file descriptors created by >> Python. This alternative just changes the default value of the new >> ``cloexec`` argument. > > In a perfect world, all FDS should have been cloexec by default. > But it's too late now, I don't think we can change the default, it's > going to break some applications, and would be a huge deviation from > POSIX (however broken this design decision is). I disagree, according to all emails exchanged: only a very few applications rely on file descriptor inherance. Most of them are already using subprocess with pass_fds, and it should be easy to fix the last applications. I'm not sure that it's so different than the change on subproces
Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors
On Mon, Jan 14, 2013 at 9:23 PM, Victor Stinner wrote: > I'm not sure that it's so different than the change on subprocess > (close_fds=True by default on UNIX since Python 3.2). Do you think > that it would break more applications? Don't ignore the possible explanation that the intersection of the sets "users with applications affected by that change" and "users migrating to Python 3.x" may currently be close to zero. > I disagree but I'm waiting other votes :-) I'm a fan of the conservative approach, with an environment variable and command line option to close FDs by default in 3.4 (similar to PYTHONHASHSEED and -R in the pre-3.3 security releases), and the cloexec/noinherit behaviour becoming the default (with no way to turn it off globally) in 3.5. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors
On Mon, Jan 14, 2013 at 9:23 PM, Victor Stinner wrote: >>> XXX Should ``subprocess.Popen`` set the close-on-exec flag on file XXX >>> XXX descriptors of the constructor the ``pass_fds`` argument? XXX >> >> What? >> Setting them cloexec would prevent them from being inherited in the >> child process! > > Oops, it's just the opposite: pass_fds should (must?) *clear* the flag > :-) (I'm not sure of what should be done here.) Turning off a security feature implicitly isn't a good idea. If someone passes such a descriptor, their child application will fail noisily - it's then up to the developer to decide if they passed the wrong file descriptor, or simply need to ensure the one they passed remains open in the child process. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] Daily reference leaks (aef7db0d3893): sum=287
On 01/12/2013 02:46 PM, Eli Bendersky wrote: The first report is legit, however. PyTuple_New(0) was called and its return value wasn't checked for NULL. The author might have been relying on Python caching the empty tuple. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors
2013/1/14 Nick Coghlan : > I'm a fan of the conservative approach, with an environment variable > and command line option to close FDs by default in 3.4 (similar to > PYTHONHASHSEED and -R in the pre-3.3 security releases), and the > cloexec/noinherit behaviour becoming the default (with no way to turn > it off globally) in 3.5. Do you mean "environment variable and command line option" *instead of* a new sys.setdefaultcloexec() function? An environment variable and a command line option have an advantage over a function: libraries cannot modify the value at runtime (so 2 libraries cannot set different values :-)). 2013/1/14 Nick Coghlan : > Turning off a security feature implicitly isn't a good idea. If > someone passes such a descriptor, their child application will fail > noisily - it's then up to the developer to decide if they passed the > wrong file descriptor, or simply need to ensure the one they passed > remains open in the child process. For my subprocess/pass_fds comment: I wrote it initially while the PEP was proposing to setting close-on-exec flag by default. I will move this comment to the "Set close-on-exec flag by default". Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.3): fix for previous commit related to issue 10527 which didn't have the intended
In article <[email protected]>, giampaolo.rodola wrote: > http://hg.python.org/cpython/rev/88fadc0d7b20 > changeset: 81498:88fadc0d7b20 > branch: 3.3 > parent: 81495:e1c81ab5ad97 > user:Giampaolo Rodola' > date:Mon Jan 14 02:24:25 2013 +0100 > summary: > fix for previous commit related to issue 10527 which didn't have the > intended effect as per http://bugs.python.org/issue10527#msg179895 In article <[email protected]>, giampaolo.rodola wrote: > http://hg.python.org/cpython/rev/831f49cc00fc > changeset: 81497:831f49cc00fc > user:Giampaolo Rodola' > date:Mon Jan 14 02:24:05 2013 +0100 > summary: > fix for previous commit related to issue 10527 which didn't have the > intended effect as per http://bugs.python.org/issue10527#msg179895 > > files: > Lib/multiprocessing/connection.py | 48 +- > Lib/test/test_multiprocessing.py | 2 +- > 2 files changed, 26 insertions(+), 24 deletions(-) > It appears these two changesets were checked in independently rather than merging from branch 3.3 to default. As a result, the 3.3 branch is currently "open"; you'll see this as a +1 heads when doing a hg pull -u. A null merge needs to happen from 3.3 to default to close the 3.3 branch again. Giampaolo, can you take care of this? -- Ned Deily, [email protected] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors
Nick Coghlan wrote: The problem is the mechanism is *not the same* on Windows and POSIX - the Windows mechanism (noinherit) means the file won't be accessible in child processes, the POSIX mechanism (cloexec) means it won't survive exec(). But since there is no such thing as a fork without exec on Windows, the term "cloexec" is still correct. The fd is closed during the exec part of the fork+exec implied by spawning a child process. This difference really shows up with multiprocessing, as that uses a bare fork() without exec() on POSIX systems, meaning even flagged descriptors will still be inherited by the child processes. There are a lot of other ways that this difference will be visible as well, so the difference in cloexec behaviour just needs to be documented in the multiprocessing module along with the rest. I agree with Glenn's point that "sensitive" in particular is a bad choice, but a more value neutral term like "protect" may still work. I think "protect" is *far* too vague. We don't want something so neutral that it gives no clue at all about its meaning. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Issue about LoadError
http://bugs.python.org/issue16715 I assume that someone changes the LoadError class and goes into the revert function. How he can figure out that the cookies should return to the previous state if LoadError was raised ? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
There you go. Obsoleted-By (optional) ::: Indicates that this project is no longer being developed. The named project provides a substitute or replacement. A version declaration may be supplied and must follow the rules described in `Version Specifiers`_. The most common use of this field will be in case a project name changes. Examples:: Name: BadName Obsoleted-By: AcceptableName Obsoleted-By: AcceptableName (>=4.0.0) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
