[Python-Dev] FreeBSD-9.0 bot running --without-doc-strings
Hi, I've subverted the build master authority on the FreeBSD-9.0 bot by exporting with_doc_strings=no. This is to test #16143 and #10156. Some tests assume that docstrings are present, so there will be a couple of failures. If someone wants to work on easy issues, this is your chance. If the failures get annoying just open a tracker issue, and I'll revert the change. Stefan Krah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 433: Choose the default value of the new cloexec parameter
> So, if we agree that "cloexec-by-default" is a desirable goal, despite > the inconsistency with POSIX (just as changing integer division was > seen as desirable, despite the inconsistency with C) ... I don't plan to enable cloexec by default in a near future, nor in python 4. Someone else may change that later. Having a global flag should be enough. Enabling cloexec by default is interesting if *all* libraries respect the global flag (even C libraries), so subprocess can be used with close_fds=False. Closing all file descriptors is slow, especially on FreeBSD (it's now faster on linux, python lists /proc/pid/fd/). Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FreeBSD-9.0 bot running --without-doc-strings
On 26.01.13 12:55, Stefan Krah wrote: Some tests assume that docstrings are present, so there will be a couple of failures. If someone wants to work on easy issues, this is your chance. http://bugs.python.org/issue17040 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FreeBSD-9.0 bot running --without-doc-strings
Hello, On Sat, 26 Jan 2013 11:55:12 +0100 Stefan Krah wrote: > > I've subverted the build master authority on the FreeBSD-9.0 bot by > exporting with_doc_strings=no. This is to test #16143 and #10156. Well... Speaking personally, I'd much rather stop shipping and "supporting" such obscure build options. I'd like to hear about people using production Pythons --without-doc-strings. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FreeBSD-9.0 bot running --without-doc-strings
On Sun, Jan 27, 2013 at 12:45 AM, Antoine Pitrou wrote: > > Hello, > > On Sat, 26 Jan 2013 11:55:12 +0100 > Stefan Krah wrote: >> >> I've subverted the build master authority on the FreeBSD-9.0 bot by >> exporting with_doc_strings=no. This is to test #16143 and #10156. > > Well... Speaking personally, I'd much rather stop shipping and > "supporting" such obscure build options. I'd like to hear about people > using production Pythons --without-doc-strings. I was going to suggest that folks like the PyMite developers might use it to create versions that run in severely constrained environments, but it looks like they wrote their interpreter from scratch instead. On the other hand, I've definitely heard at least a few people talking about using -OO to drop docstrings as a technique to reduce memory consumption. Some of these failures would likely show up there as well. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FreeBSD-9.0 bot running --without-doc-strings
On Sun, 27 Jan 2013 01:07:14 +1000 Nick Coghlan wrote: > On Sun, Jan 27, 2013 at 12:45 AM, Antoine Pitrou wrote: > > > > Hello, > > > > On Sat, 26 Jan 2013 11:55:12 +0100 > > Stefan Krah wrote: > >> > >> I've subverted the build master authority on the FreeBSD-9.0 bot by > >> exporting with_doc_strings=no. This is to test #16143 and #10156. > > > > Well... Speaking personally, I'd much rather stop shipping and > > "supporting" such obscure build options. I'd like to hear about people > > using production Pythons --without-doc-strings. > > I was going to suggest that folks like the PyMite developers might use > it to create versions that run in severely constrained environments, > but it looks like they wrote their interpreter from scratch instead. > > On the other hand, I've definitely heard at least a few people talking > about using -OO to drop docstrings as a technique to reduce memory > consumption. Some of these failures would likely show up there as > well. Using -OO is far less extreme than building your own Python with docstrings disabled. And the test suite already passes with -OO: http://buildbot.python.org/all/builders/AMD64%20Mountain%20Lion%20Optimized%20%5BSB%5D%203.x Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Anyone building Python --without-doc-strings?
Antoine Pitrou wrote: > Well... Speaking personally, I'd much rather stop shipping and > "supporting" such obscure build options. I'd like to hear about people > using production Pythons --without-doc-strings. I'm not sure how accurate the output is for measuring these things, but according to ``ls'' and ``du'' the option is indeed quite worthless: ./configure CFLAGS="-Os -s" LDFLAGS="-s" && make $ du -h build/lib.linux-x86_64-3.4/ 24K build/lib.linux-x86_64-3.4/__pycache__ 3.9Mbuild/lib.linux-x86_64-3.4/ $ ls -lh python 1.8M Jan 26 16:36 python $ ls -lh libpython3.4m.a 9.6M Jan 26 16:36 libpython3.4m.a === ./configure --without-doc-strings CFLAGS="-Os -s" LDFLAGS="-s" && make $ du -h build/lib.linux-x86_64-3.4/ 24K build/lib.linux-x86_64-3.4/__pycache__ 3.8Mbuild/lib.linux-x86_64-3.4/ $ ls -lh python 1.6M Jan 26 16:33 python $ ls -lh libpython3.4m.a 9.4M Jan 26 16:33 libpython3.4m.a Stefan Krah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anyone building Python --without-doc-strings?
Stefan Krah wrote: > I'm not sure how accurate the output is for measuring these things, but > according to ``ls'' and ``du'' the option is indeed quite worthless: > > ./configure CFLAGS="-Os -s" LDFLAGS="-s" && make > 1.8M Jan 26 16:36 python > ./configure --without-doc-strings CFLAGS="-Os -s" LDFLAGS="-s" && make > 1.6M Jan 26 16:33 python The original contribution *was* in fact aiming for "10% smaller", see: http://docs.python.org/release/2.3/whatsnew/node20.html So apparently people thought it was useful. Stefan Krah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anyone building Python --without-doc-strings?
On Sat, 26 Jan 2013 17:03:59 +0100 Stefan Krah wrote: > Stefan Krah wrote: > > I'm not sure how accurate the output is for measuring these things, but > > according to ``ls'' and ``du'' the option is indeed quite worthless: > > > > ./configure CFLAGS="-Os -s" LDFLAGS="-s" && make > > 1.8M Jan 26 16:36 python > > ./configure --without-doc-strings CFLAGS="-Os -s" LDFLAGS="-s" && make > > 1.6M Jan 26 16:33 python > > The original contribution *was* in fact aiming for "10% smaller", see: > > http://docs.python.org/release/2.3/whatsnew/node20.html > > So apparently people thought it was useful. After a bit of digging, I found the following discussions: http://mail.python.org/pipermail/python-dev/2001-November/018444.html http://mail.python.org/pipermail/python-dev/2002-January/019392.html http://bugs.python.org/issue505375 Another reason for accepting the patch seemed to be that it introduced the Py_DOCSTR() macros, which were viewed as helpful for other reasons (some people talked about localizing docstrings). I would point out that if 200 KB is really a big win for someone, then Python (and especially Python 3) is probably not the best language for them. It is also ironic how the executable size went up since then (from 0.6 to more than 1.5 MB) :-) Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anyone building Python --without-doc-strings?
On Sat, 26 Jan 2013 17:19:32 +0100, Antoine Pitrou wrote: > On Sat, 26 Jan 2013 17:03:59 +0100 > Stefan Krah wrote: > > Stefan Krah wrote: > > > I'm not sure how accurate the output is for measuring these things, but > > > according to ``ls'' and ``du'' the option is indeed quite worthless: > > > > > > ./configure CFLAGS="-Os -s" LDFLAGS="-s" && make > > > 1.8M Jan 26 16:36 python > > > ./configure --without-doc-strings CFLAGS="-Os -s" LDFLAGS="-s" && make > > > 1.6M Jan 26 16:33 python > > > > The original contribution *was* in fact aiming for "10% smaller", see: > > > > http://docs.python.org/release/2.3/whatsnew/node20.html > > > > So apparently people thought it was useful. > > After a bit of digging, I found the following discussions: > http://mail.python.org/pipermail/python-dev/2001-November/018444.html > http://mail.python.org/pipermail/python-dev/2002-January/019392.html > http://bugs.python.org/issue505375 > > Another reason for accepting the patch seemed to be that it introduced > the Py_DOCSTR() macros, which were viewed as helpful for other reasons > (some people talked about localizing docstrings). > > I would point out that if 200 KB is really a big win for someone, then > Python (and especially Python 3) is probably not the best language for > them. > > It is also ironic how the executable size went up since then (from 0.6 > to more than 1.5 MB) :-) 200K can make a difference. It does on the QNX platform, for example, where there is no virtual memory. It would be nice to reduce that executable size, toobut I'm not volunteering to try (at least not yet) :) --David ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 433: Choose the default value of the new cloexec parameter
On 25Jan2013 12:24, Victor Stinner wrote: | 2013/1/25 Charles-François Natali : | > You can actually count me in the cloexec=False camp, and against the | > idea of a configurable default value. | | Oh ok. I'm leaning to this view myself also. | > Why the default value shouldn't be tunable: | > - I think it's useless: if the default cloexec behavior can be altered | > (either by a command-line flag, an environment variable or a sys | > module function), then libraries cannot rely on it and have to make | > file descriptors cloexec on an individual basis, since the default | > flag can be disabled. | | In my experience, in most cases, the default value of cloexec just | doesn't matter at all. If your program relies on the state of the | close-on-exec flag: you have to make it explicit, specify cloexec | paramater. Example: os.pipe(cloexec=True). If you don't modify your | application, it will just not work using -e command line option (or | PYTHONCLOEXEC environment variable). But why would you enable cloexec | by default if your application is not compatible? Because we're not all writing applications. I write a lot of small library routines and then use them in my apps. On a UNIX system I can plow on presuming cloexec=False, and will feel no desire to label it in any calls. And so I might be making pipes or whatever and expecting to pass them to subprocesses which will be told the file descriptor numbers on the command line. (Let me say up front that I haven't such an example in my code at present; generally that kind of thing happens in shell script for me.) Let someone change the global default and suddening my library code doesn't work, and people will be tearing their hair out trying to figure out why; because the code was written correctly in the old regime. We have recently seen a mail thread where people were arguing (generally correctly IMO) that programs should not do chdir, because it changes a process wide global. The situation with cloexec is in my mind very analogous. So I am: +1 on offering a cloexec parameter, defaulting to False -0.5 on offering a global default that can be changed [...] | > - We should go over the standard library, and create FDs cloexec if | > they're not handed over to the caller, either because they're | > opened/closed before returning, or because the underlying file | > descriptor is kept private (not fileno() method, although it's | > relatively rare). That's the approach chosen by glibc, and it makes | > sense: if another thread forks() while a thread is in the middle of | > getpwnam(), you don't want to leak an open file descriptor to | > /etc/passwd (or /etc/shadow). | | I started to make cloexec explicit *everywhere* in the Python stdlib. | I reverted my commit because I think that only a few application start | child processes, so doing extra work (additionnal syscalls to set | cloexec) would slowdown Python for no gain. See my revert commit to | see how many functions need to be modified: | http://hg.python.org/features/pep-433/rev/963e450fc24f I see what you mean, it is a quite intrusive change. However, what you're mostly backing out is cloexec=True. There are therefore two changes coming here: - present the cloexec parameter - unilaterally make a global policy decision that in these many places in the stdlib, cloexec=True is now both correct/acceptable and _necessary_ If it were only the former, the change would be much smaller. | That's why I really like the idea of being able to configure the | default value of the cloexec parameter. By default: no overhead nor | backward compatibility issue. Whereas you can set close-on-exec flag | *everywhere* if you are concern by all issues of inheriting file | descriptors (cases listed in the PEP). Quietly breaking libraries that relied on cloexec=False being the status quo... The global tunable still make me feel uneasy. I am in agreement that the general leakage of open files because cloexec defaults to False is untidy and sometimes problematic; the tension here isn't lost on me:-) Cheers, -- Cameron Simpson Out on the road, feeling the breeze, passing the cars. - Bob Seger ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 433: Choose the default value of the new cloexec parameter
On 25Jan2013 21:07, Nick Coghlan wrote: | It's a configurable setting in the same way that -Q makes the | behaviour of "/" configurable in Python 2 (so your hypothetical | example isn't hypothetical at all - it's a description the -Q option), | and -R makes random hashing configurable in 2.7 and 3.2: it means we | can change the default behaviour in a future version (perhaps Python | 4) while allowing people to easily check if their code operates | correctly in that state in the current version. | | I think the default behaviour needs to be configurable from the | environment and the command line, but I don't believe it should be | configurable from within the interpreter. Hmm. This I can live with more happily, though I'm still uneasy. As an aside, I tend to feel that if something is tuneable it should be exposed within the interpreter. Maybe only in an exciting new module called shoot_self_in_foot or some similarly alarming name... Cheers, -- Cameron Simpson That said, I'm inclined to agree that that's not necessarily a good idea. I always wanted to write a little program that would pop up a Mac window to ask ``I'm going to amputate a limb at random from you now.'' to see how many people would instinctively click "OK". - Marc VanHeyningen ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 433: Choose the default value of the new cloexec parameter
On Sat, Jan 26, 2013 at 8:48 PM, Cameron Simpson wrote: > On 25Jan2013 21:07, Nick Coghlan wrote: > | It's a configurable setting in the same way that -Q makes the > | behaviour of "/" configurable in Python 2 (so your hypothetical > | example isn't hypothetical at all - it's a description the -Q option), > | and -R makes random hashing configurable in 2.7 and 3.2: it means we > | can change the default behaviour in a future version (perhaps Python > | 4) while allowing people to easily check if their code operates > | correctly in that state in the current version. > | > | I think the default behaviour needs to be configurable from the > | environment and the command line, but I don't believe it should be > | configurable from within the interpreter. > > Hmm. This I can live with more happily, though I'm still uneasy. > > As an aside, I tend to feel that if something is tuneable it should be > exposed within the interpreter. Maybe only in an exciting new module > called shoot_self_in_foot or some similarly alarming name... I had missed this detail. I agree that it should be exposed in the interpreter. To my mind it is more like PYTHONPATH (which corresponds roughly to sys.path manipulations) than like -R (which changes something that should never be changed again, otherwise the sanity of the interpreter be at risk). It would seem hard to unittest the feature if it cannot be changed from within. But I can also think of other use cases for changing it from within (e.g. a script that decides on how to set it using a computation based on its arguments). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Touch up exception messaging
On Sun, Jan 27, 2013 at 3:25 AM, Ezio Melotti wrote:
> Hi,
> I'm not sure these changes are an improvement.
>
> On Fri, Jan 25, 2013 at 8:49 PM, brett.cannon
> wrote:
>> http://hg.python.org/cpython/rev/792810303239
>> changeset: 81735:792810303239
>> user:Brett Cannon
>> date:Fri Jan 25 13:49:19 2013 -0500
>> summary:
>> Touch up exception messaging
>> [...]
>> magic = data[:4]
>> raw_timestamp = data[4:8]
>> raw_size = data[8:12]
>> if magic != _MAGIC_BYTES:
>> -msg = 'bad magic number in {!r}: {!r}'.format(name, magic)
>> +msg = 'incomplete magic number in {!r}: {!r}'.format(name, magic)
>
> Here 2 things could go wrong:
> 1) magic is less than 4 bytes (so it could be "incomplete");
> 2) magic is 4 bytes, but it's different (so it's "invalid" or "bad");
> For this to be "incomplete" the size of the whole file should be less
> than 4 bytes, and it's unlikely that in a 3bytes-long file the content
> is an incomplete magic number. It's much more likely than it's not a
> magic number at all, or that it is a bad/wrong/invalid 4-bytes magic
> number, so the previous error looked better to me.
Oops, I missed that - this one wasn't supposed to change. It was only
the other two that were misleading.
>
>> raise ImportError(msg, **exc_details)
>> elif len(raw_timestamp) != 4:
>> -message = 'bad timestamp in {!r}'.format(name)
>> +message = 'incomplete timestamp in {!r}'.format(name)
>> _verbose_message(message)
>> raise EOFError(message)
>> elif len(raw_size) != 4:
>> -message = 'bad size in {!r}'.format(name)
>> +message = 'incomplete size in {!r}'.format(name)
>
> If we arrived here the magic number was right, so this are probably
> the timestamp and size, but for this to fail the whole file should be
> less than 8 or 12 bytes (unless I misread the code).
> Something like "reached EOF while reading timestamp/size" would
> probably be more informative.
I don't mind one way or the other here. "bad" was wrong, since it
implied this code actually checked these values against the expected
ones, but it doesn't, it merely requires that they exist. "incomplete
" or "reached EOF while reading " are both correct and a
sufficient hint for anyone already familiar with how the bytecode
cache format works.
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (3.3): Fix a few typos and a double semicolon. Patch by Eitan Adler.
On Sun, Jan 27, 2013 at 2:21 PM, ezio.melotti wrote: > http://hg.python.org/cpython/rev/07488c3c85f1 > changeset: 81772:07488c3c85f1 > branch: 3.3 > parent: 81770:a0756411c527 > user:Ezio Melotti > date:Sun Jan 27 06:20:14 2013 +0200 > summary: > Fix a few typos and a double semicolon. Patch by Eitan Adler. Misc/ACKS? Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 32 bit to 64 bit migration of C-API(Python)
Hi, I am working in Python-C ApI (Integrated Project). The Project has migrated to 64 bit machine from 32 bit machine. Every thing was working fine in 32 bit machine. But we getting following error in 64 bit machine. " Python int too large to convert to C long". When I start debug the code and found that we are using long variable which has size of 8 bit in C code. If I replace long to int32_t variable in C and all long value become garbage and I got index out of error from Python.The reason to become garbage value of long(after int conversion),we are using lot of Place Python api which give the long result and we use in further process like ( write/reading file).I have tried to replace with int but it did not work. I am not sure really this is only reason but it may be one of the reason to get the above error.I have also done google search but not able to get exact reason. After spending 3 days,I am not able to figure out the solution of this issue. Does anyone provide some insight. we are using Python 2.7.2 and Gcc : 4.7.0 Please let me know if I need to provide more info. Your earlier response will help us lot as we need to fix this issue as soon as possible.( it is kind of blockage for us) Thanks Abhishek Goswami Bangalore Phone No -07829580867 Skype : abhishekgoswami1 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 32 bit to 64 bit migration of C-API(Python)
On Sun, Jan 27, 2013 at 3:42 PM, abhishek goswami wrote: > Hi, > I am working in Python-C ApI (Integrated Project). The Project has migrated > to 64 bit machine from 32 bit machine. Every thing was working fine in 32 > bit machine. > But we getting following error in 64 bit machine. > " Python int too large to convert to C long". Hi Abhishek, python-dev is for the development *of* CPython, rather than development *with* CPython (including the C API). More appropriate places for this question are the general python-list mailing list or the C API special interest group. However, from the sounds of it, you are running into trouble with the fact that on a 64-bit machine, Python stores many internal values as "Py_ssize_t" (which is always 8 bytes on a 64 bit machine), rather than a C int or long. PEP 353 [1] describes the background for this change, which allows Python containers to exploit the full size of the address space on 64-bit systems. C API functions that may be of interest in that context include: http://docs.python.org/2/c-api/number.html#PyNumber_AsSsize_t http://docs.python.org/2/c-api/int.html#PyInt_FromSsize_t http://docs.python.org/2/c-api/int.html#PyInt_AsSsize_t http://docs.python.org/2/c-api/long.html#PyLong_FromSsize_t http://docs.python.org/2/c-api/long.html#PyLong_AsSsize_t As well as the "n" format character for argument processing http://docs.python.org/2/c-api/arg.html#parsing-arguments-and-building-values [1] http://www.python.org/dev/peps/pep-0353/ Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
