[issue13148] simple bug in mmap size check

[Charles-François Natali]

Charles-François Natali  added the comment:

> The condition contradicts the exception text:

Why?
The offset is zero-based, so 0 <= offset < size is a valid check.

> First of all, it doesn't fail (at least on Linux), I tested it before 
> posting.

Hmmm.
You got lucky, since the offset must be a multiple of the page size.

> tried on newer Linux - crashes with my patch.

That's exactly why we perform such checks. Here's what POSIX says:
"""
[EINVAL]
The addr argument (if MAP_FIXED was specified) or off is not a multiple of the 
page size as returned by sysconf(), or are considered invalid by the 
implementation.
[ENXIO]
Addresses in the range [off, off + len) are invalid for the object specified by 
fildes.
"""

Since we usually want to avoid implementation-defined behavior (and crashes), I 
think it's better to stick with the current checks (note that issue #12556 
concerned a really corner case: /proc entries supporting mmap but reporting a 
zero-length).

> Therefore, I'm no longer pushing for this change, I will need another 
> workaround anyway.

Alright, closing then.

--
resolution:  -> rejected
stage:  -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1673007] urllib2 requests history + HEAD support

[Patrick Westerhoff]

Patrick Westerhoff  added the comment:

Senthil, I highly disagree with what you said:

> The next problem comes when a user has specified both data and method="GET".
> This becomes an invalid scenario, but a decision has been to taken as what
> should be given preference?
That is incorrect, RFC2616 states that the server should ignore the message 
body when the request method does not define any semantics for it, but there is 
nothing that makes the inclusion of a message body along with the GET request 
method invalid.

> - As the user has sent "data", should the request be considered a POST?
No, absolutely not. Sending data via other request methods, like DELETE or PUT, 
is semantically correct and should be supported completely if we are going to 
include a way to set the request method. If I set the method to PUT, and 
include data, I don’t want the library to override that to POST just because I 
set data.

> - But user specified it as "GET" (intentionally or by mistake), so should the
> data not be used and Request should do only a GET?
If I data is included and the request method is explicitely set to GET, make a 
GET request and include the data in the request body. It might not be a 
semantically good decision to send data over GET, but it still is not 
disallowed and as such should be possible (for whatever reasons).

> - Or should we throw an error?
We especially should’t throw an error, as this is not invalid at all.

> A person would just send data and forget about changing the method to "POST".
I agree that the library should still default to POST if data is included and 
the request method was not explicitely set (see also below).
> 1) should method always have priority or should 'POST' always be used whenever
>data is passed?
> If data is passed use POST.
No, if data is passed and no special method is set, use POST, otherwise use the 
method the user specified, because that is what he expects.

> 2) if the method is e.g. 'GET' and data is passed, should an error be raised?
> Nope, give data the priority and do POST. (As urllib is currently doing)
Don't give data any priority if the method was set explicitely. Otherwise the 
ability to set a custom method is totally useless, especially with request 
methods where a message body is semantically useful (i.e. all other than HEAD 
and GET).

> 3) should Request.method be private?
> Not necessarily, it should  be public.
Depends on the way the method will be set. Looking at the way, the other 
request parameters are set (especially with the accessors being “deprecated”), 
it makes sense to leave this public.

> 4) should the value of Request.method be initialized in the __init__ or can it
>also be None?
> My take - It should be initialized to default (GET), instead of None.
Initializing it to GET will make the implementation difficult, especially if we 
want to continue supporting the behaviour of setting the method to POST when 
data is changed (and the method was not explicitely set). Unless we override 
the built-in property accessors I don’t think this is possible.

> 5) if the value of Request.method is always initialized, should we deprecate
>get_method?
> This is an interesting question. get_method essentially becomes less useful or
> it could serve as an arbiter when data and GET is sent and may be used as
> reliable way to get the Request's method. It should not be deprecated.
To my understanding, and this is also why I provided the same patch on the 
duplicate bug, `get_method` is used by the other libraries to get the used 
request method. Unless we change the other libraries to determine the method in 
a different way, deprecating `get_method` won’t get us anywhere.

What I tried to respect with the patch, and I think this was also Denver’s 
intention, is to add this functionality without breaking the current behaviour. 
That behaviour is that GET is default, and POST is set as default if there is 
any data. The functionality requires that the request method can be set (and 
the default behaviour can be overriden) without looking at the data (as 
explained above).

Ideally I would probably like to see the functionality of `get_method` done in 
another library, which performs the request. I.e. check `request.method` and 
use that if it’s set, otherwise check `request.data` and choose either POST or 
GET. But again this would require far too many changes in other libraries for 
this simple change.

And again on the `data` property: I think the name “data” is a bit confusing. 
Request does not provide any encoding details on that “data”, and it is 
actually just the request body in its original form. What I usually do in my 
subclass of Request is to provide a way to encode the data I pass to the 
constructor (often even with multipart encoding for file streams), while the 
`request.data` attribute to me still means “request body”.


Regarding those questions on the implementation, I agree with Ezio, and as I 
said this is 

[issue13148] simple bug in mmap size check

[Maxim Yanchenko]

Maxim Yanchenko  added the comment:

> You got lucky, since the offset must be a multiple of the page size.
That's why our header is exactly the page size :)

> Here's what POSIX says
Then it's just another discrepancy between POSIX and Linux, as I received 
ENOMEM instead of EINVAL (RHEL6 on 2.6.32).


Regarding the contradiction, it's probably still worth changing the exception 
message to "mmap offset is greater than _or equal to_ file size", to match the 
condition. Just 'greater than' means the '>' check, not the '>=' check from the 
code, mathematically.

--
resolution: rejected -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13150] Most of Python's startup time is sysconfig

[Nadeem Vawda]

Changes by Nadeem Vawda :


--
nosy: +nadeem.vawda

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6715] xz compressor support

[Nadeem Vawda]

Nadeem Vawda  added the comment:

Awesome stuff! I'll post an updated patch during the course of the day.

Martin: I've been having problems with Rietveld lately, so I'm posting
my replies to your comments here instead.

>> Modules/_lzmamodule.c:115: return _PyBytes_Resize(buf, size + BIGCHUNK);
> This has quadratic performance.

Correct. I copied the algorithm from _io.FileIO, under the assumption
that there was a reason for not using a simpler O(n log n) doubling
strategy. Do you know of any reason for this? Or is it safe to ignore it?

>> Modules/_lzmamodule.c:364: Py_BEGIN_ALLOW_THREADS
> It seems that the Windows version at least is not thread-safe. If so, you
> would need an LZMA lock when releasing the GIL.

Does the class need to be thread-safe, though? ISTM that there isn't any
sensible use case for having two threads feeding data through the same
compressor concurrently.

(If we *do* want thread-safety, then it doesn't matter whether the
underlying lib is internally thread-safe or not. We would still need to
guard against the possibility of the _lzmamodule.c code in one thread
modifying the lzma_stream's input or output pointer while lzma_code is
operating on the stream's data in another thread.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1673007] urllib2 requests history + HEAD support

[Senthil Kumaran]

Senthil Kumaran  added the comment:

Patrick, 

Lots of valid points. I had not looked at the RFC spec when I mentioned about 
data over request (GET) method, but was trying to derive the current 
functionality of module (so that users can have a seamless experience) with 
additional method="GET" as default.  Note, my intention was to be explicit when 
we give method arg.

But yeah, when user has specified the methods (PUT/DELETE etc) and given the 
data, correct rules should apply on how that method should deal with data.

As you pointed out to RFC, I realize RFC clearly points out the data 
(message-body) should be ignored and method should be given preference, 
whenever specification of method has nothing to with data.  I should take back 
my argument on giving data as preference even over GET.

Now, question arises- Can we in anyway default the method="GET" and maintain 
compatibility as well consistency with user expectations? At the moment, just 
by sending data over Request, the method is assumed to be POST. If that is not 
possible, then the way, current patch does seems to be a good way to acheive 
the purpose.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13151] pysetup3 run bdist_wininst fails

[Vinay Sajip]

New submission from Vinay Sajip :

When you run

pysetup3 run bdist_wininst

in a project directory with a valid setup.cfg, it fails with

error: Invalid command install

after the build, build_py and build_scripts steps.

For info, pysetup3 run bdist_dumb runs without error.

--
assignee: tarek
components: Distutils2, Library (Lib)
messages: 145339
nosy: alexis, eric.araujo, tarek, vinay.sajip
priority: normal
severity: normal
status: open
title: pysetup3 run bdist_wininst fails
versions: Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6715] xz compressor support

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> >> Modules/_lzmamodule.c:115: return _PyBytes_Resize(buf, size + BIGCHUNK);
> > This has quadratic performance.
> 
> Correct. I copied the algorithm from _io.FileIO, under the assumption
> that there was a reason for not using a simpler O(n log n) doubling
> strategy. Do you know of any reason for this? Or is it safe to ignore it?

I don't know, but I'd say it's safe to ignore it.
Maybe open a separate bug about that.
Generally we use a less-than-doubling strategy, to conserve memory (see
e.g. bytearray.c).

(note: it is O(n), not O(n log n))

> >> Modules/_lzmamodule.c:364: Py_BEGIN_ALLOW_THREADS
> > It seems that the Windows version at least is not thread-safe. If so, you
> > would need an LZMA lock when releasing the GIL.
> 
> Does the class need to be thread-safe, though? ISTM that there isn't any
> sensible use case for having two threads feeding data through the same
> compressor concurrently.

We certainly want to avoid crashes so, if that's what one risks from
using lzma from several threads, we should protect against it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13152] textwrap: support custom tabsize

[John Feuerstein]

New submission from John Feuerstein :

The textwrap module calls .expandtabs() to expand tabs to spaces.

This patch adds support for a custom tabsize, so that .expandtabs(tabsize) is 
called instead.

Includes test case.

--
components: Library (Lib)
files: textwrap_tabsize.diff
keywords: patch
messages: 145341
nosy: jfeuerstein
priority: normal
severity: normal
status: open
title: textwrap: support custom tabsize
type: feature request
versions: Python 3.3
Added file: http://bugs.python.org/file23376/textwrap_tabsize.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13152] textwrap: support custom tabsize

[Ezio Melotti]

Changes by Ezio Melotti :


--
keywords: +needs review
nosy: +georg.brandl
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13150] Most of Python's startup time is sysconfig

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Actually, a big part of that is compiling some regexes in the tokenize module. 
Just relying on the re module's internal caching shaves off 20% of total 
startup time.

Before:

$ time ./python -S -c 'import tokenize'

real0m0.034s
user0m0.030s
sys 0m0.003s
$ time ./python -c ''

real0m0.055s
user0m0.050s
sys 0m0.005s

After:

$ time ./python -S -c 'import tokenize'

real0m0.021s
user0m0.019s
sys 0m0.001s
$ time ./python -c ''

real0m0.044s
user0m0.038s
sys 0m0.006s

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10976] json.loads() throws TypeError on bytes object

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

I'll just mention that the elimination of bytes handling is a bit unfortunate, 
since this idiom which works in Python 2 no longer works:

fp = urlopen(url)
json_data = json.load(fp)

/me sad

--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13150] Most of Python's startup time is sysconfig

[Roundup Robot]

Roundup Robot  added the comment:

New changeset df950158dc33 by Antoine Pitrou in branch 'default':
Issue #13150: The tokenize module doesn't compile large regular expressions at 
startup anymore.
http://hg.python.org/cpython/rev/df950158dc33

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10976] json.loads() throws TypeError on bytes object

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> I'll just mention that the elimination of bytes handling is a bit
> unfortunate, since this idiom which works in Python 2 no longer works:
> 
> fp = urlopen(url)
> json_data = json.load(fp)

What if the returned JSON uses a charset other than utf-8 ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13150] Most of Python's startup time is sysconfig

[Tarek Ziadé]

Tarek Ziadé  added the comment:

I am curious: wouldn't be a way of keeping the compiled expressions in a static 
cache somewhere, so we would compile them just once and have both import time 
and runtime fast ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13150] Most of Python's startup time is sysconfig

[Roundup Robot]

Roundup Robot  added the comment:

New changeset ed0bc92fed68 by Antoine Pitrou in branch 'default':
Use a dict for faster sysconfig startup (issue #13150)
http://hg.python.org/cpython/rev/ed0bc92fed68

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13150] Most of Python's startup time is sysconfig

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> I am curious: wouldn't be a way of keeping the compiled expressions in
> a static cache somewhere, so we would compile them just once and have
> both import time and runtime fast ?

Runtime shouldn't be affected. The re module has its own LRU caching.

That said, it seems regular expressions are pickleable:

b'\x80\x03cre\n_compile\nq\x00X\x00\x00\x00\x00q\x01K \x86q\x02Rq\x03.'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13150] Most of Python's startup time is sysconfig

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Arg damn roundup e-mail gateway.
I wanted to paste:

>>> pickle.dumps(re.compile(''))
b'\x80\x03cre\n_compile\nq\x00X\x00\x00\x00\x00q\x01K \x86q\x02Rq\x03.'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13150] Most of Python's startup time is sysconfig

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Pre-parsing and building a cached module of built-time variables (from Makefile 
and pyconfig.h) under POSIX also removes more than 15% of startup time. Patch 
attached.

--
keywords: +patch
Added file: http://bugs.python.org/file23377/sysconfigdata.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13145] Documentation of PyNumber_ToBase() wrong

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 61eb59516b55 by Mark Dickinson in branch '3.2':
Issue #13145. Fix incorrect documentation for PyNumber_ToBase.  Thanks Sven 
Marnach.
http://hg.python.org/cpython/rev/61eb59516b55

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13145] Documentation of PyNumber_ToBase() wrong

[Roundup Robot]

Roundup Robot  added the comment:

New changeset a644e2c43c4b by Mark Dickinson in branch 'default':
Merge issue #13145 fix.
http://hg.python.org/cpython/rev/a644e2c43c4b

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13145] Documentation of PyNumber_ToBase() wrong

[Mark Dickinson]

Mark Dickinson  added the comment:

Fixed.  Thanks for the report!

--
nosy: +mark.dickinson
resolution:  -> fixed
status: open -> closed
versions: +Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13145] Documentation of PyNumber_ToBase() wrong

[Mark Dickinson]

Changes by Mark Dickinson :


--
assignee: docs@python -> mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10517] test_concurrent_futures crashes with "--with-pydebug" on RHEL5 with "Fatal Python error: Invalid thread state for this thread"

[Charles-François Natali]

Charles-François Natali  added the comment:

I did a quick test (calling fork() from a subinterpreter), and as
expected, I couldn't reproduce the problem.
So I still favor an OOM condition making pthread_setspecific bail out
with ENOMEM, othe other option being a nasty libc bug.
If the problem persists, please open a new issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10881] test_site and macframework builds fails

[Bryce Verdier]

Bryce Verdier  added the comment:

I would like to help by reviewing the patch, however I'm unsure about how to do 
the framework aspect of this. Can someone send me instructions or a link on how 
I would set this up to test?

Thanks in advance,

--
nosy: +louiscipher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10881] test_site and macframework builds fails

[Vinay Sajip]

Vinay Sajip  added the comment:

IIRC what I did was clone the repo, configure and build, then

sudo make frameworkinstall

For me (on Leopard), this installed Python 3.3 in 
/Library/Frameworks/Python.framework, leaving my system Python (2.5.1) alone. 
Then you can just invoke the regression tests using

python3.3 Lib/test/regrtest.py test_site

from the appropriate directory. But note that this is a case where the test 
needs fixing, not the code being tested.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13134] speed up finding of one-character strings

[Roundup Robot]

Roundup Robot  added the comment:

New changeset af4a89f4f466 by Antoine Pitrou in branch 'default':
Issue #13134: optimize finding single-character strings using memchr
http://hg.python.org/cpython/rev/af4a89f4f466

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13150] Most of Python's startup time is sysconfig

[Ross Lagerwall]

Ross Lagerwall  added the comment:

#11454 is another case where pre-parsing and pickling the regular expressions 
in the email module may improve import time considerably.

--
nosy: +rosslagerwall

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13134] speed up finding of one-character strings

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I went ahead and committed.

--
resolution:  -> fixed
stage:  -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13136] speed-up conversion between unicode widths

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 5b077c962a16 by Antoine Pitrou in branch 'default':
Issue #13136: speed up conversion between different character widths.
http://hg.python.org/cpython/rev/5b077c962a16

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13136] speed-up conversion between unicode widths

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Patch committed. It is of course still not as fast as memcpy, but it's a small 
step towards improving performance.

--
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13146] Writing a pyc file is not atomic

[Antoine Pitrou]

Antoine Pitrou  added the comment:

This new patch also fixes importlib.

--
stage:  -> patch review
versions:  -Python 2.7
Added file: http://bugs.python.org/file23378/importrename2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13153] IDLE crash with unicode bigger than 0xFFFF

[João Bernardo]

New submission from João Bernardo :

I was playing with some unicode chars on Python 3.2 (x64 on Windows 7), but 
when pasted a char bigger than 0x, IDLE crashes without any error message.

Example (works fine):
>>> '\U000104a2'
'𐒢'

But, if I try to paste the above char, the window will instantly close.

The interpreter uses 2-bytes per char (UTF-16) and I don't know if that's 
causing the problem (as side note, why don't the default Windows build uses 
4-bytes char?).

I can't check now with my Ubuntu install (UTF-32) if the problem persists.

--
components: IDLE, Unicode, Windows
messages: 145363
nosy: JBernardo
priority: normal
severity: normal
status: open
title: IDLE crash with unicode bigger than 0x
versions: Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10881] test_site and macframework builds fails

[Ned Deily]

Ned Deily  added the comment:

>   sudo make frameworkinstall

While that shouldn't affect the Apple-supplied system Pythons, be aware that 
with default ./configure settings this will overwrite any third-party Python 
installs (like from the python.org installers) of the version in question in 
/Library/Frameworks.  For unreleased versions, like from default that shouldn't 
be a big issue.  But it is safer and easy to pick some other location to 
install the framework.  I also usually use a different framework name to reduce 
the chances of accidentally dynamically linking to the wrong framework.  On a 
current 10.6 or 10.7 system, I'd recommend something like this:

./configure --enable-universalsdk=/Developer/SDKs/MacOSX10.6.sdk 
--with-universal-archs=intel '--prefix=/path/to/test/root' 
'--enable-framework=/path/to/test/Library' --with-framework-name=pytest_10_6 
MACOSX_DEPLOYMENT_TARGET=10.6 CC=/usr/bin/gcc-4.2

On 10.4 or 10.5 systems, to simulate the 32-bit-only installer build:

./configure --enable-universalsdk=/Developer/SDKs/MacOSX10.4u.sdk 
--with-universal-archs=32-bit '--prefix=/path/to/test/root' 
'--enable-framework=/path/to/test/Library' --with-framework-name=pytest_10_3 
--with-pydebug MACOSX_DEPLOYMENT_TARGET=10.3 CC=/usr/bin/gcc-4.0

Bryce, are you interested in writing a patch for the test?

--
assignee: ronaldoussoren -> ned.deily
nosy: +ned.deily
stage: patch review -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10881] test_site and macframework builds fails

[Ned Deily]

Ned Deily  added the comment:

I forgot to mention that one advantage of using another framework and root 
location, say, under your home directory somewhere, is that it is unnecessary 
to use sudo.  And you don't need to use the special target in any case, simply 
"make install".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13153] IDLE crash with unicode bigger than 0xFFFF

[Ned Deily]

Ned Deily  added the comment:

This is related to Issue12342.  The problem is that Tcl/Tk 8.5 (and earlier) do 
not support Unicode code points outside the BMP range as in this example.  So 
IDLE will be unable to display such characters but it should not crash either.

--
assignee:  -> ned.deily
nosy: +ned.deily
stage:  -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13134] speed up finding of one-character strings

[STINNER Victor]

STINNER Victor  added the comment:

changeset:   72874:bfd3fcfb02f3
user:Victor Stinner 
date:Tue Oct 11 23:22:22 2011 +0200
files:   Objects/stringlib/asciilib.h Objects/stringlib/fastsearch.h 
Objects/stringlib/stringdefs.h Objects/stringlib/ucs1lib.h Objects/stri
description:
Fix fastsearch for UCS2 and UCS4

 * If needle is 0, try (p[0] >> 16) & 0xff for UCS4
 * Disable fastsearch_memchr_1char() if needle is zero for UCS2 and UCS4

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13146] Writing a pyc file is not atomic

[STINNER Victor]

STINNER Victor  added the comment:

So if a process replaces the PYC file whereas another is reading the PYC, the 
reader may read corrupted data? The ideal fix is maybe to use a file lock?

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13153] IDLE crash with unicode bigger than 0xFFFF

[João Bernardo]

João Bernardo  added the comment:

@Ned

That looks like a bit different case. IDLE *can* print the char after you 
entered the '\U' version of it.

It doesn't accept you to paste those caracters...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13154] pep-0000.txt doesn't build anymore

[Antoine Pitrou]

New submission from Antoine Pitrou :

$ make
make: Dépendance circulaire pep-.txt <- pep-.txt abandonnée.
python genpepindex.py .
Traceback (most recent call last):
  File "genpepindex.py", line 67, in 
main(sys.argv)
  File "genpepindex.py", line 64, in main
write_pep0(peps, pep0_file)
  File "/home/antoine/cpython/peps/pep0/output.py", line 195, in write_pep0
authors_dict = verify_email_addresses(peps)
  File "/home/antoine/cpython/peps/pep0/output.py", line 104, in 
verify_email_addresses
"listed:\n" + '\n'.join(err_output))
ValueError: some authors have more than one email address listed:
Gregory Ewing: [u'g...@cosc.canterbury.ac.nz', 
u'greg.ew...@canterbury.ac.nz']
make: *** [pep-.txt] Erreur 1

--
assignee: docs@python
components: Documentation
messages: 145370
nosy: docs@python, pitrou
priority: normal
severity: normal
status: open
title: pep-.txt doesn't build anymore
type: compile error

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13146] Writing a pyc file is not atomic

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> So if a process replaces the PYC file whereas another is reading the
> PYC, the reader may read corrupted data?

No, this is the whole point of the patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13155] Optimize finding the max character width

[Antoine Pitrou]

New submission from Antoine Pitrou :

This patch optimizes scanning for the max character width in an unicode buffer.

Micro-benchmarking some worst case situations:

$ ./python -m timeit -s "x='é'+'x'*10" "x[1:]"
-> before:
1 loops, best of 3: 74.9 usec per loop
-> after:
10 loops, best of 3: 15.5 usec per loop

$ ./python -m timeit -s "x='\U0001'+'x'*10" "x[1:]"
-> before:
1 loops, best of 3: 138 usec per loop
-> after:
1 loops, best of 3: 71.3 usec per loop

--
components: Interpreter Core
files: find_max_char.patch
keywords: patch
messages: 145372
nosy: pitrou
priority: normal
severity: normal
status: open
title: Optimize finding the max character width
type: performance
versions: Python 3.3
Added file: http://bugs.python.org/file23379/find_max_char.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13155] Optimize finding the max character width

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I hadn't noticed the STRINGLIB_SIZEOF_CHAR constant. Reuse it instead of adding 
STRINGLIB_CHAR_SIZE.

--
Added file: http://bugs.python.org/file23380/find_max_char2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13155] Optimize finding the max character width

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Slightly cleaned up patch after Victor's comments in private.

--
Added file: http://bugs.python.org/file23381/find_max_char3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13155] Optimize finding the max character width

[STINNER Victor]

STINNER Victor  added the comment:

find_max_char() returns 0x1 instead of 0x10, which may be wrong (or at 
least, surprising). You may add a max_char variable using other macros like 
MAX_CHAR_ASCII, MAX_CHAR_UCS1, ..., which will be set at the same time than 
mask. Or restore your if (ret >= 0x1) return 0x10; else return ret;.

Constants look inconsistent:

+const STRINGLIB_CHAR *unrolled_end = begin + (n & ~ (Py_ssize_t) 7);
+p += 4;

You may use STRINGLIB_CHAR in:

+Py_UCS4 bits = p[0] | p[1] | p[2] | p[3];

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13155] Optimize finding the max character width

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Ok, updated patch.

--
Added file: http://bugs.python.org/file23382/find_max_char4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13155] Optimize finding the max character width

[STINNER Victor]

STINNER Victor  added the comment:

> Ok, updated patch.

"ret = ~mask + 1;" looks wrong: (~0xFF80+1) gives 128, not 127. I don't see 
why you need:

+if (ret < 128)
+return 127;
+if (ret < 256)
+return 255;

#undef ASCII_CHAR_MASK should be #undef UCS1_ASCII_CHAR_MASK


#error Invalid STRINGLIB_SIZEOF_CHAR (must be 1, 2 or 4)
should be
#error Invalid STRINGLIB_SIZEOF_CHAR (must be 2 or 4)

Why do you need these forward declarations? It's maybe related to another patch?

 static PyObject *
+unicode_fromascii(const unsigned char *s, Py_ssize_t size);
+static PyObject *
+_PyUnicode_FromUCS1(const unsigned char *s, Py_ssize_t size);
+static PyObject *
+_PyUnicode_FromUCS2(const Py_UCS2 *s, Py_ssize_t size);
+static PyObject *
+_PyUnicode_FromUCS4(const Py_UCS4 *s, Py_ssize_t size);

(You kept Py_UCS4 for "Py_UCS4 bits", but Py_UCS4 is maybe just fine.)

By the way, your function rocks :-) Use bit masks is a great idea, especially 
your "bits = p[0] | p[1] | p[2] | p[3]" "hack".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13155] Optimize finding the max character width

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> > Ok, updated patch.
> 
> "ret = ~mask + 1;" looks wrong: (~0xFF80+1) gives 128, not 127.

That's on purpose, since the mask has just matched. If 0xFF80
matches, then the max char can't be 127, it has to be at least 128.

>  I don't see why you need:
> 
> +if (ret < 128)
> +return 127;
> +if (ret < 256)
> +return 255;

Because otherwise you're returning 0x in the following line:

if (ret < 0x1)
return 0x;

> #undef ASCII_CHAR_MASK should be #undef UCS1_ASCII_CHAR_MASK

Ha, funny that gcc doesn't say anything.
I also forgot to #under MASK_*.

> #error Invalid STRINGLIB_SIZEOF_CHAR (must be 1, 2 or 4)
> should be
> #error Invalid STRINGLIB_SIZEOF_CHAR (must be 2 or 4)

Well, no. 1 is a legal value, even though it's handled elsewhere, so the
error message is right.

> Why do you need these forward declarations? It's maybe related to another 
> patch?

It's because I'm moving the stringlib #includes up in unicodeobject.c,
and stringlib calls these functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12641] Remove -mno-cygwin from distutils

[Santoso Wijaya]

Changes by Santoso Wijaya :


--
nosy: +santa4nt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12807] Optimization/refactoring for {bytearray, bytes, unicode}.strip()

[John O'Connor]

John O'Connor  added the comment:

New patch.

Please double check the removal of _PyUnicode_XStrip.

--
Added file: http://bugs.python.org/file23383/stringlib_strip2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12807] Move strip() to stringlib

[John O'Connor]

Changes by John O'Connor :


--
title: Optimization/refactoring for {bytearray, bytes, unicode}.strip() -> Move 
strip() to stringlib

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12555] PEP 3151 implementation

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 41a1de81ef2b by Antoine Pitrou in branch 'default':
PEP 3151 / issue #12555: reworking the OS and IO exception hierarchy.
http://hg.python.org/cpython/rev/41a1de81ef2b

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12555] PEP 3151 implementation

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Thanks everyone for the reviews!

--
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3902] Packages containing only extension modules have to contain __init__.py

[Mike Hoy]

Mike Hoy  added the comment:

setupscript did not have the changes that were made in apiref. v4 is now 
consistent and line wrapping at 80 chars.

--
Added file: http://bugs.python.org/file23384/apiref-setupscript-v4.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13156] _PyGILState_Reinit assumes auto thread state will always exist which is not true.

[Graham Dumpleton]

New submission from Graham Dumpleton :

This is a followup bug report to fix wrong implementation of 
_PyGILState_Reinit() introduced by http://bugs.python.org/issue10517.

I don't have a standalone test case yet. Problem occurs under mod_wsgi with 
Python 2.7.2 and thus similarly 3.2 where _PyGILState_Reinit() was also added.

The Python code part which triggers the problem is:

pid = os.fork()
if pid:
   sys.stderr.write('Fork succeeded (PID=%s)\n' % pid)
else:
   sys.stderr.write('Fork succeeded (child PID=%s)\n' % os.getpid())
   time.sleep(60.0)
   os._exit(0)

To trigger the problem requires this code be executed from a thread originally 
created outside of Python and then calling into a sub interpreter.

This thread would have created its own thread state object for the sub 
interpreter call since auto thread states don't work for sub interpreters. 
Further, it would never have called into the main interpreter so auto thread 
state simply doesn't exist for main interpreter.

When this thread has a fork() occur and _PyGILState_Reinit() is called, the 
call of PyGILState_GetThisThreadState() returns NULL because auto thread state 
for main interpreter was never initialised for this thread. When it then calls 
into PyThread_set_key_value() it is value of NULL and rather than set it, it 
thinks internally in find_key() you are doing a get which results in 
PyThread_set_key_value() returning -1 and so the fatal error.

So _PyGILState_Reinit() is broken because it assumes that an auto thread state 
will always exist for the thread for it to reinit, which will not always be the 
case.

The simple fix may be that if PyGILState_GetThisThreadState() returns NULL then 
don't do any reinit. Making that change does seem to fix the problem. Code that 
works then is:

void
_PyGILState_Reinit(void)
{
PyThreadState *tstate = PyGILState_GetThisThreadState();

if (tstate) {
PyThread_delete_key(autoTLSkey);
if ((autoTLSkey = PyThread_create_key()) == -1)
Py_FatalError("Could not allocate TLS entry");

/* re-associate the current thread state with the new key */
if (PyThread_set_key_value(autoTLSkey, (void *)tstate) < 0)
Py_FatalError("Couldn't create autoTLSkey mapping");
}
}

Diff file also attached.

--
components: Extension Modules
files: pystate.c.diff
keywords: patch
messages: 145383
nosy: grahamd, neologix
priority: normal
severity: normal
status: open
title: _PyGILState_Reinit assumes auto thread state will always exist which is 
not true.
type: crash
versions: Python 2.7, Python 3.2
Added file: http://bugs.python.org/file23385/pystate.c.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13156] _PyGILState_Reinit assumes auto thread state will always exist which is not true.

[Graham Dumpleton]

Graham Dumpleton  added the comment:

Whoops. Missed the error. The fatal error that occurs is:

Fatal Python error: Couldn't create autoTLSkey mapping

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13156] _PyGILState_Reinit assumes auto thread state will always exist which is not true.

[Charles-François Natali]

Changes by Charles-François Natali :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com