[issue12282] ABCMeta import error during build

[Ned Deily]

Ned Deily  added the comment:

It's being called for hg:

   -DHGVERSION="\"`LC_ALL=C hg id -i .`\"" ...

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12197] non-blocking SSL write in Windows sends large data but raises exception

[David Siroky]

David Siroky  added the comment:

Sorry, I attached wrong example version. It uses repeated sslsock.write() of 
the same buffer after catching SSL_ERROR_WANT_WRITE. It delivers the full block 
but this is a blocking operation.

I'm troubled with non-blocking writes. But as I dig deeper into the problem it 
looks like an inconsistency in OpenSSL (Linux vs. MSW). In Linux 
sslsock.write() always (as far as I can tell) sends some or all of the data and 
returns the amount. Like the plain socket.send(). In Windows it raises an 
exception if the data is larger then some uncertain size (mostly 16kB).

I'm working on snakeMQ asynchronous messaging library where the "packeter" 
layer relies on the sock.send() returning actual amount of sent bytes. This is 
not working in Windows if the link uses SSL.

Tested on WinXP, py2.6, py3.2.

--
Added file: http://bugs.python.org/file22280/ssl_sock_test_fix.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12283] python3.2 smtplib _quote_periods

[DDarko]

New submission from DDarko :

File "/usr/lib/python3.2/smtplib.py", line 166, in _quote_periods

def _quote_periods(bindata):
return re.sub(br'(?m)^\.', '..', bindata)

should be:
return re.sub(br'(?m)^\.', b'..', bindata)

--
components: Extension Modules
messages: 137899
nosy: DDarko
priority: normal
severity: normal
status: open
title: python3.2 smtplib _quote_periods
versions: Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12283] python3.2 smtplib _quote_periods

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
components: +Library (Lib) -Extension Modules
nosy: +r.david.murray
stage:  -> patch review
type:  -> behavior
versions: +Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11941] Support st_atim, st_mtim and st_ctim attributes in os.stat_result

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I'm not sure if that's deliberate, but the new attributes don't appear in the 
result repr():

>>> s = os.stat("LICENSE")
>>> s
posix.stat_result(st_mode=33204, st_ino=524885, st_dev=2053, st_nlink=1, 
st_uid=500, st_gid=500, st_size=14597, st_atime=1307474138, 
st_mtime=1299630588, st_ctime=1299630588)
>>> s.st_mtim
(1299630588, 90781883)

In the docs, you also need a "versionchanged" tag to mention that the 
attributes were added in 3.3.

The patch fails to compile under Windows: MSVC forbids variable declarations 
after code (atim, ctim, mtim), you have to put them at the beginning of a 
block. Once this is fixed, it seems to work ok.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11277] Crash with mmap and sparse files on Mac OS X

[Steffen Daode Nurpmeso]

Steffen Daode Nurpmeso  added the comment:

Ok, this patch could be used.
*Unless* the code is not protected by the GIL.

- Gestalt usage is a bit more complicated according to

http://www.cocoadev.com/index.pl?DeterminingOSVersion

  unless Python only supports OS X 10.4 and later.
  (And platform.py correctly states that in _mac_ver_gestalt(),
  but see below.)

- Due to usage of Gestalt, '-framework CoreServices' must be
  linked against mmapmodule.c.
  The Python configuration stuff is interesting for me, i managed
  compilation by adding the line

mmap mmapmodule.c -framework CoreServices

  to Modules/Setup, but i guess it's only OS X which is happy
  about that.

platform.py: _mac_ver_xml() should be dropped entirely according
to one of Ned Deily's links ("never officially supported"), and
_mac_ver_gestalt() obviously never executed because i guess it
would fail due to "versioninfo".  Unless i missed something.

By the way: where do you get the info from?  "sys1", "sys2",
"sys3"?  Cannot find it anywhere, only the long names, e.g.
gestaltSystemVersionXy.

Note that i've mailed Apple.  I did not pay 99$ or even 249$, so
i don't know if there will be a response.
--
Ciao, Steffen
sdaoden(*)(gmail.com)
() ascii ribbon campaign - against html e-mail
/\ www.asciiribbon.org - against proprietary attachments

--
Added file: http://bugs.python.org/file22281/11277.apple-fix-2.diff

___
Python tracker 

___diff --git a/Doc/library/mmap.rst b/Doc/library/mmap.rst
--- a/Doc/library/mmap.rst
+++ b/Doc/library/mmap.rst
@@ -88,7 +88,8 @@
 
To ensure validity of the created memory mapping the file specified
by the descriptor *fileno* is internally automatically synchronized
-   with physical backing store on Mac OS X and OpenVMS.
+   with physical backing store on operating systems where this is
+   necessary, e.g. OpenVMS, and some buggy versions of Mac OS X.
 
This example shows a simple way of using :class:`mmap`::
 
diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c
--- a/Modules/mmapmodule.c
+++ b/Modules/mmapmodule.c
@@ -25,6 +25,8 @@
 #define UNIX
 # ifdef __APPLE__
 #  include 
+
+#  include 
 # endif
 #endif
 
@@ -65,6 +67,39 @@
 #define my_getpagesize getpagesize
 #endif
 
+# ifdef __APPLE__
+static void
+apple_osx_needs_fullsync(long *use_fullsync)
+{
+/* Issue #11277: mmap(2) bug with >32 bit sparse files.
+ * Apple fixed the bug before announcement of OS X "Lion", but since we
+ * need to handle buggy versions, perform a once-only check to see if the
+ * running kernel requires the expensive sync.
+ * >0: F_FULLSYNC is required, <0: kernel has mmap(2) bug fixed */
+SInt32 ver;
+
+*use_fullsync = 1;
+if (Gestalt(gestaltSystemVersion, &ver) != noErr && ver >= 0x1040) {
+if (Gestalt(gestaltSystemVersionMajor, &ver) != noErr)
+goto jleave;
+if (ver > 10) {
+*use_fullsync = -1;
+goto jleave;
+}
+
+if (Gestalt(gestaltSystemVersionMinor, &ver) != noErr)
+goto jleave;
+if (ver >= 7) {
+*use_fullsync = -1;
+goto jleave;
+}
+}
+
+jleave:
+return;
+}
+# endif /* __APPLE__ */
+
 #endif /* UNIX */
 
 #include 
@@ -1128,8 +1163,14 @@
 #ifdef __APPLE__
 /* Issue #11277: fsync(2) is not enough on OS X - a special, OS X specific
fcntl(2) is necessary to force DISKSYNC and get around mmap(2) bug */
-if (fd != -1)
-(void)fcntl(fd, F_FULLFSYNC);
+if (fd != -1) {
+/* (GIL protected) */
+static long use_fullsync /*= 0*/;
+if (!use_fullsync)
+apple_osx_needs_fullsync(&use_fullsync);
+if (use_fullsync > 0)
+(void)fcntl(fd, F_FULLFSYNC);
+}
 #endif
 #ifdef HAVE_FSTAT
 #  ifdef __VMS
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11277] Crash with mmap and sparse files on Mac OS X

[Steffen Daode Nurpmeso]

Changes by Steffen Daode Nurpmeso :


Removed file: http://bugs.python.org/file22273/11277.apple-fix.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12281] bytes.decode('mbcs', 'ignore') does replace undecodable bytes on Windows Vista or later

[STINNER Victor]

STINNER Victor  added the comment:

mbcs.patch fixes PyUnicode_DecodeMBCS():
 - only use flags=0 if errors="replace" on Windows >= Vista or if 
errors="ignore" on Windows < Vista
 - support any error handler
 - support any code page (but the code page is hardcoded to CP_ACP)

My patch always tries to decode in strict mode. On decode error: it decodes 
byte per byte, and call unicode_decode_call_errorhandler() on error.

TODO:

 - don't use insize=1 (decode byte per byte): it doesn't work with multibyte 
encodings (like UTF-8)
 - use final in decode_mbcs_errors(): a multibyte character may be splitted 
between two chunks of INT_MAX bytes
 - fix all FIXME
 - patch also PyUnicode_EncodeMBCS()
 - implement suggested Martin's optimizations?
 - MB_ERR_INVALID_CHARS is not supported by some code pages (e.g. UTF-7 code 
page)

Is it necessary to write a NUL character at the end? ("*out = 0;")

It would be nice to support any code page, and maybe support more options (e.g. 
MB_COMPOSITE, MB_PRECOMPOSED, MB_USEGLYPHCHARS to decode).

It is possible to test different code pages by changing the hardcoded code_page 
value in PyUnicode_DecodeMBCS. Change your region in the control panel if you 
would like to change the Windows ANSI code page. You can also play with 
SetThreadLocale() and CP_THREAD_ACP to test the ANSI code page of the current 
thread.

--
keywords: +patch
Added file: http://bugs.python.org/file22282/mbcs.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12281] bytes.decode('mbcs', 'ignore') does replace undecodable bytes on Windows Vista or later

[STINNER Victor]

STINNER Victor  added the comment:

Example with ANSI=cp932 (on Windows Seven):
 - b'abc\xffdef'.decode('mbcs', 'replace') gives 'abc\uf8f3def'
 - b'abc\xffdef'.decode('mbcs', 'ignore') gives 'abcdef'

--
nosy: +ocean-city

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12281] bytes.decode('mbcs', 'ignore') does replace undecodable bytes on Windows Vista or later

[STINNER Victor]

STINNER Victor  added the comment:

> Example with ANSI=cp932 (on Windows Seven):
>  - b'abc\xffdef'.decode('mbcs', 'replace') gives 'abc\uf8f3def'
>  - b'abc\xffdef'.decode('mbcs', 'ignore') gives 'abcdef'

Oh, and b'\xff'.decode('mbcs', 'surrogateescape') gives '\udcff' as expected. 
At least for surrogateescape, it would be nice that mbcs supports any error 
handler on encoding.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12284] argparse.ArgumentParser: usage example option

[Jonas H.]

New submission from Jonas H. :

I'd like to see an `examples` option added to argparse.ArgumentParser as found 
in many man pages.

This could also be done using the `epilog` option, but that misses the 
"%(proc)s" replacement which makes usage examples like this

  Example usage:
./script.py option1 option2

impossible.

--
components: Library (Lib)
messages: 137905
nosy: jonash
priority: normal
severity: normal
status: open
title: argparse.ArgumentParser: usage example option
type: feature request
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11549] Build-out an AST optimizer, moving some functionality out of the peephole optimizer

[Nick Coghlan]

Nick Coghlan  added the comment:

As Raymond noted though, some of the block stack fiddling doesn't make sense 
until after the bytecode has already been generated. It's OK to have multiple 
optimisers at different layers, each taking care of the elements that are best 
suited to that level.

And yes, an updated patch against the current tip would be good. Of my earlier 
review comments, the ones I'd still like to see addressed are:
- finish clearing out the now redundant special casing of None/True/False
- separating out the non-AST related compiler tweaks (i.e. 3b and 3c and the 
associated test changes) into their own patch (including moving the relevant 
tests into a separate @cpython_only test case)

I'm still not 100% convinced on that latter set of changes, but I don't 
want my further pondering on those to hold up the rest of the patch. (they 
probably make sense, it's just that the AST level changes are much easier to 
review than the ones right down at the bytecode generation level - reviewing 
the latter means getting back up to speed on precisely how the block stack 
works and it will be a while before I get around to doing that. It's just one 
of those things where the details matter, but diving that deep into the 
compiler is such a rare occurrence that I have to give myself a refresher 
course each time it happens).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11277] Crash with mmap and sparse files on Mac OS X

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Steffen: _mac_ver_xml should not be dropped, it is a perfectly fine way to 
determine the system version.  Discussing it is also off-topic for this issue, 
please keep the discussion focussed.

Wrt. mailing Apple: I wouldn't expect and answer. Is there something specific 
you want to know? I'm currently at WWDC and might be able to ask the question 
at one of the labs (where Apple's engineers hang out).

If it is really necessary to check for the OS version to enable the 
OSX-specific bugfix it is possible to look at the uname information instead of 
using gestalt.  In particular something simular to this Python code:

   v = os.uname()[2]
   major = int(v.split('.')[0])
   if major <= 10:
  # We're on OSX 10.6 or earlier
  enableWorkaround()

This tests the kernel version instead of the system version, but until now the 
major version of the kernel has increased with every major release of the OS 
and I have no reason to suspect that Lion will be any different.

BTW2: OSX 10.7 is not released yet and should not be discussed in public fora, 
as you should know if you have legal access.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12285] Unexpected behavior for 0 or negative processes in multiprocessing.pool()

[Jorgen Skancke]

New submission from Jorgen Skancke :

A normal way to start a multiprocessing-pool is like this:

Multiprocessing.Pool(processes=some_number).

However, if 'some_number' is 0 or negative, Python hangs and must be killed. I 
would expect an error message of the type: "Number of processes must be at 
least 1".

Attaching a script that reproduces this.

I struggled a bit with this for a program that uses cpu_count() - 4 to 
calculate 'some_number'. Guess what happens when the number of cores is 4 :) 
Even though the fix is easy (if some_number < 1), it would be nice to be warned 
about it.

--
components: Library (Lib)
files: multitest_pool.py
messages: 137908
nosy: jorgsk
priority: normal
severity: normal
status: open
title: Unexpected behavior for 0 or negative processes in multiprocessing.pool()
versions: Python 2.6
Added file: http://bugs.python.org/file22283/multitest_pool.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12284] argparse.ArgumentParser: usage example option

[R. David Murray]

Changes by R. David Murray :


--
nosy: +bethard

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12282] ABCMeta import error during build

[R. David Murray]

R. David Murray  added the comment:

As far as I can tell my python2.7 installation is fine and bug free.  hg is 
working fine when called from the command line.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12279] Add build_distinfo command to packaging

[higery]

higery  added the comment:

Now, the workaround of my code is just setting the 'distinfo-dir' option with 
os.curdir value through calling a 'reinitialize_command(self, command, 
reinit_subcommands=False, **kw)' function , which is added in packaging's 
Command class for my own purpose, but I think it should be taken as a standard 
function in Command - it's useful and setuptools' Command class does have this 
function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12282] build process adds CWD (null entry) to PYTHONPATH if PYTHONPATH is set in the build environment

[R. David Murray]

R. David Murray  added the comment:

It turns out that I had a stray abc.py file in my current working directory as 
the result of some previous tests of module-load-order rules.  That by itself 
wouldn't have triggered the problem, but in addition, I have PYTHONPATH set in 
my environment because I have the hg whitespace hook installed.  Apparently the 
build process turns a non-empty PYTHONPATH into a null entry in the final 
computed PYTHONPATH it uses when hg is called, and this results in anything 
located in the CWD being loaded.

I don't think this rises quite to the level of a security issue, since it 
applies only at build time, but it certainly seems like a bug.

--
title: ABCMeta import error during build -> build process adds CWD (null entry) 
to PYTHONPATH if PYTHONPATH is set in the build environment
versions: +Python 2.7, Python 3.1, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9205] Parent process hanging in multiprocessing if children terminate unexpectedly

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 6d6099f7fe89 by Antoine Pitrou in branch 'default':
Issue #9205: concurrent.futures.ProcessPoolExecutor now detects killed
http://hg.python.org/cpython/rev/6d6099f7fe89

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9205] Parent process hanging in multiprocessing if children terminate unexpectedly

[Antoine Pitrou]

Antoine Pitrou  added the comment:

So, concurrent.futures is fixed now. Unless someone wants to patch 
multiprocessing.Pool, I am closing this issue.

--
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12021] mmap.read requires an argument

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Patch looks good to me, thank you :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12187] subprocess.wait() with a timeout uses polling on POSIX

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Why not use signalfd() when available?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12197] non-blocking SSL write in Windows sends large data but raises exception

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> Sorry, I attached wrong example version. It uses repeated
> sslsock.write() of the same buffer after catching
> SSL_ERROR_WANT_WRITE. It delivers the full block but this is a
> blocking operation.

In "normal" non-blocking code you would use select() (or an equivalent)
until the socket is ready for writing, and then send() some chunk of
data on it. Can't you use that approach?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12021] mmap.read requires an argument

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 964d0d65a2a9 by Charles-François Natali in branch 'default':
Issue #12021: Make mmap's read() method argument optional. Patch by Petri
http://hg.python.org/cpython/rev/964d0d65a2a9

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12286] float('nan') breaks sort() method on a list of floats

[Johann C. Rocholl]

New submission from Johann C. Rocholl :

>>> l = [1.0, 2.0, 3.0, float('nan'), 4.0, 3.0, 2.0, 1.0]
>>> l.sort()
>>> l
[1.0, 2.0, 3.0, nan, 1.0, 2.0, 3.0, 4.0]

The expected result is either of the following:
[nan, 1.0, 1.0, 2.0, 2.0, 3.0, 3.0, 4.0] (similar to None)
[1.0, 1.0, 2.0, 2.0, 3.0, 3.0, 4.0, nan] (similar to string 'nan')

--
messages: 137918
nosy: jcrocholl
priority: normal
severity: normal
status: open
title: float('nan') breaks sort() method on a list of floats
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12286] float('nan') breaks sort() method on a list of floats

[Ezio Melotti]

Ezio Melotti  added the comment:

This is expected.  See also #11986 and #11949.

--
nosy: +belopolsky, ezio.melotti, mark.dickinson, rhettinger
resolution:  -> invalid
stage:  -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12021] mmap.read requires an argument

[Charles-François Natali]

Charles-François Natali  added the comment:

Patch committed.
Thanks for the report and the patch!

--
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12286] float('nan') breaks sort() method on a list of floats

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

This is actually a duplicate of issue7915.

I don't think there is nothing we can do to improve the situation.  In fact 
discussion at #11949 ends with a +0 from Mark Dickinson to issue a warning 
whenever nans participate in order comparison. Discussion at #11986 ends 
without any clear consensus.  I think we should keep at least one issue on this 
topic open rather than close new issues as invalid by referring users to older 
closed issues that don't explain why silently producing nonsensical results is 
better than raising an error or issuing a warning.

--
resolution: invalid -> duplicate

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12197] non-blocking SSL write in Windows sends large data but raises exception

[Giampaolo Rodola']

Changes by Giampaolo Rodola' :


--
nosy: +giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12187] subprocess.wait() with a timeout uses polling on POSIX

[Giampaolo Rodola']

Changes by Giampaolo Rodola' :


--
nosy: +giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8407] expose signalfd(2) and pthread_sigmask in the signal module

[Giampaolo Rodola']

Changes by Giampaolo Rodola' :


--
nosy: +giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1195571] simple callback system for Py_FatalError

[Tom Whittock]

Changes by Tom Whittock :


--
nosy: +Tom.Whittock

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1195571] simple callback system for Py_FatalError

[Gregory P. Smith]

Gregory P. Smith  added the comment:

This makes sense, I'll add it to 3.3.

--
assignee:  -> gregory.p.smith
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12287] ossaudiodev: stack corruption with FD >= FD_SETSIZE

[Charles-François Natali]

New submission from Charles-François Natali :

ossaudiodev's writeall method doesn't check that the FD is less than FD_SETSIZE 
when passing it to FD_SET: since FD_SET typically doesn't do bound check, it 
will write to a random location in memory (in this case on the stack).
I've attached a test that triggers a segfault on my 32-bit Linux box:
- you must have an OSS-compatible device as /dev/dsp (if you don't you can use 
"modprobe snd_pcm_oss")
- it tries to increase RLIMIT_NOFILE since it's usually defined to be the same 
as FD_SETSIZE (1024 on Linux). The script must be run as root for that.
A patch is attached.
The only other place where I've seen a similar problem is in Module/readline.c: 
I'm not sure it's worth adding this check there :-)

--
components: Library (Lib)
files: oss_select.diff
keywords: needs review, patch
messages: 137923
nosy: haypo, neologix, pitrou
priority: normal
severity: normal
stage: patch review
status: open
title: ossaudiodev: stack corruption with FD >= FD_SETSIZE
type: crash
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3
Added file: http://bugs.python.org/file22284/oss_select.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12287] ossaudiodev: stack corruption with FD >= FD_SETSIZE

[Charles-François Natali]

Changes by Charles-François Natali :


Added file: http://bugs.python.org/file22285/test_oss.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12287] ossaudiodev: stack corruption with FD >= FD_SETSIZE

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> ossaudiodev's writeall method doesn't check that the FD is less than
> FD_SETSIZE when passing it to FD_SET: since FD_SET typically doesn't
> do bound check, it will write to a random location in memory (in this
> case on the stack).
> I've attached a test that triggers a segfault on my 32-bit Linux box:
> - you must have an OSS-compatible device as /dev/dsp (if you don't you
> can use "modprobe snd_pcm_oss")
> - it tries to increase RLIMIT_NOFILE since it's usually defined to be
> the same as FD_SETSIZE (1024 on Linux). The script must be run as root
> for that.
> A patch is attached.

Well, the test doesn't work here ("IOError: [Errno 16] Device or
resource busy: '/dev/dsp'", probably because of PulseAudio already using
it), but the patch looks simple enough.

By the way, this function still uses "y#" instead of "y*", this could be
the topic of another issue if you are interested.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12288] Python 2.7.1 tkSimpleDialog initialvalue

[Tom Middleton]

New submission from Tom Middleton :

Using Tkinter under Python 2.7.1 (Windows XP FWIF)
If using the tkSimpleDialog.askinteger() function with an initialvalue = 0, the 
0 is not displayed in the dialog box. The same is true for 
tkSimpleDialog.askfloat().

The cause of this seems to be the following:
in Lib\libi-tk\tkSimpleDialog.py (attached for convenience) 
under class _QueryDialog(Dialog)

def body(self, master):

w = Label(master, text=self.prompt, justify=LEFT)
w.grid(row=0, padx=5, sticky=W)

self.entry = Entry(master, name="entry")
self.entry.grid(row=1, padx=5, sticky=W+E)

if self.initialvalue:
self.entry.insert(0, self.initialvalue)
self.entry.select_range(0, END)

return self.entry

The if self.initialvalue will evaluate to a FALSE when the initial value is a 0.

Changing this line to:

if self.initialvalue is not None:

fixed this issue.
I am not certain that this is as intended or a bug.

--
components: Tkinter
files: tkSimpleDialog.py
messages: 137925
nosy: busfault
priority: normal
severity: normal
status: open
title: Python 2.7.1 tkSimpleDialog initialvalue
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file22286/tkSimpleDialog.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1195571] simple callback system for Py_FatalError

[STINNER Victor]

STINNER Victor  added the comment:

fatalhook-2.patch: I don't understand the documentation. It says "Cause 
:cfunc:`Py_FatalError` to invoke the given function instead of printing to 
standard error and aborting out of the process.", but if the callback does 
nothing, the message+traceback is printed.

If the "fatal hook" is supposed to replace the function displaying the message, 
you should move the code displaying message+traceback in a subfunction and use 
it as the default hook function.

Pseudo code:
---
static fatal_error(const char *msg)
{
   printf("Fatal error: %s\n", msg);
   ... print traceback ...
}

static PyFatalHook fatalhook_func = fatal_error;

void
Py_FatalError(const char *msg)
{
if (fatalhook_func != NULL)
   fatalhook_func(msg);
... windows debugger code ...
abort();
}
---

NULL is a special hook value: don't print message+traceback, but exit (call 
abort()).

The hook can exit the process using something else than abort(). But if the 
hook doesn't exit, the default exit code is executed (call abort()), but the 
"Fatal error..."+traceback is not printed.

> fatalhook_func != Py_FatalError

I think that this test is just overkill, or it should be moved to 
Py_SetFatalHook (e.g. set the hook to NULL if Py_FatalError is passed).

> I also made Py_SetFatalHook() return the previous hook; 
> it could be useful to set a function temporarily,
> even if this is not thread safe.

The previous hook can also be used to chain hooks. For example, if you would 
like to display the traceback but also do a special thing before exit, you can 
do something like:
--
void init()
{
   previous = Py_SetFatalHook(my_hook)
}

void my_hook(const char *msg)
{
  ... cleanup ...
  previous(msg);
}
--

About thread safety: because Py_FatalError() is called in the worst case (when 
something really bad happens), I think that it is better to not use something 
related to thread to avoid issues like deadlocks, and keep the code simple.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12287] ossaudiodev: stack corruption with FD >= FD_SETSIZE

[STINNER Victor]

STINNER Victor  added the comment:

You don't check that 0 <= fd (e.g. oss.close()).

The select has a specific code for Visual Studio (don't check v < FD_SETSIZE):

#if defined(_MSC_VER)
max = 0; /* not used for Win32 */
#else  /* !_MSC_VER */
if (v < 0 || v >= FD_SETSIZE) {
PyErr_SetString(PyExc_ValueError,
"filedescriptor out of range in select()");
goto finally;
}
if (v > max)
max = v;
#endif /* _MSC_VER */

Python has a _PyVerify_fd() function. We might write a similar function/macro 
to check if a file descriptor can be used in a file descriptor set. FD_SET() is 
used in the oss, readline, socket and _ssl modules. The socket module has a 
IS_SELECTABLE() macro:

#ifdef Py_SOCKET_FD_CAN_BE_GE_FD_SETSIZE
/* Platform can select file descriptors beyond FD_SETSIZE */
#define IS_SELECTABLE(s) 1
#elif defined(HAVE_POLL)
/* Instead of select(), we'll use poll() since poll() works on any fd. */
#define IS_SELECTABLE(s) 1
/* Can we call select() with this socket without a buffer overrun? */
#else
/* POSIX says selecting file descriptors beyond FD_SETSIZE
   has undefined behaviour.  If there's no timeout left, we don't have to
   call select, so it's a safe, little white lie. */
#define IS_SELECTABLE(s) ((s)->sock_fd < FD_SETSIZE || s->sock_timeout <= 0.0)
#endif

Note: do you really use the OSS module? On which OS? :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12187] subprocess.wait() with a timeout uses polling on POSIX

[Charles-François Natali]

Charles-François Natali  added the comment:

> For subprocess.wait(), we can do something with signals (SIGCHLD and/or 
> SIGCLD).

There's just one problem: SIGCHLD is ignored by default, which means that 
sigwait and friends won't return when a child exits.
Well, it actually works on recent Linux kernels, but POSIX makes no such 
guarantee, and it's at least known to fail on Solaris (see Dave Butenhof's 
comment):
http://www.multithreadedprogramming.info/sigwait-ing-for-sigchld

To be portable, we would need to set a handler for SIGCHLD, which has the 
following problems:
- you have to do that from the main thread
- it impacts every thread
- it will make syscalls fail with EINTR
- once you've changed SIGCHLD setting, you can't go back to the original 
semantic (setting it to SIG_IGN again will prevent children from becoming 
zombies, and waitpid wait until all children exited and will fail with ECHILD)

Note that even if it does work, there's a problem in multi-threaded programs, 
because the signal must be blocked by all the threads...

But since we use it with a timeout, we could also consider that this will work 
on systems that allow ignore signals to be catched by sigtimedwait, and it will 
wait the full timeout on other systems. I don't know if that's acceptable.

> Why not use signalfd() when available?

It suffers from the same issue, and it's Linux-specific (sigwait and friends 
are POSIX).

Note that exposing sigtimedwait is probably useful anyway, and I'd like to work 
on a patch.
Note that I'm not sure that exposing sigtimedwait is necessary (I don't think 
that the info field is going to be used by Python applications): how about just 
adding an optional timeout argument to signal_sigwait?

--
nosy: +neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12187] subprocess.wait() with a timeout uses polling on POSIX

[STINNER Victor]

STINNER Victor  added the comment:

> To be portable, we would need to ...

Antoine is right: we don't have to be portable. We can write an "optimized" 
implementations (without polling) for a specific OS, even for a specific 
version of an OS (e.g. like Linux kernel >= 2.6.22 for signalfd).

I like the idea of signalfd(), but I don't know exactly how it works. Should we 
block the signal? What happens when we unblock the signal? It would be nice if 
the signal handler is called on unblock, because it would not change the 
current behaviour. Is it possible to block a signal in all threads? 
pthread_sigmask() blocks signals in the current thread, the manual page of 
sigprocmask() has a funny comment: "The use of sigprocmask() is unspecified in 
a multithreaded process; see pthread_sigmask(3)."

Extract of signalfd() manual page: "Normally,  the  set of signals to be 
received via the file descriptor should be blocked using sigprocmask(2), to 
prevent the signals being handled according to their default dispositions."

Is SIGCHLD only raised once at child process exit? "SIGCLD would be delivered 
constantly (unless blocked) while any child is ready to be waited for." 
according to http://lwn.net/Articles/414618/

> There's just one problem: SIGCHLD is ignored by default,
> which means that sigwait and friends won't return when a child exits.

sigwait() is not impacted by the associated signal handler, but sigwait() only 
works if the signal is blocked (e.g. by pthread_sigmask):

"If no signal in set is pending at the time of the call, the thread is 
suspended until one or more becomes pending. The signals defined by set will 
been blocked at the time of the call to sigwait(); otherwise the behaviour is 
undefined."
http://pubs.opengroup.org/onlinepubs/007908799/xsh/sigwait.html

Example (for Python 3.3):
--
from signal import *
import subprocess

signum = SIGCHLD
process = subprocess.Popen("sleep 1", shell=True)
print("Wait %s..." % signum)
pthread_sigmask(SIG_BLOCK, [signum])
sigwait([signum])
pthread_sigmask(SIG_UNBLOCK, [signum])
print("done")
process.wait()
--

Same question than signalfd(): how can we block a signal in all threads 
(including C threads, e.g. _tkinter event looop thread)? Use sigprocmask()?

sigwait() removes the signal from the list of pending signals, so the signal 
handler will not be called.

> Note that exposing sigtimedwait is probably useful anyway,
> and I'd like to work on a patch.

See also issue #8407 for sigtimedwait() and signalfd() in Python.

---

sigprocmask(), sigwait() and signals in general seem to behave differently on 
each OS, so anyway, we cannot write a single portable implementation to solve 
this issue. If we cannot write a reliable non-polling implementation for an OS, 
you should use the polling implementation instead (which *is* reliable).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12289] http.server.CGIHTTPRequestHandler doesn't check if a Python script is executable

[STINNER Victor]

New submission from STINNER Victor :

CGIHTTPRequestHandler.run_cgi() only checks if the script processing the 
request is executable if the file is not a Python script, but later it uses 
os.execve(scriptfile, ...) if os has a fork() function.

Moreover, the executable() functions checks if os.stat(path).st_mode & 0o111 != 
0: this test is wrong if st_mode & 0o111 != 0o111. For example, if the script 
has mode 0700 and is not owned by the current user, executable() returns True, 
whereas it should be False. os.access(filename, os.X_OK) should be used instead.

I found these issues while trying to understand the following failure on 
"FreeBSD 7.2 x86 3.x" buildbot:

[320/356/2] test_httpservers
Traceback (most recent call last):
  File "/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/http/server.py", 
line 1123, in run_cgi
OSError: [Errno 13] Permission denied
(...)

I don't understand how it happens because test_httpservers uses 
os.chmod(script, 0o777).

--
components: Library (Lib)
messages: 137930
nosy: haypo
priority: normal
severity: normal
status: open
title: http.server.CGIHTTPRequestHandler doesn't check if a Python script is 
executable
versions: Python 2.7, Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11583] os.path.isdir() is slow on windows

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 88e318166eaf by Brian Curtin in branch '3.2':
Fix #11583. Changed os.path.isdir to use GetFileAttributes instead of os.stat.
http://hg.python.org/cpython/rev/88e318166eaf

New changeset 567f30527913 by Brian Curtin in branch 'default':
Fix #11583. Changed os.path.isdir to use GetFileAttributes instead of os.stat.
http://hg.python.org/cpython/rev/567f30527913

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11583] os.path.isdir() is slow on windows

[Brian Curtin]

Brian Curtin  added the comment:

This was also pushed to 2.7 in f1509fc75435.

--
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed
versions:  -Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12285] Unexpected behavior for 0 or negative processes in multiprocessing.pool()

[STINNER Victor]

STINNER Victor  added the comment:

multiprocessing_pool.patch: raise a ValueError if processes is less than 1. A 
test should be added.

--
keywords: +patch
nosy: +haypo
versions: +Python 2.7, Python 3.2, Python 3.3 -Python 2.6
Added file: http://bugs.python.org/file22287/multiprocessing_pool.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12289] http.server.CGIHTTPRequestHandler doesn't check if a Python script is executable

[STINNER Victor]

STINNER Victor  added the comment:

cgi.patch: fix the test checking that the script file is executable.

The patch removes the executable() function. This function is not documented 
but is public. The patch can be easily modified to keep this function if needed.

--
keywords: +patch
Added file: http://bugs.python.org/file22288/cgi.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12290] __setstate__ is called for false values

[Eugene Toder]

New submission from Eugene Toder :

Pickle documentation [1] says:

""" Note: If __getstate__() returns a false value, the __setstate__() method 
will not be called upon unpickling. """

However, this isn't quite true. This depends on the version of pickle protocol. 
A small example:

>>> class Pockle(object):
def __getstate__(self):
return 0
def __setstate__(self, state):
sys.stdout.write('__setstate__ is called!\n')

>>> for p in range(4):
sys.stdout.write('protocol %d: ' % p)
pickle.loads(pickle.dumps(Pockle(), p))

protocol 0: <__main__.Pockle object at 0x02EAE3C8>
protocol 1: <__main__.Pockle object at 0x02EAE358>
protocol 2: __setstate__ is called!
<__main__.Pockle object at 0x02EAE3C8>
protocol 3: __setstate__ is called!
<__main__.Pockle object at 0x02EAE358>

So for protocols >= 2 setstate is called. This is caused by 
object.__reduce_ex__ returning different tuples for different protocol versions:

>>> for p in range(4):
sys.stdout.write('protocol %d: %s\n' % (p, Pockle().__reduce_ex__(p)))


protocol 0: (, (, , None))
protocol 1: (, (, , None))
protocol 2: (, (,), 0, None, None)
protocol 3: (, (,), 0, None, None)

Implementation of reduce_ex for protos 0-1 in copy_reg.py contains the 
documented check: 
http://hg.python.org/cpython/file/f1509fc75435/Lib/copy_reg.py#l85

Implementation for proto 2+ in typeobject.c is happy with any value: 
http://hg.python.org/cpython/file/f1509fc75435/Objects/typeobject.c#l3205

Pickle itself only ignores None, not any false value: 
http://hg.python.org/cpython/file/f1509fc75435/Lib/pickle.py#l418

I think this is a documentation issue at this point.

[1] http://docs.python.org/py3k/library/pickle.html#pickle.object.__setstate__

--
components: Library (Lib)
messages: 137935
nosy: eltoder
priority: normal
severity: normal
status: open
title: __setstate__ is called for false values
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12290] __setstate__ is called for false values

[R. David Murray]

R. David Murray  added the comment:

See also #6827, just for some background on the current docs.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12290] __setstate__ is called for false values

[Eugene Toder]

Changes by Eugene Toder :


--
nosy: +alexandre.vassalotti, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12243] getpass.getuser works on OSX

[R. David Murray]

R. David Murray  added the comment:

It requires adding code to support a new Sphinx directive.  I'm not sure if it 
should be a generic Sphinx directive (in which case we should open an issue on 
the Sphinx bug tracker) or Python specific.  If the latter, I can't at the 
moment tell you where the python-specific Sphinx code lives, but someone will 
know.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10884] pkgutil EggInfoDistribution requirements for .egg-info metadata

[michael mulich]

michael mulich  added the comment:

Sure, I'll have a look. I'm only getting back into the swing of
things, but hopefully I can get more involved. Trying to do one or two
small tasks a night.

-Michael Mulich (pumazi)

On Mon, Jun 6, 2011 at 11:33 AM, Éric Araujo  wrote:
>
> Éric Araujo  added the comment:
>
> Can you check if this is covered in test_database?
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--
nosy: +michael.mulich2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11583] os.path.isdir() is slow on windows

[Eli Bendersky]

Eli Bendersky  added the comment:

Brian - great, thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7946] Convoy effect with I/O bound threads and New GIL

[Julian Mehnle]

Changes by Julian Mehnle :


--
nosy: +jmehnle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7753] newgil backport

[Julian Mehnle]

Changes by Julian Mehnle :


--
nosy: +jmehnle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com