[ python-Bugs-1779233 ] PyThreadState_SetAsyncExc and the main thread

[SourceForge.net]

Bugs item #1779233, was opened at 2007-08-22 10:49
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1779233&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Threads
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Rotem (rotem_yaari)
Assigned to: Nobody/Anonymous (nobody)
Summary: PyThreadState_SetAsyncExc and the main thread

Initial Comment:
Hi,

The following does not work in python 2.5:
##
import ctypes
import thread
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(
   thread.get_ident(),
   ctypes.py_object(SystemExit))
##

Although according to my understanding this should "schedule" an async 
exception for the main thread, it does not (res receives the value of 0).

When raising exceptions in other threads in this way, it works and the call to 
PyThreadState_SetAsyncExc returns 1 like it should. Doing so on the main thread 
doesn't seem to work, even when performed from threads other than the main one.

--

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1779233&group_id=5470
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[ python-Bugs-1776160 ] Buffer overflow when listing deeply nested directory

[SourceForge.net]

Bugs item #1776160, was opened at 2007-08-17 13:24
Message generated for change (Comment added) made by sonderblade
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1776160&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bj�rn Lindqvist (sonderblade)
Assigned to: Nobody/Anonymous (nobody)
Summary: Buffer overflow when listing deeply nested directory

Initial Comment:
This code:

import os
import os.path
TARGET='C:/code/python/foo'
base = TARGET
for x in range(200):
subdirs = os.listdir(base)
base = os.path.join(base, subdirs[0])
print base

Produces a TypeError (buffer overflow) when run on a to deeply nested directory 
for windows to handle:

.. more output here..
C:code/python/foo\foo bar.png\foo bar.png\foo bar.png\foo bar.png\foo 
bar.png\foo bar.png\foo bar.png\foo bar.png\foo bar.png\foo bar.png\foo 
bar.png\foo bar.png\foo bar.p
ng\foo bar.png\foo bar.png\foo bar.png\foo bar.png\foo bar.png\foo bar.png\foo 
bar.png
Traceback (most recent call last):
  File "killdir.py", line 6, in 
subdirs = os.listdir(base)
TypeError: listdir() argument 1 must be (buffer overflow), not str


--

>Comment By: Bj�rn Lindqvist (sonderblade)
Date: 2007-08-22 12:26

Message:
Logged In: YES 
user_id=51702
Originator: YES

Yes, it is the error message and the exception that is the problem. First,
it shouldn't raise TypeError (which indicates a programming error), it
should raise either IOError, OSError or WindowsError. Second, the exception
message is whacky: "listdir() argument 1 must be (buffer overflow), not
str" I realize that it is probably impossible to detect this specific error
condition but I still want something more explanatory than what it
currently is.

--

Comment By: Martin v. Löwis (loewis)
Date: 2007-08-22 07:56

Message:
Logged In: YES 
user_id=21627
Originator: NO

Can you please explain what specifically you consider a bug here?

I can see that the error message is confusing, so it could be improved.
However, there is nothing we can do to make the error go away. The
Microsoft C library simply does not support file names longer than
MAX_PATH; you have to use Unicode file names to go beyond this limit.

--

Comment By: Bj�rn Lindqvist (sonderblade)
Date: 2007-08-21 10:49

Message:
Logged In: YES 
user_id=51702
Originator: YES

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)]
on win32
MS Windows XP, Version 5.1, SP2


--

Comment By: Martin v. Löwis (loewis)
Date: 2007-08-21 10:18

Message:
Logged In: YES 
user_id=21627
Originator: NO

To rephrase Skip's comment: Can you please report what operating system
and Python version you are using?

--

Comment By: Skip Montanaro (montanaro)
Date: 2007-08-18 13:38

Message:
Logged In: YES 
user_id=44345
Originator: NO

Worked as expected for me on Mac OS X 10.4.10 running from
the trunk (you didn't mention what version you were using).
In ~/tmp/deep I created a maximally nested directory tree from the shell
like so:

cd /Users/skip/tmp/deep
for i in `range 1000` ; do
x=`printf %04d $i`
echo $x
mkdir $x
cd $x
done

where the range command is analogous to Python's range
builtin:

% range 20
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

The for loop barfed after making directory 0205.

In Python I then executed these statements:

import os.path
base = "/Users/skip/tmp/deep"
for x in range(210):
subdirs = os.listdir(base)
base = os.path.join(base, subdirs[0])
print base

This went until it got to dir 0200 where it raised an
OSError:

[Errno 63] File name too long:
'/Users/skip/tmp/deep//0001/.../0199/0200'

which stands to reason since base was 1025 characters long
at that point.  MAXPATHLEN is defined to be 1024 on my
system, so the OSError is to be expected.

Skip


--

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1776160&group_id=5470
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1000] test

[user]

Changes by user:


--
type:  -> rfe

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1000] test

[user]

New submission from user:

test patch

--
files: python.diff
messages: 55112
nosy: user
severity: normal
status: open
title: test

__
Tracker <[EMAIL PROTECTED]>

__Index: Modules/_ssl.c
===
--- Modules/_ssl.c  (revision 51574)
+++ Modules/_ssl.c  (working copy)
@@ -317,7 +317,7 @@
 
if (!PyArg_ParseTuple(args, "O!|zz:ssl",
  PySocketModule.Sock_Type,
- (PyObject*)&Sock,
+ &Sock,
  &key_file, &cert_file))
return NULL;
 
Index: Modules/_ctypes/callproc.c
===
--- Modules/_ctypes/callproc.c  (revision 51574)
+++ Modules/_ctypes/callproc.c  (working copy)
@@ -1230,11 +1230,11 @@
 
 static PyObject *py_dl_close(PyObject *self, PyObject *args)
 {
-   void * handle;
+   int handle;
 
if (!PyArg_ParseTuple(args, "i:dlclose", &handle))
return NULL;
-   if (dlclose(handle)) {
+   if (dlclose((void*)handle)) {
PyErr_SetString(PyExc_OSError,
   ctypes_dlerror());
return NULL;
@@ -1246,12 +1246,12 @@
 static PyObject *py_dl_sym(PyObject *self, PyObject *args)
 {
char *name;
-   void *handle;
+   int handle;
void *ptr;
 
if (!PyArg_ParseTuple(args, "is:dlsym", &handle, &name))
return NULL;
-   ptr = ctypes_dlsym(handle, name);
+   ptr = ctypes_dlsym((void*)handle, name);
if (!ptr) {
PyErr_SetString(PyExc_OSError,
   ctypes_dlerror());
@@ -1269,7 +1269,7 @@
 static PyObject *
 call_function(PyObject *self, PyObject *args)
 {
-   PPROC func;
+   int func;
PyObject *arguments;
PyObject *result;
 
@@ -1279,7 +1279,7 @@
  &PyTuple_Type, &arguments))
return NULL;
 
-   result =  _CallProc(func,
+   result =  _CallProc((PPROC)func,
arguments,
 #ifdef MS_WIN32
NULL,
@@ -1300,7 +1300,7 @@
 static PyObject *
 call_cdeclfunction(PyObject *self, PyObject *args)
 {
-   PPROC func;
+   int func;
PyObject *arguments;
PyObject *result;
 
@@ -1310,7 +1310,7 @@
  &PyTuple_Type, &arguments))
return NULL;
 
-   result =  _CallProc(func,
+   result =  _CallProc((PPROC)func,
arguments,
 #ifdef MS_WIN32
NULL,
@@ -1493,7 +1493,7 @@
 #else
  "On:resize",
 #endif
- (PyObject *)&obj, &size))
+ &obj, &size))
return NULL;
 
dict = PyObject_stgdict((PyObject *)obj);
Index: Modules/dbmmodule.c
===
--- Modules/dbmmodule.c (revision 51574)
+++ Modules/dbmmodule.c (working copy)
@@ -208,11 +208,13 @@
 static PyObject *
 dbm_has_key(register dbmobject *dp, PyObject *args)
 {
+   char *tmp_ptr;
datum key, val;
int tmp_size;

-   if (!PyArg_ParseTuple(args, "s#:has_key", &key.dptr, &tmp_size))
+   if (!PyArg_ParseTuple(args, "s#:has_key", &tmp_ptr, &tmp_size))
return NULL;
+   key.dptr = tmp_ptr;
key.dsize = tmp_size;
 check_dbmobject_open(dp);
val = dbm_fetch(dp->di_dbm, key);
@@ -224,11 +226,13 @@
 {
datum key, val;
PyObject *defvalue = Py_None;
+   char *tmp_ptr;
int tmp_size;
 
if (!PyArg_ParseTuple(args, "s#|O:get",
-  &key.dptr, &tmp_size, &defvalue))
+  &tmp_ptr, &tmp_size, &defvalue))
return NULL;
+   key.dptr = tmp_ptr;
key.dsize = tmp_size;
 check_dbmobject_open(dp);
val = dbm_fetch(dp->di_dbm, key);
@@ -245,11 +249,13 @@
 {
datum key, val;
PyObject *defvalue = NULL;
+   char *tmp_ptr;
int tmp_size;
 
if (!PyArg_ParseTuple(args, "s#|S:setdefault",
-  &key.dptr, &tmp_size, &defvalue))
+  &tmp_ptr, &tmp_size, &defvalue))
return NULL;
+   key.dptr = tmp_ptr;
key.dsize = tmp_size;
 check_dbmobject_open(dp);
val = dbm_fetch(dp->di_dbm, key);
Index: Modules/audioop.c
===
--- Modules/audioop.c   (revision 51574)
+++ Modules/audioop.c   (working copy)
@@ -472,8 +472,12 @@
 double aj_m1, aj_lm1;
 double sum_ri_2, sum_aij_2, sum_aij_ri, result, best_result, factor;
 
+   /* Passing a short** for an 's' argumen

[issue1001] Test 2

[user]

Changes by user:


--
severity: normal
status: open
title: Test 2

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[ python-Bugs-1731717 ] race condition in subprocess module

[SourceForge.net]

Bugs item #1731717, was opened at 2007-06-06 00:19
Message generated for change (Comment added) made by gjb1002
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1731717&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: dsagal (dsagal)
Assigned to: Peter Åstrand (astrand)
Summary: race condition in subprocess module

Initial Comment:
Python's subprocess module has a race condition: Popen() constructor has a call 
to global "_cleanup()" function on whenever a Popen object gets created, and 
that call causes a check for all pending Popen objects whether their subprocess 
has exited - i.e. the poll() method is called for every active Popen object.

Now, if I create Popen object "foo" in one thread, and then a.wait(), and 
meanwhile I create another Popen object "bar" in another thread, then a.poll() 
might get called by _clean() right at the time when my first thread is running 
a.wait(). But those are not synchronized - each calls os.waitpid() if 
returncode is None, but the section which checks returncode and calls 
os.waitpid() is not protected as a critical section should be.

The following code illustrates the problem, and is known to break reasonably 
consistenly with Python2.4. Changes to subprocess in Python2.5 seems to address 
a somewhat related problem, but not this one.

import sys, os, threading, subprocess, time

class X(threading.Thread):
  def __init__(self, *args, **kwargs):
super(X, self).__init__(*args, **kwargs)
self.start()

def tt():
  s = subprocess.Popen(("/bin/ls", "-a", "/tmp"), stdout=subprocess.PIPE,
  universal_newlines=True)
  # time.sleep(1)
  s.communicate()[0]

for i in xrange(1000):
  X(target = tt)

This typically gives a few dozen errors like these:
Exception in thread Thread-795:
Traceback (most recent call last):
  File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap
self.run()
  File "/usr/lib/python2.4/threading.py", line 422, in run
self.__target(*self.__args, **self.__kwargs)
  File "z.py", line 21, in tt
s.communicate()[0]
  File "/usr/lib/python2.4/subprocess.py", line 1083, in communicate
self.wait()
  File "/usr/lib/python2.4/subprocess.py", line 1007, in wait
pid, sts = os.waitpid(self.pid, 0)
OSError: [Errno 10] No child processes

Note that uncommenting time.sleep(1) fixes the problem. So does wrapping 
subprocess.poll() and wait() with a lock. So does removing the call to global 
_cleanup() in Popen constructor.

--

Comment By: Geoffrey Bache (gjb1002)
Date: 2007-08-22 19:53

Message:
Logged In: YES 
user_id=769182
Originator: NO

There should be test cases aplenty - this bug was first reported in March
2006 and this is one of four incarnations of it. See also 1753891, 1754642,
1183780

I think all of these contain testcases (though two roughly coincide) and
any fix would be wise to try them all out...


--

Comment By: Peter Åstrand (astrand)
Date: 2007-08-06 22:02

Message:
Logged In: YES 
user_id=344921
Originator: NO

>I can create a patch to make current head a bit cleaner, if anyone is
>interested...

Sure, this would be great. 

I would also love a test case that reproduces this problem. 

--

Comment By: Donovan Baarda (abo)
Date: 2007-08-06 21:45

Message:
Logged In: YES 
user_id=10273
Originator: NO

I can create a patch to make current head a bit cleaner, if anyone is
interested...

--

Comment By: Donovan Baarda (abo)
Date: 2007-08-03 23:29

Message:
Logged In: YES 
user_id=10273
Originator: NO

I just looked at subprocess in svs trunk and it looks like it might
already be fixed in both subprocess.py and popen2.py. The part where
__del__() adds abandoned Popen instances to _active and _cleanup() removes
them is already there. _cleanup() has been made more thread resistant by
catching and ignoring exceptions that arise when two _cleanups() try to
remove the same instances. Popen.poll() has been made more robust by having
it set self.returncode to an optional _deadstate argument value when poll
gets an os.error from os.pidwait() on a child that gets cleaned up by
another thread. I think there are still a few minor race conditions around
Popen.poll(), but it will only affect what non-zero returncode you get...
it's not so much "thread safe" as "thread robust".

I think the _deadstate argument approach to try and make Popen.poll()
thread-robust is a bit hacky. I would rather see _cleanup() be properly
threadsafe by having it remove the inst from _active before proce

[ python-Bugs-1779700 ] urlparse.urljoin does not obey current uri rfc (rfc 3986)

[SourceForge.net]

Bugs item #1779700, was opened at 2007-08-22 12:57
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1779700&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: John Adams (icosine)
Assigned to: Nobody/Anonymous (nobody)
Summary: urlparse.urljoin does not obey current uri rfc (rfc 3986)

Initial Comment:
First, there are a lot of RFC's. The latest is RFC3986 (see 
http://en.wikipedia.org/wiki/Uniform_Resource_Identifier)

A base url of the form: 

http://a/b/c/d;p?q

when resolved with a relative link of the form:

?y 

resolves to:

http://a/b/c/?q

rather than:

http://a/b/c/d;p?q


It may seem that this doesn't matter, but try this in firefox or IE and you'll 
see that they exhibit the up-to-date correct behavior.

Thanks!


--

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1779700&group_id=5470
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com