Re: [Python-Dev] Segfault
On Thu, 2007-08-23 at 11:57 -0700, Neal Norwitz wrote: > > "allow threads" section that runs with the GIL released, file_close > > might acquire the GIL and be running in parallel to this code. If > > file_close sets f_fp to NULL after the "if" condition evaluates, but > > before the call to _portable_fseek completes, we still get a segfault. > > However, the setting of f_fp to NULL happens with the GIL set, ie > while only one thread is running. I thought other "IO-bound" threads (the sections that "allow threads") could also run in parallel. Otherwise what is the point of BEGIN_ALLOW_THREADS? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] any problem with py's RC?
hi all,
Here is 3 PYs, we found there is a big memory wasting, specially the program
c.py, we found the memory will increase very soon.
any body knows why? any RC's problem? thanks
--
eSX
class a():
def __init__(self):
fp = open("a_file_lager_1M","rb")
self.ll = fp.read()
fp.close()
self.h = b(self.aaa)
def aaa(self):
pass
class b:
def __init__(self,bbb):
self.a = bbb #remove the "self.", the memory use will be very
low!
pass
def c():
k=a()
while True:
c()
import haha
import time
def c():
b=haha.Client(["127.0.0.1"])
while True:
# time.sleep(1)
c()
#!/usr/bin/env python
from threading import local
class Client(local):
def __init__(self, servers, debug=0):
self.k = [_Host(s, self.debuglog) for s in servers]
def debuglog(self, str):
pass
class _Host:
def __init__(self, host, debugfunc=None):
self.debuglog = debugfunc #a -> b b->a, make the memory be huge!
pass
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] any problem with py's RC?
On Fri, Aug 24, 2007, eSX wrote: > > Here is 3 PYs, we found there is a big memory wasting, specially the > program c.py, we found the memory will increase very soon. any body > knows why? any RC's problem? thanks Please post your question to comp.lang.python; python-dev is not a suitable place for questions about how Python works. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "If you don't know what your program is supposed to do, you'd better not start writing it." --Dijkstra ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Segfault
On 8/24/07, Hrvoje Nikšić <[EMAIL PROTECTED]> wrote: > On Thu, 2007-08-23 at 11:57 -0700, Neal Norwitz wrote: > > > "allow threads" section that runs with the GIL released, file_close > > > might acquire the GIL and be running in parallel to this code. If > > > file_close sets f_fp to NULL after the "if" condition evaluates, but > > > before the call to _portable_fseek completes, we still get a segfault. > > > > However, the setting of f_fp to NULL happens with the GIL set, ie > > while only one thread is running. > > I thought other "IO-bound" threads (the sections that "allow threads") > could also run in parallel. Otherwise what is the point of > BEGIN_ALLOW_THREADS? Right. I looked at this with Jeffrey Yasskin and agree that a lock is needed for f_fp. The fix is ugly because it's needed around all accesses to f_fp and there are a ton of them. Some are with the GIL held and others when it isn't. I would be fine with the much simpler approach in the patch I sent (assuming the patch is finished). It doesn't completely address the problem, but does make the race condition much, much smaller. Part of the reason I'm ok with this is that in 3.0, all this code has already been removed. n ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] tarfile and directory traversal vulnerability
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
hi,
once upon a time there was a known vulnerability in tar (CVE-2001-1267,
[1]), and while tar is now long fixed, python's tarfile module is
affected too.
The vulnerability goes basically like this: If you tar a file named
"../../../../../etc/passwd" and then make the admin untar it,
/etc/passwd gets overwritten.
Another variety of this bug is a symlink one: if tar contains files like:
./-directory -> /etc
./-directory/passwd
then the "-directory" symlink would be created first and /etc/passwd
will be overwritten once again.
I was wondering how to fix it.
The symlink problem obviously applies only to extractall() method and is
easily fixed by delaying external (or possibly all) symlink creation,
similar to how directory attributes are delayed now.
I've attached a draft of the patch, if you like it, i'll polish it.
The traversal problem is harder, and it applies to extract() method as well.
For extractall() alone, i would use something like:
if tarinfo.name.startswith('../'):
self.extract(tarinfo, path)
else:
warnings.warn("non-local file skipped: %s" % tarinfo.name,
RuntimeWarning, stacklevel=1)
For extract(), i am not sure. Maybe it should throw exception when it
encounters such file, and have a special option to extract such files
anyway. Or maybe it should be left alone altogether.
Any suggestions are welcome.
regards
jan matejek
[1] http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2001-1267
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org
iD8DBQFGzxcpjBrWA+AvBr8RAlduAKCk0iiSoBF+wA9xgXmDlpWsECZ7KgCfQORg
lZ85inT1FGwhGqBfxJvCGGU=
=TiWx
-END PGP SIGNATURE-
--- Lib/tarfile.py
+++ Lib/tarfile.py
@@ -1503,6 +1503,7 @@
list returned by getmembers().
"""
directories = []
+symlinks = []
if members is None:
members = self
@@ -1516,6 +1517,9 @@
except EnvironmentError:
pass
directories.append(tarinfo)
+elif tarinfo.issym() and (tarinfo.linkpath.startswith('../') or tarinfo.linkpath.startswith('/')):
+# external symlink is delayed
+symlinks.append(tarinfo)
else:
self.extract(tarinfo, path)
@@ -1536,6 +1540,12 @@
else:
self._dbg(1, "tarfile: %s" % e)
+# Handle external symlinks
+symlinks.sort(lambda a, b: cmp(a.name, b.name))
+symlinks.reverse()
+for tarinfo in symlinks:
+self.extract(tarinfo, path)
+
def extract(self, member, path=""):
"""Extract a member from the archive to the current working directory,
using its full name. Its file information is extracted as accurately
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] tarfile and directory traversal vulnerability
> The vulnerability goes basically like this: If you tar a file named
> "../../../../../etc/passwd" and then make the admin untar it,
> /etc/passwd gets overwritten.
> Another variety of this bug is a symlink one: if tar contains files like:
> ./-directory -> /etc
> ./-directory/passwd
> then the "-directory" symlink would be created first and /etc/passwd
> will be overwritten once again.
I must admit I fail to see the bug. If root untars a file, and that tar
file contains an instruction to overwrite /etc/passwd, why is an error
to execute that instruction? Shouldn't root just be more careful when
untaring files?
> if tarinfo.name.startswith('../'):
> self.extract(tarinfo, path)
> else:
> warnings.warn("non-local file skipped: %s" % tarinfo.name,
> RuntimeWarning, stacklevel=1)
Ok. You seem to be claiming that the tarfile is incorrect in some
sense. Can you please point to some spec that says this is an incorrect
tarfile?
In any case, if you fix what you consider broken, you should do
it exactly the same way as GNU tar does it (assuming you consider
GNU tar fixed).
Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
