ebfe added the comment:
Haypo, we can probably reduce overhead by defining ENTER_HASHLIB like this:
#define ENTER_HASHLIB(obj) \
if ((obj)->lock) { \
if (!PyThread_acquire_lock((obj)->lock, 0)) { \
Py_BEGIN_ALLOW_THREADS \
PyThread_acquire_lock((obj)
New submission from ebfe :
Here is another patch, this time for the fallback-md5-module. I know
that situations are rare where openssl is not present but threading is.
However they might occur out there and the md5module needed some love
anyway:
- The MD5 class from the fallback module can now
Changes by ebfe :
Removed file: http://bugs.python.org/file12557/md5module_small_locks.diff
___
Python tracker
<http://bugs.python.org/issue4751>
___
___
Python-bug
ebfe added the comment:
Here is another patch, this time for the fallback-md5-module. I know
that situations are rare where openssl is not present but threading is.
However they might occur out there and the md5module needed some love
anyway:
- The MD5 class from the fallback module can now
ebfe added the comment:
haypo, the patch will not compile when WITH_THREADS is not defined. The
'lock'-member in the object structure is not present without
WITH_THREADS however the line 'if (self->lock == NULL && view.len >=
HASHLIB_GIL_MINS
ebfe added the comment:
I don't think so.
The interface should stay simple - python has very few such magic knobs.
People will optimize for their own box as you said - and that code will
run worse on all the others...
Besides, we've lived so long with single-threaded openssl.
ebfe added the comment:
I don't think this is actually worth the trouble. You run into situation
where one thread might decide that it needs a lock now with other
threads being in the to-be-locked-area at that time.
___
Python tracker
ebfe added the comment:
gnarf, actually it should be 'threads.append(Hasher(md))' in the script :-\
___
Python tracker
<http://bugs.python.org/issue4751>
___
___
Changes by ebfe :
Removed file: http://bugs.python.org/file12461/hashopenssl_threads-3.diff
___
Python tracker
<http://bugs.python.org/issue4751>
___
___
Python-bug
ebfe added the comment:
test-script
Added file: http://bugs.python.org/file12534/hashlibtest2.py
___
Python tracker
<http://bugs.python.org/issue4751>
___
___
Python-bug
ebfe added the comment:
Releasing the GIL is somewhat expensive and should be avoided if
possible. I've moved LEAVE_HASHLIB in EVP_update so the object gets
unlocked before we call Py_END_ALLOW_THREADS. This is *only* possible
because EVP_update does not use the object beyond those lines.
ebfe added the comment:
test-script
Added file: http://bugs.python.org/file12532/zlibtest2.py
___
Python tracker
<http://bugs.python.org/issue4738>
___
___
Python-bug
ebfe added the comment:
Here is a small test-script with concurrent access to a single
compressosbj. The original patch will immediately deadlock.
The patch attached releases the GIL before trying to get the zlib-lock.
This allows the other thread to release the zlib-lock but comes at the
cost
Changes by ebfe :
Removed file: http://bugs.python.org/file12466/zlib_threads-2.diff
___
Python tracker
<http://bugs.python.org/issue4738>
___
___
Python-bugs-list mailin
ebfe added the comment:
Whenever the documentation says "you must not" it really says "don't do
that or your application *will* crash, burn and die"... Of course I can
allocate storage for the string, copy it's content and then free or -
nothing will happen. How wo
ebfe added the comment:
I can't reproduce the problem here.
Python 2.5.2 running on Linux lueg-desktop 2.6.24-22-generic #1 SMP Mon
Nov 24 18:32:42 UTC 2008 i686 GNU/Linux
--
nosy: +ebfe
___
Python tracker
<http://bugs.python.org/i
ebfe added the comment:
I don't think Python 2.x should be changed - but 3.0 or 3.1 should be:
- Characters don't mean a thing in zlib-land, all operations are based
on bytes and their (implicit) default encoding. This behaviour is hidden
and somewhat violates the rule of leas
Changes by ebfe :
Removed file: http://bugs.python.org/file12463/zlib_threads-2.diff
___
Python tracker
<http://bugs.python.org/issue4738>
___
___
Python-bugs-list mailin
ebfe added the comment:
new svn diff attached
the indentation in this file is not my fault, it has tabs all over it...
The 5kb limits protects from the overhead of releasing the GIL. With
very small buffers the overall runtime in my benchmark tends to double.
I set it based on my testing and
ebfe added the comment:
Here is another simple benchmarker. For me it shows almost perfect
scaling (2 cores = 196% performance) if the buffer put into .update() is
large enough.
I deliberately did not move Py_BEGIN_ALLOW_THREADS into EVP_hash as we
might call this function without having some
Changes by ebfe :
Removed file: http://bugs.python.org/file12448/zlib_threads.diff
___
Python tracker
<http://bugs.python.org/issue4738>
___
___
Python-bugs-list mailin
ebfe added the comment:
new svn diff attached
- GIL is now released for adler32 and crc32 if the buffer is larger than
5kb (we don't want to risk burning cpu cycles by GIL-stuff)
- adler32 got it's param by s# but now does s* - why s# anyway?
- ENTER_ZLIB no longer gives away the
ebfe added the comment:
Thanks for the advices.
Antoine, maybe you could clarify the situation regarding buffer-locks
for me. In older versions of PEP 3118 the PyBUF_LOCK flag was still
present but it doesn't seem to have made it's way into the final draft.
Is it save to assume tha
Changes by ebfe :
Removed file: http://bugs.python.org/file12453/hashopenssl_threads.diff
___
Python tracker
<http://bugs.python.org/issue4751>
___
___
Python-bugs-list m
New submission from ebfe :
The hashlib functions provided by _hashopenssl.c hold the GIL all the
time although the underlying openssl-library is basically thread-safe.
I've attached a patch (svn diff) which basically does four things:
* If python is compiled with thread-support, the EVPo
ebfe added the comment:
new svn diff
Added file: http://bugs.python.org/file12448/zlib_threads.diff
___
Python tracker
<http://bugs.python.org/issue4738>
___
___
Pytho
New submission from ebfe :
Quote from http://docs.python.org/3.0/c-api/arg.html, regarding the "s"
argument:
"""
s (string or Unicode object) [const char *]
Convert a Python string or Unicode object to a C pointer to a
character string. You must not provide storage
New submission from ebfe :
My application needs to pack and unpack workunits all day long and does
this using multiple threading.Threads. I've noticed that the zlib module
seems to use only one thread at a time when using [de]compressobj(). As
the comment in the sourcefile zlibmodule.c al
28 matches
Mail list logo
