[ python-Bugs-775414 ] bsddb3 hash craps out with threads
Bugs item #775414, was opened at 2003-07-21 19:29 Message generated for change (Comment added) made by greg You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=775414&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Extension Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Gregory P. Smith (greg) Summary: bsddb3 hash craps out with threads Initial Comment: Richie Hindle presented something like the attached (hammer.py) on the spambayes-dev mailing list. On Win98SE and Win2K w/ Python 2.3c1 I usually see this death pretty quickly: Traceback (most recent call last): File "hammer.py", line 36, in ? main() File "hammer.py", line 33, in main hammer(db) File "hammer.py", line 15, in hammer x = db[str(int(random.random() * 10))] File "C:\CODE\PYTHON\lib\bsddb\__init__.py", line 86, in __getitem__ return self.db[key] bsddb._db.DBRunRecoveryError: (-30982, 'DB_RUNRECOVERY: Fatal error, run database recovery -- fatal region error detected; run recovery') Richie also reported "illegal operation" crashes on Win98SE. It's not clear whether a bsddb3 hash *can* be used with threads like this. If it can't, there's a doc bug. If it should be able to, there's a more serious problem. Note that it looks like hashopen() always merges DB_THREAD into the flags, so the absence of specifying DB_THREAD probably isn't the problem. -- >Comment By: Gregory P. Smith (greg) Date: 2005-11-05 08:31 Message: Logged In: YES user_id=413 oh good i see you already suggested the simple thread calling lock_detect that I was about to suggest. :) regardless a thread isn't needed. see dbenv.set_lk_detect which tells BerkeleyDB to run deadlock detection automatically anytime a lock conflict occurs. http://www.sleepycat.com/docs/api_c/env_set_lk_detect.html Just add e.set_lk_detect(db.DB_LOCK_DEFAULT) to bsddb/__init__.py's _openDBEnv() function. That causes hammer.py to get DBLockDeadlockError exceptions as expected (dying if the main thread gets one). No lock_detect thread needed. The bsddb legacy interface in __init__.py could have all of its database accesses wrapped in the bsddb.dbutils.DeadlockWrap function. to prevent this. (testing now) -- Comment By: Mark Hammond (mhammond) Date: 2005-11-03 20:20 Message: Logged In: YES user_id=14198 The db_deadlock program ends up being equivalent to a thread repeatedly calling: dbenv.lock_detect(bsddb.db.DB_LOCK_DEFAULT, 0) For completeness, I attach deadlock_hammer.py - a version that uses yet another thread to perform this lock detection. It also catches the deadlock exceptions, printing but ignoring them. Also, due to the way shutdown is less than graceful, I found I needed to add DB_RECOVER_FATAL to the env flags, otherwise I would often hang on open unless I clobbered the DB directory. On both my box (where it took a little while to see a deadlock) and on a dual-processor box (which provoked it much quicker), this version seems to run forever (although with sporadic performance) -- Comment By: Mark Hammond (mhammond) Date: 2005-11-03 18:00 Message: Logged In: YES user_id=14198 Sadly, I believe bsddb is working "as designed". Quoting from http://www.sleepycat.com/docs/api_c/env_open.html "When the DB_INIT_LOCK flag is specified, it is usually necessary to run a deadlock detector, as well." So I dig into my bsddb build tree, and found db_deadlock.exe. Sure enough, once studly_hammer.py had deadlocked, executing db_deadlock in the DB directory got things running again - although the threads all eventually died with: bsddb._db.DBLockDeadlockError: (-30996, 'DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock') Obviously it is PITA to need to run an external daemon, and as Python doesn't distribute db_deadlock.exe, the sleepycat license may mean not all applications are allowed to distribute it. This program also polls for deadlocks, meaning your app may hang as long as the poll period. All in all, it seems to suck :) -- Comment By: Gregory P. Smith (greg) Date: 2003-10-05 18:17 Message: Logged In: YES user_id=413 if you believe your application is properly using BerkeleyDB and you are having DB_RUNRECOVERY issues I suggest contacting sleepycat. -- Comment By: Rick Bradley (roundeye) Date: 2003-10-05 12:46 Message: Logged In: YES user_id=58334 This is also showing up in Syncato (http://www.syncato.org/), and the database isn't r
[ python-Bugs-1349106 ] email.Generators does not separates headers with "\r\n"
Bugs item #1349106, was opened at 2005-11-05 16:50 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1349106&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Manlio Perillo (manlioperillo) Assigned to: Nobody/Anonymous (nobody) Summary: email.Generators does not separates headers with "\r\n" Initial Comment: Regards. The email.Generator module does not separates headers with "\r\n". Manlio Perillo -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1349106&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-775414 ] bsddb3 hash craps out with threads
Bugs item #775414, was opened at 2003-07-21 19:29 Message generated for change (Comment added) made by greg You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=775414&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Extension Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Gregory P. Smith (greg) Summary: bsddb3 hash craps out with threads Initial Comment: Richie Hindle presented something like the attached (hammer.py) on the spambayes-dev mailing list. On Win98SE and Win2K w/ Python 2.3c1 I usually see this death pretty quickly: Traceback (most recent call last): File "hammer.py", line 36, in ? main() File "hammer.py", line 33, in main hammer(db) File "hammer.py", line 15, in hammer x = db[str(int(random.random() * 10))] File "C:\CODE\PYTHON\lib\bsddb\__init__.py", line 86, in __getitem__ return self.db[key] bsddb._db.DBRunRecoveryError: (-30982, 'DB_RUNRECOVERY: Fatal error, run database recovery -- fatal region error detected; run recovery') Richie also reported "illegal operation" crashes on Win98SE. It's not clear whether a bsddb3 hash *can* be used with threads like this. If it can't, there's a doc bug. If it should be able to, there's a more serious problem. Note that it looks like hashopen() always merges DB_THREAD into the flags, so the absence of specifying DB_THREAD probably isn't the problem. -- >Comment By: Gregory P. Smith (greg) Date: 2005-11-05 08:54 Message: Logged In: YES user_id=413 modifying bsddb/__init__.py to wrap all calls with DeadlockWrap will be a bit of a pita (but would be doable). I've attached an example wrapped_hammer.py that demonstrates the _openDBEnv change as well as DeadlockWrap wrapping to work properly. -- Comment By: Gregory P. Smith (greg) Date: 2005-11-05 08:31 Message: Logged In: YES user_id=413 oh good i see you already suggested the simple thread calling lock_detect that I was about to suggest. :) regardless a thread isn't needed. see dbenv.set_lk_detect which tells BerkeleyDB to run deadlock detection automatically anytime a lock conflict occurs. http://www.sleepycat.com/docs/api_c/env_set_lk_detect.html Just add e.set_lk_detect(db.DB_LOCK_DEFAULT) to bsddb/__init__.py's _openDBEnv() function. That causes hammer.py to get DBLockDeadlockError exceptions as expected (dying if the main thread gets one). No lock_detect thread needed. The bsddb legacy interface in __init__.py could have all of its database accesses wrapped in the bsddb.dbutils.DeadlockWrap function. to prevent this. (testing now) -- Comment By: Mark Hammond (mhammond) Date: 2005-11-03 20:20 Message: Logged In: YES user_id=14198 The db_deadlock program ends up being equivalent to a thread repeatedly calling: dbenv.lock_detect(bsddb.db.DB_LOCK_DEFAULT, 0) For completeness, I attach deadlock_hammer.py - a version that uses yet another thread to perform this lock detection. It also catches the deadlock exceptions, printing but ignoring them. Also, due to the way shutdown is less than graceful, I found I needed to add DB_RECOVER_FATAL to the env flags, otherwise I would often hang on open unless I clobbered the DB directory. On both my box (where it took a little while to see a deadlock) and on a dual-processor box (which provoked it much quicker), this version seems to run forever (although with sporadic performance) -- Comment By: Mark Hammond (mhammond) Date: 2005-11-03 18:00 Message: Logged In: YES user_id=14198 Sadly, I believe bsddb is working "as designed". Quoting from http://www.sleepycat.com/docs/api_c/env_open.html "When the DB_INIT_LOCK flag is specified, it is usually necessary to run a deadlock detector, as well." So I dig into my bsddb build tree, and found db_deadlock.exe. Sure enough, once studly_hammer.py had deadlocked, executing db_deadlock in the DB directory got things running again - although the threads all eventually died with: bsddb._db.DBLockDeadlockError: (-30996, 'DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock') Obviously it is PITA to need to run an external daemon, and as Python doesn't distribute db_deadlock.exe, the sleepycat license may mean not all applications are allowed to distribute it. This program also polls for deadlocks, meaning your app may hang as long as the poll period. All in all, it seems to suck :) -- Comment By: Gregory P. Smith (greg) Date: 2003-10-05 18:17 Message: Logge
[ python-Bugs-495682 ] cannot handle http_proxy with user:pass@
Bugs item #495682, was opened at 2001-12-21 01:22
Message generated for change (Comment added) made by jonico
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=495682&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Feature Request
Status: Open
Resolution: None
Priority: 3
Submitted By: Matthias Klose (doko)
Assigned to: Nobody/Anonymous (nobody)
Summary: cannot handle http_proxy with user:pass@
Initial Comment:
[please CC [EMAIL PROTECTED]; the original report
can be found at http://bugs.debian.org/120013 ]
I tried to use an http_proxy variable which looks
like:
http://user:[EMAIL PROTECTED]:3128/
with pass like \jkIoPd{
And I got this error :
Traceback (most recent call last):
File "/usr/bin/reportbug", line 1146, in ?
main()
File "/usr/bin/reportbug", line 628, in main
http_proxy)
File "/usr/lib/site-python/reportbug_ui_text.py",
line 314, in
handle_bts_query
archived=archived)
File "/usr/lib/site-python/debianbts.py", line 575,
in get_reports
result = get_cgi_reports(package, system,
http_proxy, archived)
File "/usr/lib/site-python/debianbts.py", line 494,
in get_cgi_reports
page = urlopen(url, proxies=proxies)
File "/usr/lib/site-python/debianbts.py", line 382,
in urlopen
return _urlopener.open(url)
File "/usr/lib/python2.1/urllib.py", line 176, in
open
return getattr(self, name)(url)
File "/usr/lib/python2.1/urllib.py", line 277, in
open_http
h = httplib.HTTP(host)
File "/usr/lib/python2.1/httplib.py", line 663, in
__init__
self._conn = self._connection_class(host, port)
File "/usr/lib/python2.1/httplib.py", line 342, in
__init__
self._set_hostport(host, port)
File "/usr/lib/python2.1/httplib.py", line 348, in
_set_hostport
port = int(host[i+1:])
ValueError: invalid literal for int(): \jkIoPd
[EMAIL PROTECTED]:3128
But if I use http_proxy=http://10.0.0.1:3128/, it
works well.
--
Comment By: Johannes Nicolai (jonico)
Date: 2005-11-05 18:09
Message:
Logged In: YES
user_id=863272
I have proposed a patch for this in the patch section:
1349118 is the number of the patch
URL:
https://sourceforge.net/tracker/index.php?func=detail&aid=1349118&group_id=5470&atid=305470
The patch also solves some other issues with proxies (now it
correct handles protocols, where a proxy was specified but not
supported and https proxies will also be used if a host requires
www-authentification)
--
Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-28 23:28
Message:
Logged In: YES
user_id=6380
This is a feature request. If someone submits a patch, we'll
happily apply it.
It looks like urllib2 already supports this feature; you
could try using that.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=495682&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1349106 ] email.Generators does not separates headers with "\r\n"
Bugs item #1349106, was opened at 2005-11-05 11:50 Message generated for change (Settings changed) made by bwarsaw You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1349106&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Manlio Perillo (manlioperillo) >Assigned to: Barry A. Warsaw (bwarsaw) Summary: email.Generators does not separates headers with "\r\n" Initial Comment: Regards. The email.Generator module does not separates headers with "\r\n". Manlio Perillo -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1349106&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1349316 ] xmlrpclib does not use http proxy
Bugs item #1349316, was opened at 2005-11-06 02:07 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1349316&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: greatred (greatred) Assigned to: Nobody/Anonymous (nobody) Summary: xmlrpclib does not use http proxy Initial Comment: The xmlrpclib class ServerProxy does not seem to be able to make use of a HTTP proxy. This makes it unusable behind a firewall where the proxy is the only access to the destination server. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1349316&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
