Re: [Python-Dev] bsddb tests disabled by default

2008-09-20 Thread Martin v. Löwis
> real0m13.786s

test_bsddb3 takes about 30s real time on my system (Linux, with
Berkeley DB 4.6.21).

I don't think the default (of requiring the bsddb resource) can
change for 2.6; we already have released rc2, so there won't be
any further release candidates.

For 2.7, I would still be hesitant to run this test by default - 30s
is too long, IMO. It is unlikely that regular changes to Python break
any of these tests, and if they do, the buildbots will tell. Anybody
changing the bsddb library itself will know to run the full test suite.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] bsddb tests disabled by default

2008-09-20 Thread Martin v. Löwis
> Only thing I can think of is that Berkeley DB 4.7 is a ton faster than
> 4.6 or I am running something differently than you:

My suspicion is that there is a bug somewhere, probably in Berkeley DB.
For example, it might acquire some lock with a timeout, hoping that
normally, the lock gets released elsewhere quickly. On OSX, for whatever
reason, that assumption might be false, so the timeout eventually
occurs, along with retries and whatnot.

Of course, that's just a theory - one would have to debug the test suite
to find out what's really going on.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] bsddb tests disabled by default

2008-09-20 Thread skip

Brett> Well, it has always been that way for me, so I always assumed
Brett> test_bsddb3 was just a *really* long test.

Slow for me, but not nearly as bad as for you:

% time ./python.exe ../Lib/bsddb/test/test_all.py 

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Sleepycat Software: Berkeley DB 4.4.20: (January 10, 2006)
bsddb.db.version():   (4, 4, 20)
bsddb.db.__version__: 4.7.3pre5
bsddb.db.cvsid:   $Id: _bsddb.c 66182 2008-09-03 17:50:32Z jesus.cea $
py module:/Users/skip/src/python/trunk/Lib/bsddb/__init__.pyc
extension module: 
/Users/skip/src/python/trunk/regular/build/lib.macosx-10.3-i386-2.6/_bsddb.so
python version:   2.6rc2+ (trunk:66519M, Sep 20 2008, 08:36:03) 
[GCC 4.0.1 (Apple Inc. build 5465)]
My pid:   82520
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
. 
--
Ran 294 tests in 156.791s

OK

real2m37.188s
user0m9.907s
sys 0m6.196s

One thing I noticed was that you and I are both using BerkDB 4.4.20 while
Jesus is running 4.7.25.  I can't get to 4.7.x with MacPorts, but I can get
to 4.6.21.  I installed that, rebuild bsddb with it and reran the tests:

% time ./python.exe ../Lib/bsddb/test/test_all.py 

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Berkeley DB 4.6.21: (September 27, 2007)
bsddb.db.version():   (4, 6, 21)
bsddb.db.__version__: 4.7.3pre5
bsddb.db.cvsid:   $Id: _bsddb.c 66182 2008-09-03 17:50:32Z jesus.cea $
py module:/Users/skip/src/python/trunk/Lib/bsddb/__init__.pyc
extension module: 
/Users/skip/src/python/trunk/regular/build/lib.macosx-10.3-i386-2.6/_bsddb.so
python version:   2.6rc2+ (trunk:66519M, Sep 20 2008, 08:36:03) 
[GCC 4.0.1 (Apple Inc. build 5465)]
My pid:   21203
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=


--
Ran 300 tests in 557.679s

OK

real9m18.093s
user0m10.499s
sys 0m16.709s

Hmmm...  Those extra six tests are expensive!  I noticed there was a fair
chunk of time where the CPU meter showed the CPU essentially idle and the
dots were not moving.  I trust it was waiting for the rather modest disk on
my laptop.

Next stop, in-memory disk (all commands as root):

hdid -nomount ram://1024
newfs /dev/rdisk1
mkdir /tmp/mem
mount /dev/disk1 /tmp/mem
chmod 1777 /tmp/mem

and rerun the tests with TMPDIR pointing at /tmp/mem.  Whoops, it looks like
test_support creates temp files in the current directory, ignoring TMPDIR or
tempfile.gettempdir().  (Why is that???)  So, cd to /tmp/mem first.  Whoops!
Again, the bsddb tests force the test database into /tmp.  Fix test_all.py
to use TMPDIR if set.  Damn!  1gb isn't enough.  I tried boosting it to 2gb.
Still no go.  Jesus, how big is your ramdisk?

Here's a couple line patch for bsddb/test/test_all.py that uses TMPDIR if
it's set.

Index: Lib/bsddb/test/test_all.py
===
--- Lib/bsddb/test/test_all.py  (revision 66520)
+++ Lib/bsddb/test/test_all.py  (working copy)
@@ -443,6 +443,9 @@
 def set_test_path_prefix(path) :
 get_new_path.prefix=path
 
+if "TMPDIR" in os.environ:
+set_test_path_prefix(os.path.join(os.environ["TMPDIR"], "z-Berkeley_DB"))
+
 def remove_test_path_directory() :
 test_support.rmtree(get_new_path.prefix)

Skip
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com