I'm not quite sure, but I think I'm pretty careful of sharing objects between threads. 1st, I only cached as few as possible orm objects. I tried to detach them, but I found that if I detach them, I can't access any of their fields any more.
2nd, I create new orm objects based on client request, pass them to class Database and then merge them to scoped sessions, change, commit and then discard these objects. 3rd, I switch to sqlite frequently to check if there is any database operation outside Database, because sqlite doesn't allow multi-thread access. Actually it seems to work until 2 or 3 days ago suddenly cases hang the server. Ah, as I've already written lots of code in ORM, I think maybe I should try to change Database to use a dedicated thread to handle all database operations. That might be a bottle neck of my application, but I really can't give up orm as these mapper classes are used everywhere in my application. On Thu, Mar 5, 2009 at 3:04 AM, Valentino Volonghi <dialt...@gmail.com>wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > On Mar 4, 2009, at 3:28 AM, Peter Cai wrote: > > The code doesn't work. When I limit the thread numbers to 1 >> >> reactor.suggestThreadPoolSize(1) >> >> Everything goes fine. Other wise the server would be blocked and must >> be killed by "kill 9 ...". >> >> The result conflicts with my understanding of sqlalchemy. Since I >> don't share any object between threads, there should be no problem! >> > > I'm pretty sure you can't say this with full certainty and actually you are > just wrong based on the code you showed. deferToThread doesn't use > the same thread every time you call it, there's absolutely no guarantee > in that and sqlalchemy keeps state around in that thread related to that > object it returned. If you want to do any operations you need either to > detach the object from the session before returning it and do any > modification > on the same object in another thread after reattaching it (pretty > cumbersome). > > Or write your own threadpool that gives names to threads so that you can > have a guarantee that you always call the same thread when working with > a bunch of objects. > > Nonetheless though sqlalchemy is a huge project and I'm pretty sure it has > some deadlocks and race conditions around which means that even taking > care of these issues you'll have other problems when dealing with the orm. > > If you want to use sqlalchemy don't use its orm but just the query builder, > it's the only sane way to integrate with twisted. Which anyway IMHO is the > best way to use it anyway because it uses a lot less memory since it > doesn't > have to always cache objects because you control everything and can make > that call when you really need it. > > Ah.... It always have risk to use something you haven't tried >> before .... >> I think I have no choice but always set thread pool size to 1 ... >> > > > Not entirely true. > > - -- > Valentino Volonghi aka Dialtone > Now running MacOS X 10.5 > Home Page: http://www.twisted.it > http://www.adroll.com > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (Darwin) > > iEYEARECAAYFAkmu0NoACgkQ9Llz28widGXBawCg32svBsLqsIRLzvzOThgR4sA0 > 5UkAoIgNfyUDPl9c0nwSud0sem3aKjz5 > =2XIX > -----END PGP SIGNATURE----- > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > -- look to the things around you,the immediate world around you, if you are alive,it will mean something to you ——Paul Strand
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
