Re: [Twisted-Python] Twisted developers: please donate your time!
Le lundi 02 mars 2009 à 09:08 -0500, Itamar Shtull-Trauring a écrit : > Yes - thanks to you and all thee rest of the people who are > volunteering! Anyone else willing to step up? Sorry for being even more late. I won't spend as much time as I'd like, but I think 2 weeks of reviews/small branches is a safe bet. -- Thomas ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Twisted Sprint at PyCon
There will be an extended Twisted sprint at PyCon this year. I hope that everyone on this list who is attending will be present :). Technically I'm the sprint leader, but I would very much appreciate help from anyone else who is in an organizing mood. Please sign up on the PyCon wiki here: http://us.pycon.org/2009/sprints/projects/twisted/ ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] sending large files from web2 http server
Hi, I'm trying to stream longish data via web2, but experience sudden stalls in data transfer, followed by a connection abort after a certain timeout. I can't completely reproduce the issue, yet, but figured that the size of the blocks I'm returning via the stream's read method affects the failure rate (an stream.IByteStream implementation). I've tried with chunk sizes between 1 MiB down to 256 bytes. At that rate, I suddenly get the following unhandled errors, which don't seem to have to do much with my code. 2009/03/05 16:02 CET [-] Unhandled error in Deferred: 2009/03/05 16:02 CET [-] Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py", line 267, in unpause self._runCallbacks() File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py", line 307, in _runCallbacks self.result = callback(self.result, *args, **kw) File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py", line 271, in _continue self.unpause() File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py", line 267, in unpause self._runCallbacks() --- --- File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py", line 307, in _runCallbacks self.result = callback(self.result, *args, **kw) File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py", line 271, in _continue self.unpause() File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py", line 267, in unpause self._runCallbacks() File "/usr/lib/python2.4/site-packages/twisted/internet/defer.py", line 321, in _runCallbacks self.result = failure.Failure() File "/usr/lib/python2.4/site-packages/twisted/python/failure.py", line 205, in __init__ parentCs = reflect.allYourBase(self.type) File "/usr/lib/python2.4/site-packages/twisted/python/reflect.py", line 480, in allYourBase accumulateBases(classObj, l, baseClass) exceptions.RuntimeError: maximum recursion depth exceeded Are there any known problems with web2 and large files (not that 64 MiB is *that* large...). Any hint regarding the above error? Is the same to be expected with a chunk size of 8 KiB and a file of 2 GiB size (as that would result in the same amount of chunks)? Any hints would be greatly appreciated. Kind Regards Markus Wanner ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] sending large files from web2 http server
On Thu, 05 Mar 2009 16:20:27 +0100, Markus Wanner wrote: Hi, I'm trying to stream longish data via web2, but experience sudden stalls in data transfer, followed by a connection abort after a certain timeout. I can't completely reproduce the issue, yet, but figured that the size of the blocks I'm returning via the stream's read method affects the failure rate (an stream.IByteStream implementation). I've tried with chunk sizes between 1 MiB down to 256 bytes. At that rate, I suddenly get the following unhandled errors, which don't seem to have to do much with my code. I haven't looked at how web2 handles IByteStream providers, but my first guess would be that this is an example of a somewhat common bug where Deferreds are chained to an arbitrary length based on application data and when there's too much application data, the chain gets too long (a limit imposed by how much recursion is possible when unwinding the chain), and you get this failure. Alternatively, the bug I describe above might somehow have gotten into your application code, rather than into Web2. As you say, the stack trace doesn't point to your code, but it doesn't point to Web2 code either, and it's common in cases like this for the traceback to just point at the implementation of Deferred, since that's where the bug (or lack of feature, whatever) *really* is. The main focus of web development in Twisted now is Twisted Web, not Twisted Web2. However, if you can supply a patch which fixes this (along with unit tests, please :), then we'd be happy to apply it. James Knight or David Reid might have a suggestion about where in the code to start looking for the problem. Jean-Paul ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] _Win32Waker
I have been using a custom Qt4 reactor that derives from
PosixReactorBase. As a result it creates a _Win32Waker to allow
threads and signals to wake up the IO thread. It seems though that the
current implementation only works about half of the time. The other
half it exists with :
File "...\Lib\python2.6\site-packages\twisted\internet\posixbase.py",
line 170, in __init__
ReactorBase.__init__(self)
File "...\Lib\python2.6\site-packages\twisted\internet\base.py",
line 424, in __init__
self._initThreads()
File "...\Lib\python2.6\site-packages\twisted\internet\base.py",
line 813, in _initThreads
self.installWaker()
File "...\Lib\python2.6\site-packages\twisted\internet\posixbase.py",
line 206, in installWaker
self.waker = _Waker(self)
File "...\Lib\python2.6\site-packages\twisted\internet\posixbase.py",
line 77, in __init__
client.connect(server.getsockname())
File "", line 1, in connect
socket.error: [Errno 10049] The requested address is not valid in its context
I have attached a simple test that shows that the following code does
not always return "127.0.0.1", but sometimes returns "0.0.0.0" as the
IP address.
# Following select_trigger (from asyncore)'s example;
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.setsockopt(socket.IPPROTO_TCP, 1, 1)
server.bind(('127.0.0.1', 0))
server.listen(1)
client.connect(server.getsockname())
My current workaround just calls the following instead:
client.connect(('127.0.0.1', server.getsockname()[1]))
Any ideas on what is really causing the error? If there is not a
better solution can this be added to trunk for future releases?
Thanks,
Aron
test.py
Description: Binary data
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Re:how to pass on the connection failed or connection lost error
Hi,
I tried to use this suggestion but I could not make it work.
here is the test I am using
in my application class
def callSend(self, msg):
plugin.send(msg)
Now in plugin class
def send(self, msg):
print 'before call'
threads.blockingCallFromThread(reactor, remotesend, msg)
print 'after call'
def handleError(self, error):
error.raiseException()
def remotesend(self, msg):
deferred = defer.Deferred()
reactor.connectTCP('localhost',,myfactory)
deffered.addErrback(self.handleError)
return deffered
now myfactory class .. clientConnectionFailed
def clientConnectionFailed(self, connector, reason):
reason.raiseException()
now the problem is, code has become synchronous as code is waiting after
threads.blockingCallFromThread, how to fire the defer returned from
'remotesend'. Do i need to fire it or twisted will take care of it.
I think I can't refer this deferred object since it is being passed to
threads.blockingCallFromThread function on which I have no control.
any suggestions
--- On Wed, 3/4/09, twisted-python-requ...@twistedmatrix.com
wrote:
From: twisted-python-requ...@twistedmatrix.com
Subject: Twisted-Python Digest, Vol 60, Issue 6
To: twisted-python@twistedmatrix.com
Date: Wednesday, March 4, 2009, 11:00 AM
Send Twisted-Python mailing list submissions to
twisted-python@twistedmatrix.com
To subscribe or unsubscribe via the World Wide Web, visit
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
or, via email, send a message with subject or body 'help' to
twisted-python-requ...@twistedmatrix.com
You can reach the person managing the list at
twisted-python-ow...@twistedmatrix.com
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Twisted-Python digest..."
Today's Topics:
1. Re: how to pass on the connection failed or connection lost
error (Jean-Paul Calderone)
--
Message: 1
Date: Wed, 4 Mar 2009 09:59:42 -0500
From: Jean-Paul Calderone
Subject: Re: [Twisted-Python] how to pass on the connection failed or
connection lost error
To: twisted-python@twistedmatrix.com
Message-ID:
<20090304145942.12853.42253504.divmod.quotient.17...@henry.divmod.com>
Content-Type: text/plain; format=flowed
On Wed, 4 Mar 2009 06:39:59 -0800 (PST), khawar hasham
wrote:
>Hi
>
>let me first explain the application that I am developing. I have an
application that will use the twisted part as a plugin. this twisted part will
act as server and as client both.
>my application call the plugin method to send data to server module using
connectTCP. now the problem is I can not pass on the connection failed exception
to my calling application.
Since you're running the reactor in one thread and the rest of your
application in another thread, your question is basically one of message
passing. You've already found reactor.callFromThread which is good; I
think you just want the slightly more featureful version, available in
twisted.internet.threads, blockingCallFromThread. This is implemented
in terms of callFromThread, but additionally allows the caller to get
the result of the function being called - including waiting on a Deferred
if the function being called returns one. If you switch to this, then
you only need to make Plugin.clientsend return a Deferred which eventually
fires with a result or a Failure.
Jean-Paul
--
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
End of Twisted-Python Digest, Vol 60, Issue 6
*
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] How to invoke the remote method based on twisted in django web app?
hi,all:
I developed a web app in diango and a remote service in twisted, and I want
to invoke the twisted remote method in django web.example:
the remote service code :
class Echoer(pb.Root):
def remote_echo(self, task):
print 'echoing:', task
return task
if __name__ == '__main__':
reactor.listenTCP(8789, pb.PBServerFactory(Echoer()))
reactor.run()
-
and the djiango views.py code :
def register_task(requst):
"""register the task"""
.
factory = pb.PBClientFactory()
reactor.connectTCP("localhost", 8789, factory)
d = factory.getRootObject()
d.addCallback(lambda object: object.callRemote("echo", task))
d.addCallback(lambda echo: 'server echoed: '+echo[0]+str(echo[1]))
d.addErrback(lambda reason: 'error: '+str(reason.value))
d.addCallback(util.println)
d.addCallback(lambda _: reactor.stop())
reactor.run()
..
return HttpResponseRedirect('/push_task/') # Redirect after POST
-
but when I post a request to django web app ,a exception display:
Traceback (most recent call last):
File "C:\Python25\Lib\site-packages\twisted\internet\base.py", line 374,
in fi
reEvent
DeferredList(beforeResults).addCallback(self._continueFiring)
File "C:\Python25\Lib\site-packages\twisted\internet\defer.py", line 195,
in a
ddCallback
callbackKeywords=kw)
File "C:\Python25\Lib\site-packages\twisted\internet\defer.py", line 186,
in a
ddCallbacks
self._runCallbacks()
File "C:\Python25\Lib\site-packages\twisted\internet\defer.py", line 328,
in _
runCallbacks
self.result = callback(self.result, *args, **kw)
--- ---
File "C:\Python25\Lib\site-packages\twisted\internet\base.py", line 387,
in _c
ontinueFiring
callable(*args, **kwargs)
File "C:\Python25\Lib\site-packages\twisted\internet\base.py", line 1123,
in _
reallyStartRunning
self._handleSignals()
File "C:\Python25\Lib\site-packages\twisted\internet\base.py", line 1068,
in _
handleSignals
signal.signal(signal.SIGINT, self.sigInt)
exceptions.ValueError: signal only works in main thread
--
so ,How I correctly invoke the remote method?
--
Boern Parx
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
