Hi there,
I encountered a problem while dealing with file transfers using PB and
t.s.u.FilePager.
The sscce illustrates the problem:
--------- server.py --------------------------------------------------
from twisted.spread import pb
from twisted.internet import reactor
ROOT = '/tmp/'
class ImageCollector(pb.Referenceable):
def __init__(self, fd):
self.fd = fd
def remote_dummy(self):
pass
def remote_gotPage(self, page):
self.fd.write(page)
def remote_endedPaging(self):
print "Completed"
self.fd.close()
class VurmController(pb.Root):
def remote_createImage(self, imageId):
return ImageCollector(open(ROOT + imageId, 'w'))
reactor.listenTCP(8789, pb.PBServerFactory(VurmController()))
reactor.run()
----------------------------------------------------------------------
--------- client.py --------------------------------------------------
import sys
from twisted.spread import pb
from twisted.internet import reactor, defer
from twisted.spread import util
FILE = 'myfile.something'
factory = pb.PBClientFactory()
reactor.connectTCP("localhost", 8789, factory)
d = factory.getRootObject()
def createImage(controller):
return controller.callRemote('createImage', 'image-id')
d.addCallback(createImage)
def sendImage(ctl):
print ctl.callRemote('dummy') ############# LINE 19 #############
d = defer.Deferred()
util.FilePager(ctl, open(FILE), callback=lambda: d.callback(None))
return d
d.addCallback(sendImage)
def done(_):
print "Transfer completed"
#reactor.callLater(1, reactor.stop)
reactor.stop()
d.addCallback(done)
reactor.run()
----------------------------------------------------------------------
I'm trying to *upload* a file from the client to the server by calling a method
on the
server which returns a collector to be used with a FilePager instance.
When run with the line client.py:19 commented out, the script blocks before
sending
out any file chunks, if the line is uncommented (effectively calling the
'dummy' method
remotely), everything works as expected.
Digging around in the sources, I found out that t.p.u.Pager registers itself to
the
collector's pb broker as a pageProducer. When resumeProducing is called on the
broker,
it asks the FilePager instance for the next page, which leads to the invocation
of
the following method on FilePager::
def sendNextPage(self):
"""
Get the first chunk read and send it to collector.
"""
if not self.chunks:
return
val = self.chunks.pop(0)
self.producer.resumeProducing()
self.collector.callRemote("gotPage", val)
As the t.b.FileSender producer does not yet had a chance to write something to
the
FilePager, the method returns straight away without sending anything.
As anything was sent, the Broker.resumeProducing method is never called again,
and
thus the FilePager.sendNextPage neither.
The problem is caused by the calling chain of the FilePager constructor:
FilePager.__init__()
-> Pager.__init__()
-> broker.registerPageProducer(FilePager)
-> transport.registerProducer(broker)
-> broker.resumeProducing()
-> FilePager.sendNextPage()
-> <no data, don't send anything>
-> FilePager.startProducing(fd)
-> FileSender().beginFileTransfer(fd, FilePager)
-> FilePager.registerProducer(FileSender)
-> FileSender.resumeProducing()
-> FilePager.write(data)
-> <store data, wait for sendNextPage to be called>
Simply inverting these two method calls (Pager.__init__ and
FilePager.startProducing)
solves the problem.
The same problem does not appear when calling the 'dummy' method (client.py:19)
because the data for the method call is waiting to be sent out by the reactor
and
the transport does not call 'resumeProducing' on the broker until the next
iteration.
This allows the call to FilePager.startProducing to complete before
sendNextPage is
ever called on it.
Does this sound correct? It seems only strange to me that nobody else had this
problem
previously. If no errors on my side are found, I'll submit this as a ticket.
Cheers,
Jonathan
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
