It would be really cool if an rpc call could return a generator. I know
that there are a few reasons why one would not expect it to be part of
the library (e.g. the client would need to receive asynchronous
messages, and it's not part of the rpc standard), however below I show a
possible implementation, given that the client is also able to be a
server...
I am aware of MultiCall, but that is actually something like the inverse
of what I want. I need a consumer.
Example:
------------------------------------------------------------------------
# hypothetical client code (running at "http://rpc.myserver.com";)
from fancyxmlrpc import FancyXMLRPCServer
server = FancyXMLRPCServer(('localhost', 9000))
def primes():
n = 2
p = []
while True:
if not any( n % f == 0 for f in p ):
yield n
p.append( n )
n += 1
server.register_generator(primes)
server.serve_forever()
------------------------------------------------------------------------
# hypothetical client code (running at http://www.mywebsite.com):
from fancyxmlrpc import FancyServerProxy
server_url = "http://rpc.myserver.com";
local_url = "http://www.mywebsite.com";
server = FancyServerProxy(server_url, local_url)
g = server.examples.getStateNames()
for x in g:
print x
------------------------------------------------------------------------
Okay, so to implement this, the trick would be to implement the
generator wrapper as a hidden xmlrpc conversation. On the client side,
FancyServerProxy would encapsulate a hidden RPC server with a function
that receives each item that is yielded by the server and queues them
while the client generator consumes the queue and yields the items.
The practical constraints of my specific application are:
1. The rpc server is a highly specialized slave system that does heavy
duty work.
2. The rpc client is itself a web server that dispatches work requests
to the rpc server(s) and displays the current status of work done so far.
3. The generators will typically run for a long time (hours) and yield
data periodically (perhaps once a minute).
4. Trusted users will write generators on the server and consumers on
the client (web site) and use the web site to make requests.
5. It would be even better if my generator yields numpy arrays rather
than lists.
6. It would be nice to be able to scale to allow the web site to
dispatch to multiple work servers.
So my questions are:
1. Does using xmlrpc make any sense for this?
2. I am missing an easier way to do this?
3. Any good examples of things like this?
--
http://mail.python.org/mailman/listinfo/python-list
