Take a look at ampoule: https://launchpad.net/ampoule
On Fri, Jan 2, 2009 at 2:35 PM, V S P <torea...@fastmail.fm> wrote: > Hi, > I am in the process of evaluating a tool for the newtorking architecture > I need. And wanted to ask the group questions, I could not easily get > answered from reading docs and various posts: > > Basically my architecutre requires the following > > a) multi-CPU/multi-core scalability > > b) multi-machine horizontal scalability > > c) Event dispatching: having a single 'dispatcher' instance > being able to read rows (events) from postgresql database and > being able to dispatch it to available 'worker' processes > > d) chat server > > > For c) and d) am looking for a framework such that it will allow me > to develop the 'worker' processes such that they reside on mulitple > machines, and a lot of the 'hard work' of registration, message passing, > restarting, SNMP compatibilty . > > Twisted appears to already have message passing, chat server and many > many other neat things that if I do not need now -- probably will need > in the future. > > However, I cannot quite understand how the 'multi-process' part is > supported. > > I have read this: > http://www.python.org/workshops/2002-02/papers/09/index.htm > (found it via Bruce Eckel's log) > and it says that: > "... > and since forking Python processes has many disadvantages, like Python's > reference counting not playing well with copy-on-write and problems with > shared state, it was felt the best option was an event-driven framework. > " > > I also looked at the docs for the > http://twistedmatrix.com/projects/core/documentation/howto/process.html > > > But I got confused there > > > Basically I am looking to 'pre-spawn' a number of worker threads > configured from command line (and each of the worker threads will > precreate a database connection). So each multi-cpu machine will > have one twisted Server, each server will pre-spawn N worker processes > and each process will have its own database connection (and therefore > each can do its own caching and transaction control (or use memchaced > server) to share cache). > > > I did not see however how to: > a) make the Deferred mechanism to pass the event data received > in the Asynch loop to one of the worker processes > (there are appeared to be no 'inheritance' structure to where derive > the worker processes from). > > > b) I did not see how multiple twisted servers, each running on a > separate > multi-cpu machine can register together to be in one 'cluster' -- so > that > my dispatcher process that reads the events from the database can > 'round-robin' the events to them. > > c) I did not see if twisted 'figures out' that given process runs on a > local machine vs remote and optimises the IPC communication for local > IPC. > > > Because I did not see the above -- it lead me to believe that I am > trying to ask Twisted have something it was not meant to do > (My view of the architecture is somewhat similar to how would ACE/TAO > ORB notification service would work -- as I am familiar with those). > > > I kind of did not go into the multi-box architecutre of how the > Twisted chart server is working, because I thought I need to understand > the above first. > > So wanted to ask you guys if I am looking for the right documentation > or may be there is a separate sub-project that is doing what I am > looking for. > > thanks in advance, > Vlad > > > > > -- > V S P > torea...@fastmail.fm > > -- > http://www.fastmail.fm - Or how I learned to stop worrying and > love email again > > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > -- 01100011 01100001 01110010 01111001 _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
