yoda wrote: > I'm considering moving to stackless python so that I can make use of > continuations so that I can open a massive number of connections to the > gateway and pump the messages out to each user simultaneously.(I'm > thinking of 1 connection per user).
This won't help if your gateway works synchronously. You need to determine what your gateway can do. If it works asynchronously, determine the max bandwidth it can handle, then determine how many messages you can fit into 4 seconds of that bandwidth. That should provide you with a number of connections you can safely open and still recieve acceptable response times. > My questions therefore are: > 1)Should I switch to stackless python or should I carry out experiments > with mutlithreading the application? You will build a more scalable solution if you create a multi process system. This will let you deploy across multiple servers, rather than CPU's. Multithreading and Multiprocessing will only help you if your application is IO bound. If your application is CPU bound, multiprocessing and multithreading will likely hurt your performance. You will have to build a parallel processing application which will work across different machines. This is easier than it sounds, as Python has a great selection of IPC mechanisms to choose from. > 2)What architectural suggestions can you give me? Multithreading will introduce extra complexity and overhead. I've always ended up regretting any use of multithreading which I have tried. Avoid it if you can. > 3)Has anyone encountered such a situation before? How did you deal with > it? Profile each section or stage of the operation. Find the bottlenecks, and reduce it whichever way you can. Check out ping times. Use gigabit or better. Remove as many switches and other hops between machines which talk to each other. Cache content, reuse it if you can. Pregenerate content, and stick it in a cache. Cache cache cache! :-) > 4)Lastly, and probably most controversial: Is python the right language > for this? I really don't want to switch to Lisp, Icon or Erlang as yet. Absolutely. Python will let you easily implement higher level algorithms to cope with larger problems. Sw. -- http://mail.python.org/mailman/listinfo/python-list
