> I am writing a multi-user business/accounting application. It is getting > rather complex and I am looking at how to, not exactly simplify it, but > find a way to manage the complexity. > > I have realised that it is logically made up of a number of services - > database service with connection to database > workflow engine for business processes > services manager to handle automated services, such as web services > client manager to service logins from client workstations > possibly others > > I have made a start by splitting some of these off into separate modules > and running them in their own threads.
I wouldn't do that. Creating threads (or processes) with potentially interacting components ramps up complexity a great deal, with little if any benefit at your current stage, and only a vague possibility that scaling issues appear and can be remedied by that. Instead, use threading or multi-processing to create various instances of your application that synchronize only over the DB, using locks where it is needed. Introducing RPC of whatever kind to your design will make you lose a lot of the power and flexibility code-wise, because all of a sudden you can only pass data, not behavior around. And as J Kenneth already said, deal with performance issues when the crop up. At work, we had a design with a whole bunch of separated XMLRPC-connected services, all of them restricting their access to only certain sub-schemas of the DB. This was done so that we could run them on separate servers if we wanted, with several databases. The creators of that system had the same fears as you. Guess what? Most of the time the system spend in serializing and de-serializing XML for making RPC-calls. We had no referential integrity between schemas, and no single transactions around HTTP-requests, which didn't exactly make crap out of our data, but the occasional hickup was in there. And through the limited RCP-interface, a great deal of code consisted of passing around dicts, lists and strings - with no rich OO-interface of whatever kind. Once we got rid of these premature optimizations, the system improved in performance, and the code-base was open to a *lot* of cleaning up that is still under way, but already massively improved the design. Diez -- http://mail.python.org/mailman/listinfo/python-list
