Martin DeMello wrote:
I'm writing a cluster monitor, that collects information from a set of
machines and logs it to a database
In the interests of not hammering the db unnecessarily, I'm
considering the following
1. A series of independent "monitor" threads that collect information
over TCP from the cluster of machines, and write it to a queue
2. A "logger" thread that empties the queue every second or so and
inserts the collected information to the db via a single insert
statement
why are you using a queue for this case, btw? why not just use a plain list
L = []
lock = threading.Lock()
and add stuff using append in the monitor threads
with lock:
L.append(item)
and regularily reset the list in the logger thread
with lock:
data = L[:]
L[:] = [] # clear the list
for item in data:
... insert into database ...
(list append and assignments to global variables are atomic in CPython,
so you can eliminate the lock by being a bit clever, but that's probably
better left for a non-premature optimization pass).
</F>
--
http://mail.python.org/mailman/listinfo/python-list
