On 12/31/2014 01:18 PM, MRAB wrote:
On 2014-12-31 19:33, Charles Hixson wrote:
In order to allow multiple processes to access a database (currently
SQLite) I want to run the process in a separate thread. Because it will
be accessed from multiple processes I intent to use Queues for shifting
messages back and forth. But none of the individuals processes will
know when all of the others are through. It there a way to properly
close the database, or does it need to be done from the parent process?
What I want to do is detect that a request to shutdown the Process has
been received, and then execute a close procedure that cleans things up
and shutdown. Terminate is the obvious procedure to use, but that comes
with a warning not to use it if there is an queue attached (and without
defining attached).
OTOH, since the queue will never be read after the process has been
shutdown, perhaps rendering it unusable is the proper thing.
Could someone comment on this step of the design?
The basic idea is:
class handledb(Process):
def __init__(self, q, otherstuff):
self.q = q
def run(self, msg):
while (true):
if self.q.empty():
sleep(someamountoftime)
else:
while (not self.q.empty()):
read and parse message
if message says to shutdown: self.shutdown()
def shutdown(self):
close things down
?? terminate ?? <<-- should this be done
while q is live?
The parent process could tell the db process to shutdown (by putting a
message in the queue) when the child processes have terminated.
Alternatively, the parent process could tell the db process how many
child processes there are and then shutdown when it has received that
many notifications from child processes that they have finished.
BTW, it's better to let the queue read block until a message arrives
than to keep polling and sleeping if it's empty.
Thanks. This design already depends on the parent process telling the
db process when to shut down, but this depends on the queue (which is, I
think, attached to the process) and so I am uncertain about calling
terminate, as the docs say this can be expected to corrupt the queue.
I'm hoping that this isn't a problem if the db process is the only
reader of the queue, and this shouldn't happen until after all other
writers to the queue have finished. But the documentation has made me
uneasy on this point.
A blocking read is better? OK, that's an easy change, and seems quite
reasonable. But its the termination that I was worried about. (This is
only one use out of several similar cases, and I'd like to use the same
pattern throughout, so I'd really rather get it right the first time.
Particularly as MP programs won't necessarily throw the error predictably.)
--
https://mail.python.org/mailman/listinfo/python-list
