Hi Bruce, you can do it like Maildir [1] you move (os.rename()) file or directories.
Maybe something like this: You have three directories: "todo", "in-process" and "done". A process tries to os.rename from todo to in-process. If it fails, some other process has done it before. If the process is done it moves the file/directory to "done". To avoid stressing the directories, too much, It might be good to use subdirectories like todo/NN/MM/. I think git (version control system created by Linus Torvalds) does something like this. Thomas [1] http://wiki.dovecot.org/MailboxFormat/Maildir This page describes Maildir and some unneeded parts of the specification. bruce schrieb: > Hi. > > Got a bit of a question/issue that I'm trying to resolve. I'm asking this of > a few groups so bear with me. > > I'm considering a situation where I have multiple processes running, and > each process is going to access a number of files in a dir. Each process > accesses a unique group of files, and then writes the group of files to > another dir. I can easily handle this by using a form of locking, where I > have the processes lock/read a file and only access the group of files in > the dir based on the open/free status of the lockfile. > > However, the issue with the approach is that it's somewhat synchronous. I'm > looking for something that might be more asynchronous/parallel, in that I'd > like to have multiple processes each access a unique group of files from the > given dir as fast as possible. > > So.. Any thoughts/pointers/comments would be greatly appreciated. Any > pointers to academic research, etc.. would be useful. > > thanks > > > -- Thomas Guettler, http://www.thomas-guettler.de/ E-Mail: guettli (*) thomas-guettler + de -- http://mail.python.org/mailman/listinfo/python-list
