On 25 Gen, 21:11, Steve Holden <st...@holdenweb.com> wrote: > pyt...@bdurham.com wrote: > > Any suggestions on a best practice way to monitor a remote FTP site for > > the arrival of new/updated files? I don't need specific code, just some > > coaching on technique based on your real-world experience including > > suggestions for a utility vs. code based solution. > > > My goal is to maintain a local collection of files synced with a remote > > FTP site and when I download a new/updated file locally, run a script to > > process it. The arrival and format of the files that I need to sync with > > are beyond my control (eliminating a rsync solution) ... all I have is a > > generic FTP connection to a specific FTP address. Note: The remote site > > I'm monitoring may have multiple uploads occuring at the same time. > > > My basic strategy is to poll the remote directory on a regular basis and > > compare the new directory listing to a previous snapshot of the > > directory listing. If a file timestamp or size has changed (or a new > > file has appeared), then track this file as a changed file. Once a file > > has been marked as changed, wait <N> polling cycles for the file > > timestamp and size to remain stable, then download it, and trigger a > > local script to process the file. In addition to detecting new or > > changed files, I would compare remote directory listings to my local > > sync folder and delete local files that no longer exist on the remote site. > > > My concern about using a utility is the utility's ability to detect when > > a remote file has finished being updated. I don't want to download files > > that are still in the process of being updated - I only want to download > > new/updated files after they've been closed on the remote site. > > > Any ideas appreciated! > > Well, the ftpmirror will cope with most of what you want to do as it is, > but I am unsure how you can determine whether a file is in the process > of being written on the server. > > regards > Steve > -- > Steve Holden +1 571 484 6266 +1 800 494 3119 > Holden Web LLC http://www.holdenweb.com/- Nascondi testo citato > > - Mostra testo citato -
If you're going to do that manually through a script, and the server supports it, use MLSD instead of LIST command for listing files and determine their last modification time. My 2 cents --- Giampaolo http://code.google.com/p/pyftpdlib -- http://mail.python.org/mailman/listinfo/python-list
