Michael wrote: > On Thursday 5 September 2024 19:55:56 BST Frank Steinmetzger wrote: >> Am Thu, Sep 05, 2024 at 06:30:54AM -0500 schrieb Dale: >>>> Use rsync with: >>>> --checksum >>>> >>>> and >>>> >>>> --dry-run >> I suggest calculating a checksum file from your active files. Then you don’t >> have to read the files over and over for each backup iteration you compare >> it against. >> >>>> You can also run find to identify which files were changed during the >>>> period you were running with the dodgy RAM. Thankfully you didn't run >>>> for too long before you spotted it. >> This. No need to check everything you ever stored. Just the most recent >> stuff, or at maximum, since you got the new PC. >> >>> I have just shy of 45,000 files in 780 directories or so. Almost 6,000 >>> in another. Some files are small, some are several GBs or so. Thing >>> is, backups go from a single parent directory if you will. Plus, I'd >>> want to compare them all anyway. Just to be sure. >> I aqcuired the habit of writing checksum files in all my media directories >> such as music albums, tv series and such, whenever I create one such >> directory. That way even years later I can still check whether the files are >> intact. I actually experienced broken music files from time to time (mostly >> on the MicroSD card in my tablet). So with checksum files, I can verify >> which file is bad and which (on another machine) is still good. > There is also dm-verity for a more involved solution. I think for Dale > something like this should work: > > find path-to-directory/ -type f | xargs md5sum > digest.log > > then to compare with a backup of the same directory you could run: > > md5sum -c digest.log | grep FAILED > > Someone more knowledgeable should be able to knock out some clever python > script to do the same at speed.
I'll be honest here, on two points. I'd really like to be able to do this but I have no idea where to or how to even start. My setup for series type videos. In a parent directory, where I'd like a tool to start, is about 600 directories. On a few occasions, there is another directory inside that one. That directory under the parent is the name of the series. Sometimes I have a sub directory that has temp files; new files I have yet to rename, considering replacing in the main series directory etc. I wouldn't mind having a file with a checksum for each video in the top directory, and even one in the sub directory. As a example. TV_Series/ ├── 77 Sunset Strip (1958) │ └── torrent ├── Adam-12 (1968) ├── Airwolf (1984) I got a part of the output of tree. The directory 'torrent' under 77 Sunset is temporary usually but sometimes a directory is there for videos about the making of a video, history of it or something. What I'd like, a program that would generate checksums for each file under say 77 Sunset and it could skip or include the directory under it. Might be best if I could switch it on or off. Obviously, I may not want to do this for my whole system. I'd like to be able to target directories. I have another large directory, lets say not a series but sometimes has remakes, that I'd also like to do. It is kinda set up like the above, parent directory with a directory underneath and on occasion one more under that. One thing I worry about is not just memory problems, drive failure but also just some random error or even bit rot. Some of these files are rarely changed or even touched. I'd like a way to detect problems and there may even be a software tool that does this with some setup, reminds me of Kbackup where you can select what to backup or leave out on a directory or even individual file level. While this could likely be done with a script of some kind, my scripting skills are minimum at best, I suspect there is software out there somewhere that can do this. I have no idea what or where it could be tho. Given my lack of scripting skills, I'd be afraid I'd do something bad and it delete files or something. O_O LOL I been watching videos again, those I was watching during the time the memory was bad. I've replaced three so far. I think I noticed this within a few hours. Then it took a little while for me to figure out the problem and shutdown to run the memtest. I doubt many files were affected unless it does something we don't know about. I do plan to try to use rsync checksum and dryrun when I get back up and running. Also, QB is finding a lot of its files are fine as well. It's still rechecking them. It's a lot of files. Right now, I suspect my backup copy is likely better than my main copy. Once I get the memory in and can really run some software, then I'll run rsync with those compare options and see what it says. I just got to remember to reverse things. Backup is the source not the destination. If this works, I may run that each time, help detect problems maybe. Maybe?? Oh, memory made it to the Memphis hub. Should be here tomorrow. Dale :-) :-)
