Frank Steinmetzger wrote: > Am Sat, Sep 14, 2024 at 02:46:35PM -0500 schrieb Dale: > >> I was running the command again and when I was checking on it, it >> stopped with this error. >> >> >> >> File "/root/dh", line 1209, in <module> >> main() >> File "/root/dh", line 1184, in main >> directory_hash(dir_path, '', dir_files, checksums) >> File "/root/dh", line 1007, in directory_hash >> os.path.basename(old_sums[filename][1]) >> ~~~~~~~~^^^^^^^^^^ >> KeyError: 'Some Video.mp4' > What was the exact command with which you ran it? > Apparently the directory has a file 'Some Video.mp4', which was not listed > in an existing checksum file.
I'm fairly sure it was this command. /root/dh -c -f -F 1Checksums.md5 -v I may have changed the -c to -u because I think it was the second run. I'd start with the thought it was -u if it were me. There's another command running right now and I cleared the scrollback part. Once it finishes, I can up arrow and be more sure. At the moment, I'm letting it test the files against the checksum it created, to be sure everything is good. It's almost half way through and no problems so far. I might add, I did a second run with -u, which I think produced the error above, and it seems to have missed some directories. When looking I noticed some directories didn't have a checksum file in it. That's when I ran it a second time. It skipped the ones it already did but found the ones that was missed in first run. There are almost 46,000 files in almost 800 directories. Is there some tool your script relies on that could make one of those numbers to high? > I also noticed a problem recently which happens if you give dh a directory > as argument which has no checksum file in it. Or something like it, I can’t > reproduce it from memory right now. I have been doing some refactoring > recently in order to get one-file-per-tree mode working. > >> I was doing a second run because I updated some files. So, it was >> skipping some and creating new for some new ones. This is the command I >> was running, which may not be the best way. >> >> >> /root/dh -c -f -F 1Checksums.md5 -v > Yeah, using the -c option will clobber any old checksums and re-read all > files fresh. If you only changed a few files, using the -u option will > drastically increase speed because only the changed files will be read. > Use the -d option to clean up dangling entries from checksum files. That was my thinking. When I update a set, I'll likely just cd to that directory and update, -u, that one directory instead of everything. That will save time and all. Doing everything takes days. LOL > >> Also, what is the best way to handle this type of situation. Let's say >> I have a set of videos. Later on I get a better set of videos, higher >> resolution or something. I copy those to a temporary directory then use >> your dmv script from a while back to replace the old files with the new >> files but with identical names. Thing is, file is different, sometimes >> a lot different. What is the best way to get it to update the checksums >> for the changed files? Is the command above correct? > dh has some smarts built-in. If you changed a file, then its modification > timestamp will get udpated. When dh runs in -u mode and it finds a file > whose timestamp is newer than its associated checksum file, that means the > file may have been altered since the creation of that checksum. So dh will > re-hash the file and replace the checksum in the checksum file. > Sounds good. I wasn't sure if it would see the change or not. >> I'm sometimes pretty good at finding software bugs. But hey, it just >> makes your software better. ;-) > Me too, usually. If it’s not my software, anyways. ^^ > But I think you may be the first other of that tool other than me. > One thing I've noticed, when I run this tool, my video sputters at times. It does fine when not running. This tool makes that set of hard drives pretty busy. It might be nice to add a ionice setting. I'd just set it inside the script. If a person wants, they can edit and change it. Just set it to a little lower than normal stuff should be fine. If I restart smplayer, I may set its ionice to a higher priority. Just a thought and likely easy enough to do. I don't want to stop your script given it is so far along. It sometimes pops up a question. I figured out that I type in the answer with the letter that is in parentheses. Could you explain the options a bit just to be sure I understand it correctly? So far, this is a nice tool. It should find corruption, like my bad memory stick, bit rot, bad drive or anything else that could corrupt a file. Even power failure I'd think. It takes a while to do the checksums but the script itself is fast. Once you really happy with this and feel like it is ready, you should really make a announcement that it is ready. Anyone who does a lot of write once and read many but are concerned with files becoming corrupt over time for any reason should be interested in this tool. It wouldn't work well for files that change a lot but there are tons of situations where once a file is generated, video just being one of them, then never changes after that. Given the number of files I have, how I change things, I should be able to find any bugs or needed features. Example, making my video sputter and needing a lower drive priority. You may not run into that but I noticed it here. Different use case. By the way. Still using that other script you threw together a good while back. I used it the other day to update some better videos I found. Thanks much for both tools. I wish this old dog could learn new tricks. ROFL I'm having trouble remembering old tricks. :/ Dale :-) :-)
