Re: checksums to detect/correct bit-rot

2024-09-17 Thread Jonathan Thornburg
Thank you to everyone who replied -- you've pointed out some useful design approaches and software. To summarize: * OpenBSD doesn't (currently) support any filesystems with embedded checksums * mtree (8) can generate a database of files, modify-times, and checksums (and both the man page and m

Re: checksums to detect/correct bit-rot

2024-09-15 Thread Rubén Llorente
Jonathan Thornburg wrote: And a related question: I have a pool of ~10 external USB3 backup disks (all consumer-grade WD or Seagate 2.5" spinning rust, either 2TB or 4TB capacity each), all currently setup with FFS2 filesystems on top of softraid crypto (/bioctl -c C/). Each backup is to a si

Re: checksums to detect/correct bit-rot

2024-09-15 Thread lumidify
Hi, On Sun, Sep 15, 2024 at 12:12:08AM -0700, Jonathan Thornburg wrote: > Thinking about how to detect/correct bit-rot in these backups, it > occurs to me that I could hack up some Perl to walk the filesystem > tree on a mounted backup disk, /stat()/ and read each file, and build > a database of (

Re: checksums to detect/correct bit-rot

2024-09-15 Thread Stuart Henderson
On 2024-09-15, Jonathan Thornburg wrote: > Does OpenBSD support any file systems with built-in checksums to > (try to) ensure metadata and/or data integrity in the face of "bit rot" > disk (or memory/cpu/USB) errors? No. On 2024-09-15, Kirill A Korinsky wrote: > On Sun, 15 Sep 2024 09:12:08

Re: checksums to detect/correct bit-rot

2024-09-15 Thread Kirill A . Korinsky
On Sun, 15 Sep 2024 09:12:08 +0200, Jonathan Thornburg wrote: > > But before I reinvent the wheel, can anyone point me to software > which already does this? Bonus points if the software is already > in ports. > yabitrot? -- wbr, Kirill

Re: checksums to detect/correct bit-rot

2024-09-15 Thread Nick Owens
On Sun, Sep 15, 2024 at 12:22 AM Jonathan Thornburg wrote: > > Does OpenBSD support any file systems with built-in checksums to > (try to) ensure metadata and/or data integrity in the face of "bit rot" > disk (or memory/cpu/USB) errors? I'm not looking for ZFS-style storage > pools or logical vol

checksums to detect/correct bit-rot

2024-09-15 Thread Jonathan Thornburg
Does OpenBSD support any file systems with built-in checksums to (try to) ensure metadata and/or data integrity in the face of "bit rot" disk (or memory/cpu/USB) errors? I'm not looking for ZFS-style storage pools or logical volume management, "just" checksums to catch silent metadata and/or data