Agreed, Chris. There are many methods way better than the sort of RAID architecture I supplied as AN EXAMPLE easy to understand. But even so, if a hard disk or memory chip is fried or a nuclear bomb takes out all servers in or near a city, you would need some truly crazy architectures with info not only distributed across the globe but perhaps also to various space satellites or servers kept ever further out and eventually in hyperspace or within a black hole (might be write-only, alas).
The point many of us keep saying is there can not easily or even with great difficult, any perfect scheme that guarantees nothing will go wrong with the software, hardware, the people using it and so on. And in the real world, as compared to the reel world, many programs cannot remain static. Picture a program that includes many tax laws and implementations that has to be changed at least yearly as laws change. Some near-perfect code now has to either be patched with lots of errors possible, or redesigned from scratch and if it takes long enough, will come out after yet more changes and thus be wrong. A decent question you can ask is if the language this forum is supposed to be on, is better in some ways to provide the kind of Teflon-coated code he wants. Are there features better avoided? How do you make sure updates to modules you use and trust are managed as they may break your code. Stuff like that is not as abstract. In my view, one consideration can be that when people can examine your source code in the original language, that can open up ways others might find ways to break it, more so than a compiled program that you only can read in a more opaque way. -----Original Message----- From: Python-list <python-list-bounces+avi.e.gross=gmail....@python.org> On Behalf Of Chris Angelico via Python-list Sent: Tuesday, October 24, 2023 9:41 PM To: python-list@python.org Subject: Re: Question(s) On Wed, 25 Oct 2023 at 12:20, AVI GROSS via Python-list <python-list@python.org> wrote: > Consider an example of bit rot. I mean what if your CPU or hard disk has a location where you can write a byte and read it back multiple times and sometimes get the wrong result. To be really cautions, you might need your software to write something in multiple locations and when it reads it back in, check all of them and if most agree, ignore the one or two that don't while blocking that memory area off and moving your data elsewhere. Or consider a memory leak that happens rarely but if a program runs for years or decades, may end up causing an unanticipated error. > True, but there are FAR more efficient ways to do error correction :) Hamming codes give you single-bit correction and two-bit detection at a cost of log N bits, which is incredibly cheap - even if you were to go for a granularity of 64 bytes (one cache line in a modern Intel CPU), you would need just 11 bits of Hamming code for every 512 bits of data and you can guarantee to fix any single-bit error in any cache line. The "if most agree, ignore the one or two that don't" design implies that you're writing to an absolute minimum of three places, and in order to be able to ignore two that disagree, you'd probably need five copies of everything - that is to say, to store 512 bits of data, you would need 2560 bits of storage. But with a Hamming code, you need just 523 bits to store 512 reliably. Here's a great run-down on how efficiently this can be done, and how easily. https://www.youtube.com/watch?v=X8jsijhllIA Side note: If we assume that random bit flips occur at a rate of one every X storage bits, having redundant copies of data will increase the chances of a failure happening. For example, using a naive and horrendously wasteful "store 256 copies of everything" strategy, you would be 256 times more likely to have a random bitflip, which is insane :) You would also be able to guarantee detection of up to 128 random bitflips. But as you can see, this puts a maximum on your storage ratio. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
