On 1/8/07 11:43 AM, Brian Candler wrote: ..
(1) You won't see any benefit until *all* applications have been rewritten to use these new semantics instead of traditional ones. That means new versions of oracle, mysql etc.
Yes and no, the database filesystem should have an own SQL layer. It's "nuts" to let have Oracle or MySQL own caches of data on your disk.
If you want a secure data server that's easy to backup and mirror all data should be synced by the OS not by applications.
OpenBSD has now a strong focus on securing streams, connections. With a database filesystem that should also apply for storage
(2) The filesystem itself will be able to report new types of errors, such as conflicting transactions, which applications will have to be able to handle properly.
All data should be stored in one basic layer, not multiple.
(3) Depending on what level of transaction isolation you select, performance of concurrent applications may be much worse.
Exactly, if you want a transaction system with data secured over 3 servers on remote locations then you get a very slow server because everything has to be written to those 3 disks that are remote from each other. So this new database filesystem needs parameters to define what kind of quality you want. This is why I mentioned "laws of physics" in the first place. If you want a faster system the remote copies have more or less time-lag but the copies "know" this, they do have a stable state.
(4) The filesystem will still need to store this data as on-disk blocks, and at snapshot time you'll still need to ensure that only whole transactions are backed up. So arguably you're just pushing the problem down a layer - although I admit it would be easier to have a single backup operation at the filesystem layer (quiesce; snapshot; release) than to have to do this for each of the running applications.
So this is why I try to promote storage of data in 1 layer and not multiple. Storage in multiple layers without proper organisation is insecure by design.
Leaving applications such as databases to manage their own ACID requirements does at least let them be tuned to their own specific needs, and lets them run on a much wider range of platforms.
The basic database filesystem could be open so others might adapt it as a storage layer too.
See it as FAT or NTFS or or or, but more "intelligent".
Maybe there's a partial solution, where you have a journalling layer on the filesystem, and applications can opt to be aware of it (e.g. requesting a series of writes as being part of the same journal-level transaction). But this still requires changes to the applications, and I can see lots of potential pitfalls.
Data storage at multiple layers is insecure by itself. +++chefren
