On 08/01/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
I think that in addition to lzjb compression, squishing blocks that contain the same data would buy a lot of space for administrators working in many common workflows.
This idea has occurred to me too - I think there are definite advantages to 'block re-use'. When you start talking about multiple similar zones, I suspect substantial space savings could be made - and if you can re-use that saved storage to provide additional redundancy, everyone would be happy.
Assumptions: SHA256 hash used (Fletcher2/4 have too many collisions, SHA256 is 2^128 if I remember correctly) SHA256 hash is taken on the data portion of the block as it exists on disk. the metadata structure is hashed separately. In the current metadata structure, there is a reserved bit portion to be used in the future. Description of change: Creates: The filesystem goes through its normal process of writing a block, and creating the checksum. Before the step where the metadata tree is pushed, the checksum is checked against a global checksum tree to see if there is any match. If match exists, insert a metadata placeholder for the block, that references the already existing block on disk, increment a number_of_links pointer on the metadata blocks to keep track of the pointers pointing to this block. free up the new block that was written and check-summed to be used in the future. else if no match, update the checksum tree with the new checksum and continue as normal.
Unless I'm reading this wrong, this sounds a lot like Plan9s 'Venti' architecture ( http://cm.bell-labs.com/sys/doc/venti.html ) . But using a hash 'label' seems the wrong approach. ZFS is supposed to scale to terrifying levels, and the chances of a collision, however small, works against that. I wouldn't want to trade reliability for some extra space. -- Rasputin :: Jack of All Trades - Master of Nuns http://number9.hellooperator.net/ _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
