After reading all the comments it appears that there may be a 'real'
problem with unaligned block sizes that DEDUP simply will not handle.
What you seem to be after, then, is the opposite of sparse files,
'virtual files' that can be chained together as a linked list of
_fragments_ of allocation blocks as well as full allocation blocks. This
could then be leveraged by a specialized concatenation driver (userland)
to avoid realigning the blocks and missing the 'opportunity' to DEDUP or
COW the existing blocks.
As always in computing, a specialized per-use-case driver will be able
to yields the best optimizations. However, there will be a balance point
since obviously the optimization based on leaving parts of allocation
blocks unused is not going to healthy for say:
cat s1 s2 s3 s4 .... s999 > all_s_files
rm s*
Where s1...s999 all use (much) less than a block size. All the 'gain' in
DEDUP is quickly offset by the enormous waste of block space after
deletion of the constituent files.
Per Baatrup wrote:
I would like to to concatenate N files into one big file taking advantage of
ZFS copy-on-write semantics so that the file concatenation is done without
actually copying any (large amount of) file content.
cat f1 f2 f3 f4 f5 > f15
Is this already possible when source and target are on the same ZFS filesystem?
Am looking into the ZFS source code to understand if there are sufficient (private)
interfaces to make a simple "zcat -o f15 f1 f2 f3 f4 f5" userland application
in C code. Does anybody have advice on this?
TIA
Per
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
